Skip to content

Comments

feat(profiling): builds libdatadog for windows#12073

Merged
taegyunkim merged 58 commits intomainfrom
taegyunkim/prof-11194-windows-libdd
Apr 1, 2025
Merged

feat(profiling): builds libdatadog for windows#12073
taegyunkim merged 58 commits intomainfrom
taegyunkim/prof-11194-windows-libdd

Conversation

@taegyunkim
Copy link
Contributor

@taegyunkim taegyunkim commented Jan 24, 2025

A necessary step to deprecate/remove python exporter.

Tested manually on windows using the following code, both from compiling from sources and using built-wheel from this branch

import os
import time
from threading import Lock
from threading import Thread


def throw_exception():
    val = os.urandom(1)[0]
    # Throws an exception if a d20 rolls 1
    if val < 4:
        raise Exception("Oof owie ouch my bones")
    elif val < 8:
        return 1 / 0


def factorize_number(number):
    factors = []
    for i in range(1, number + 1):
        if number % i == 0:
            factors.append(i)
    return factors


def thread_task(lock, number):
    for _ in range(1, 50):
        # throw_exception()
        with lock:
            factors = factorize_number(number)
        print(factors)


if __name__ == "__main__":
    shared_lock = Lock()
    numbers = [100001, 499999, 999983]

    while True:
        T = [Thread(target=thread_task, args=(shared_lock, n)) for n in numbers]
        [t.start() for t in T]
        [t.join() for t in T]
Screenshot 2025-03-31 at 3 53 12 PM Screenshot 2025-03-31 at 3 53 23 PM

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

@github-actions
Copy link
Contributor

github-actions bot commented Jan 24, 2025

CODEOWNERS have been resolved as:

releasenotes/notes/profiling-libdd-windows-0e80f0facac09221.yaml        @DataDog/apm-python
ddtrace/internal/datadog/profiling/cmake/AnalysisFunc.cmake             @DataDog/profiling-python
ddtrace/internal/datadog/profiling/cmake/FindLibdatadog.cmake           @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/CMakeLists.txt            @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/src/ddup_interface.cpp    @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/src/profile.cpp           @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/src/sample.cpp            @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/src/uploader.cpp          @DataDog/profiling-python
ddtrace/internal/datadog/profiling/ddup/CMakeLists.txt                  @DataDog/profiling-python
setup.py                                                                @DataDog/python-guild
tests/smoke_test.py                                                     @DataDog/apm-core-python

@pr-commenter
Copy link

pr-commenter bot commented Jan 24, 2025

Benchmarks

Benchmark execution time: 2025-04-01 14:54:15

Comparing candidate commit 194ea56 in PR branch taegyunkim/prof-11194-windows-libdd with baseline commit 7efd41a in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 498 metrics, 2 unstable metrics.

@taegyunkim taegyunkim changed the title windows builds ddup feat(profiling): builds libdatadog for windows Jan 28, 2025
@taegyunkim taegyunkim force-pushed the taegyunkim/prof-11194-windows-libdd branch from 719f379 to 7804dab Compare January 31, 2025 15:09
@taegyunkim taegyunkim changed the base branch from main to 3.x-staging January 31, 2025 15:10
@datadog-dd-trace-py-rkomorn
Copy link

datadog-dd-trace-py-rkomorn bot commented Jan 31, 2025

Datadog Report

Branch report: taegyunkim/prof-11194-windows-libdd
Commit report: 96fe424
Test service: dd-trace-py

✅ 0 Failed, 475 Passed, 839 Skipped, 10m 2.3s Total duration (18m 59.71s time saved)

@taegyunkim taegyunkim marked this pull request as ready for review March 24, 2025 14:54
@taegyunkim taegyunkim requested review from a team as code owners March 24, 2025 14:54
@taegyunkim taegyunkim requested review from juanjux and tabgok March 24, 2025 14:54
Copy link

@gleocadie gleocadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@taegyunkim taegyunkim requested a review from a team as a code owner March 25, 2025 17:09
Copy link
Collaborator

@erikayasuda erikayasuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just reviewed the smoke test file to unblock the merge. LGTM 👍

@taegyunkim taegyunkim merged commit 8a4c296 into main Apr 1, 2025
700 checks passed
@taegyunkim taegyunkim deleted the taegyunkim/prof-11194-windows-libdd branch April 1, 2025 20:33
chojomok pushed a commit that referenced this pull request Apr 7, 2025
A necessary step to deprecate/remove python exporter. 

Tested manually on windows using the following code, both from compiling
from sources and using built-wheel from this branch

```Python
import os
import time
from threading import Lock
from threading import Thread


def throw_exception():
    val = os.urandom(1)[0]
    # Throws an exception if a d20 rolls 1
    if val < 4:
        raise Exception("Oof owie ouch my bones")
    elif val < 8:
        return 1 / 0


def factorize_number(number):
    factors = []
    for i in range(1, number + 1):
        if number % i == 0:
            factors.append(i)
    return factors


def thread_task(lock, number):
    for _ in range(1, 50):
        # throw_exception()
        with lock:
            factors = factorize_number(number)
        print(factors)


if __name__ == "__main__":
    shared_lock = Lock()
    numbers = [100001, 499999, 999983]

    while True:
        T = [Thread(target=thread_task, args=(shared_lock, n)) for n in numbers]
        [t.start() for t in T]
        [t.join() for t in T]
```

<img width="1201" alt="Screenshot 2025-03-31 at 3 53 12 PM"
src="https://github.com/user-attachments/assets/461c278f-d18a-41c2-a459-9d636accb0ff"
/>
<img width="1332" alt="Screenshot 2025-03-31 at 3 53 23 PM"
src="https://github.com/user-attachments/assets/07e18297-7456-464d-9893-457198a25831"
/>


## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
taegyunkim added a commit that referenced this pull request Jul 25, 2025
It has come to my attention that supporting windows takes quite an
effort

- #12073: Started on Jan 31, but finally merged on April 1. To iterate
on this PR, I had to spin up a windows vm, and iterate there. Still, it
was not as smooth as my devex on linux workspace or on corp laptop.
- #13984: This PR changes the way dd-trace-py depends on libdatadog, and
even without windows support for profiling it's already more than 50
commits and taking 2 engineers' time. And this PR is not the first
attempt, previous attempts are #13978, #13603.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Profiling Continous Profling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants