Skip to content

Commit 76b7678

Browse files
committed
Remove viztracer support
Never seemed that useful compared to py-spy. Could have kept it around, but nice to not have to worry about maintaining it.
1 parent 1cb81c9 commit 76b7678

File tree

6 files changed

+8
-522
lines changed

6 files changed

+8
-522
lines changed

README.md

Lines changed: 5 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
# dask-profiling
22

3-
Profile dask [distributed](https://github.com/dask/distributed) clusters with [py-spy](https://github.com/benfred/py-spy) or [viztracer](https://github.com/gaogaotiantian/viztracer).
3+
Profile dask [distributed](https://github.com/dask/distributed) clusters with [py-spy](https://github.com/benfred/py-spy).
44

55
```python
66
import dask
77
import distributed
88

9-
from dask_profiling import pyspy_on_scheduler, viztrace_scheduler
9+
from dask_profiling import pyspy_on_scheduler
1010

1111
client = distributed.Client()
1212

@@ -20,19 +20,15 @@ df = dask.datasets.timeseries(
2020
with (
2121
pyspy_on_scheduler("pyspy.json"),
2222
# ^ Saves a speedscope profile to `pyspy.json` locally
23-
viztrace_scheduler(
24-
"viztracer.json", trace_sparse="distributed.Scheduler.update_graph_hlg"
25-
),
26-
# ^ Saves a Chrome trace to `viztracer.json` locally
2723
):
2824
df.set_index("id").mean().compute()
2925
```
3026

31-
Using `pyspy_on_scheduler` or `viztrace_scheduler` attaches a profiler to the scheduler process, records a profile, and sends the file back to the client.
27+
Using `pyspy_on_scheduler` attaches a profiler to the scheduler process, records a profile, and sends the file back to the client.
3228

33-
By default, py-spy profiles are recorded in [speedscope](https://www.speedscope.app/) format. [Viztracer profiles](https://viztracer.readthedocs.io/en/latest/basic_usage.html#display-report) are recorded in Chrome trace format, viewable with [Perfetto](https://ui.perfetto.dev/) or <chrome://tracing>.
29+
By default, py-spy profiles are recorded in [speedscope](https://www.speedscope.app/) format.
3430

35-
`dask-profiling` (and, transitively, `py-spy`/`viztracer`) must be installed in the environment where the scheduler is running.
31+
`dask-profiling` (and, transitively, `py-spy`) must be installed in the environment where the scheduler is running.
3632

3733
## Installation
3834

@@ -56,19 +52,6 @@ In a Docker container, `dask-profiling` will "just work" for Docker/moby version
5652

5753
On Ubuntu-based containers, ptrace system calls are [further blocked](https://www.kernel.org/doc/Documentation/admin-guide/LSM/Yama.rst): processes are prohibited from ptracing each other even within the same UID. To work around this, `dask-profiling` automatically uses [`prctl(2)`](https://man7.org/linux/man-pages/man2/prctl.2.html) to mark the scheduler process as ptrace-able by itself and any child processes, then launches py-spy as a child process.
5854

59-
## Viztracer targeting with `trace_sparse`
60-
61-
`viztrace_scheduler` adds a non-standard option to VizTracer: the `trace_sparse` parameter. With this, you basically give a passlist of names (as strings) of functions to trace. The tracer will only run while those functions are active; at all other times, it'll be paused. Names are given in the same way you would to `unittest.mock.patch`.
62-
63-
`viztrace_scheduler` adds some magic so that if the function you want to trace happens to be registered as a comm handler on the Scheduler, everything will behave as you'd expect. (Since the handler functions are added into `scheduler.handlers` _before_ `viztrace_scheduler` has the chance to patch them, those dicts need to be explicitly updated to hold the patched handlers as well. `viztrace_scheduler` takes care of this for you.)
64-
65-
Examples:
66-
67-
* `trace_sparse="distributed.Scheduler.handle_task_finished"` - Trace only the `handle_task_finished` function, and everything it calls while running
68-
* `trace_sparse=["dask.highlevelgraph.HighLevelGraph.__dask_distributed_unpack__", "distributed.Scheduler.handle_task_finished"]` - Trace both high-level-graph unpacking and `handle_task_finished`
69-
70-
Note that this doesn't work very well on async functions. `distributed.comm.tcp.TCP.read` doesn't work that well, for example.
71-
7255
## Development
7356

7457
Install [Poetry](https://python-poetry.org/docs/#installation). To create a virtual environment, install dev dependencies, and install the package for local development:
@@ -83,5 +66,3 @@ There is one very very basic end-to-end test for py-spy. Running it requires Doc
8366
$ pytest tests
8467
```
8568
and wait a long time for the image to build and run.
86-
87-
There are currently no tests for viztracer.

dask_profiling/__init__.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
11
from .pyspy import pyspy, pyspy_on_scheduler
2-
from .viztracer import viztrace_scheduler
32

4-
__all__ = ["pyspy", "pyspy_on_scheduler", "viztrace_scheduler"]
3+
__all__ = ["pyspy", "pyspy_on_scheduler"]

dask_profiling/trace_sparse.py

Lines changed: 0 additions & 107 deletions
This file was deleted.

0 commit comments

Comments
 (0)