Rename to dask-profiling

Author: gjoseph92

Dockerfile

@@ -5,12 +5,15 @@ RUN apt install -y python3-pip
 RUN pip3 install --upgrade pip poetry
 # ^ pip: https://github.com/python-poetry/poetry/issues/5104
 
-WORKDIR /opt/src/scheduler-profilers
+WORKDIR /opt/src/dask-profiling
 
 COPY poetry.lock .
 COPY pyproject.toml .
 
-RUN poetry config virtualenvs.create false && poetry install --only main,docker
+RUN poetry config virtualenvs.in-project true && \
+    poetry install --only main,docker
 
-COPY scheduler_profilers /opt/src/scheduler-profilers/scheduler_profilers
+ENV PATH="/opt/src/dask-profiling/.venv/bin:$PATH"
+
+COPY dask_profiling /opt/src/dask-profiling/dask_profiling
 RUN pip3 install --no-deps .

README.md

@@ -1,12 +1,12 @@
-# scheduler-profilers
+# dask-profiling
 
-Profile the dask [distributed](https://github.com/dask/distributed) scheduler with [py-spy](https://github.com/benfred/py-spy) or [viztracer](https://github.com/gaogaotiantian/viztracer).
+Profile dask [distributed](https://github.com/dask/distributed) clusters with [py-spy](https://github.com/benfred/py-spy) or [viztracer](https://github.com/gaogaotiantian/viztracer).
 
 ```python
 import dask
 import distributed
 
-from scheduler_profilers import pyspy_on_scheduler, viztrace_scheduler
+from dask_profiling import pyspy_on_scheduler, viztrace_scheduler
 
 client = distributed.Client()
 
@@ -32,25 +32,29 @@ Using `pyspy_on_scheduler` or `viztrace_scheduler` attaches a profiler to the sc
 
 By default, py-spy profiles are recorded in [speedscope](https://www.speedscope.app/) format. [Viztracer profiles](https://viztracer.readthedocs.io/en/latest/basic_usage.html#display-report) are recorded in Chrome trace format, viewable with [Perfetto](https://ui.perfetto.dev/) or <chrome://tracing>.
 
-`scheduler-profilers` (and, transitively, `py-spy`/`viztracer`) must be installed in the environment where the scheduler is running.
+`dask-profiling` (and, transitively, `py-spy`/`viztracer`) must be installed in the environment where the scheduler is running.
 
 ## Installation
 
 ```
-python -m pip install git+https://github.com/gjoseph92/scheduler-profilers.git@main
+python -m pip install git+https://github.com/gjoseph92/dask-profiling.git@main
 ```
 
 ## Privileges for py-spy
 
-You may need to run the scheduler process as root for py-spy to be able to profile it (especially on macOS). See https://github.com/benfred/py-spy#when-do-you-need-to-run-as-sudo.
+**tl;dr:**
+* On macOS, you have to launch your cluster with `sudo`
+* For `docker run`, pass `--cap-add SYS_PTRACE`, or download this newer [`seccomp.json`](https://github.com/moby/moby/blob/d39b075302c27f77b2de413697a5aacb034d8286/profiles/seccomp/default.json) file and use `--seccomp=default.json`.
 
-In a Docker container, `scheduler-profilers` will "just work" for Docker/moby versions >= 21.xx. As of right now (May 2021), Docker 21.xx doesn't exist yet, so read on.
+You may need to run the dask process as root for py-spy to be able to profile it (especially on macOS). See https://github.com/benfred/py-spy#when-do-you-need-to-run-as-sudo.
 
-[moby/moby#42083](https://github.com/moby/moby/pull/42083/files) recently allowlisted the `process_vm_readv` system call that py-spy uses, which used to be blocked unless you set `--cap-add SYS_PTRACE`. This has been reasonable/safe to do for a while, but just wasn't enabled. So your options right now are:
+In a Docker container, `dask-profiling` will "just work" for Docker/moby versions >= 21.xx. As of right now (Nov 2022), Docker 21.xx doesn't exist yet, so read on.
+
+[moby/moby#42083](https://github.com/moby/moby/pull/42083/files) recently allowlisted the `process_vm_readv` system call that py-spy uses, which used to be blocked unless you set `--cap-add SYS_PTRACE`. Allowing this specific system call in unprivileged containers has been safe to do for a while (since linux kernel versions > 4.8), but just wasn't enabled in Docker. So your options right now are:
 * (low/no security impact) Download the newer [`seccomp.json`](https://github.com/moby/moby/blob/d39b075302c27f77b2de413697a5aacb034d8286/profiles/seccomp/default.json) file from moby/master and pass it to Docker via `--seccomp=default.json`.
 * (more convenient) Pass `--cap-add SYS_PTRACE` to Docker. This enables more than you need, but it's one less step.
 
-On Ubuntu-based containers, ptrace system calls are [further blocked](https://www.kernel.org/doc/Documentation/admin-guide/LSM/Yama.rst): processes are prohibited from ptracing each other even within the same UID. To work around this, `scheduler-profilers` automatically uses [`prctl(2)`](https://man7.org/linux/man-pages/man2/prctl.2.html) to mark the scheduler process as ptrace-able by itself and any child processes, then launches py-spy as a child process.
+On Ubuntu-based containers, ptrace system calls are [further blocked](https://www.kernel.org/doc/Documentation/admin-guide/LSM/Yama.rst): processes are prohibited from ptracing each other even within the same UID. To work around this, `dask-profiling` automatically uses [`prctl(2)`](https://man7.org/linux/man-pages/man2/prctl.2.html) to mark the scheduler process as ptrace-able by itself and any child processes, then launches py-spy as a child process.
 
 ## Viztracer targeting with `trace_sparse`
 

scheduler_profilers/__init__.py dask_profiling/__init__.py

@@ -1,7 +1,7 @@
 services:
   scheduler:
     build: .
-    command: dask-scheduler
+    command: dask scheduler
     ports:
       - "8786:8786"
       - "8787:8787"
@@ -10,11 +10,17 @@ services:
       - seccomp:./moby-20.10.6-seccomp.json
   worker-1:
     build: .
-    command: dask-worker tcp://scheduler:8786
+    command: dask worker tcp://scheduler:8786
     depends_on:
       - scheduler
+    security_opt:
+      # Remove for docker >= 20.10.6
+      - seccomp:./moby-20.10.6-seccomp.json
   worker-2:
     build: .
-    command: dask-worker tcp://scheduler:8786
+    command: dask worker tcp://scheduler:8786
     depends_on:
       - scheduler
+    security_opt:
+      # Remove for docker >= 20.10.6
+      - seccomp:./moby-20.10.6-seccomp.json