Building Speaches: Stuck due to uv install on system python environment

[JonnyTran]

Hi,

I'm trying to build Speaches for STT and TTS integration with OpenWebUI and Ollama. Since it uses faster-whisper backend, I figured I would be able to extend the base image faster-whisper to build speaches from their Dockerfile.

This has proved very difficult due to their use of uv to install python virtual environments. Since we have to use the pre-built python binding stored at /usr/local/lib/python3.11/dist-packages to make use of the CUDA libs, it is not straight forward to uv pip install or pip install directly to the python system environment without setting the proper PATH, making extending jetson-containers as a base images to other framework difficult.

Is there a way we can switch away from using the system python installation in packages/build/python to something like ~/.venv/, or even use micromamba env by default? This would make dealing with python environments much easier.

Model: NVIDIA AGX Orin DevKit - Jetpack 6.2 L4T 36.4.3

[RS-Credentive]

This proposal would solve another issue -- currently, the ComfyUI container loses all knowledge of its installed models and nodes on reboot. I attempted to solve this by having the '/opt/ComfyUI' directory on a volume, but that didn't work well because the Python libraries required by the nodes were lost on reboot. It would be fantastic to use a .venv inside the ComfyUI directory to preserve the libraries across container/machine reboots. I note that there is a pull request to allow this. Have you gotten any response from the maintainer on this? I would definitely be interested in building on top of your work if it's going to get merged. Otherwise, I'll probably maintain a local fork with the changes I want.

[johnnynunez]

Hello,
The first thing is that we use python from jetpack really.
The second thing is that the idea to use comfyui is not from docker inside, it’s better to have a volume.
When I install everything, I usually commit the docker for X demo.

This proposal would solve another issue -- currently, the ComfyUI container loses all knowledge of its installed models and nodes on reboot. I attempted to solve this by having the '/opt/ComfyUI' directory on a volume, but that didn't work well because the Python libraries required by the nodes were lost on reboot. It would be fantastic to use a .venv inside the ComfyUI directory to preserve the libraries across container/machine reboots. I note that there is a pull request to allow this. Have you gotten any response from the maintainer on this? I would definitely be interested in building on top of your work if it's going to get merged. Otherwise, I'll probably maintain a local fork with the changes I want.

[RS-Credentive]

Thanks for your reply. I'll open a separate issue on the ComfyUI thing - I don't want to pollute this discussion.

[johnnynunez]

Thanks for your reply. I'll open a separate issue on the ComfyUI thing - I don't want to pollute this discussion.

Dusty is on vacation during some days, I will ask what think about uv

[RS-Credentive]

I will say that it's fast and fabulous and very aggressive about enforcing venvs, and does some smart things with links to reduce the disk space used for multiple copies of different packages. it could be quite helpful to have it as core part of jetson-containers, but it could be very complicated to integrate and I don't know if it would cause more problems for other AI packages. I don't want to pretend it's just a drop in replacement.

[dusty-nv]

Thanks guys, i've not left yet, but it is fine adding in an optional package under packages/build like there are other package managers in there. Perhaps it is necessary to install for some things.

IMO it is normal in docker not to use venv, and do not want to change all the jetson-containers for it.

There are also now the majority of built wheels from jetson-containers on pypi.jetson-ai-lab.dev, so it is not as necessary just to stay with the same containers unless it is complicated stack.

[JonnyTran]

Hi folks! I understand the difficulties with using venv.

My proposed PR only tries to move away from the /usr/local/lib/python3.11/ system Python environment and setting /opt/conda/ as the new system Python environment. It doesn't enforce using venv at all (by setting the CONDA_PREFIX=/opt/conda environment variable), so all uv pip install commands will automatically install to this location. Additionally, all existing pip3 install commands will also target this environment, so it's compatible with other jetson-containers without requiring any changes.