Missing mount og nvoptix.bin from libnvidia-gl-535 #1872

agirault · 2023-10-23T14:33:18Z

agirault
Oct 23, 2023

Enabling Optix denoise requires the /usr/share/nvidia/nvoptix.bin file which is installed as part of libnvidia-gl-<ver> package but not present in containers with nvidia ctk runtime.

Workaround for Holoscan: https://github.com/nvidia-holoscan/holohub/pull/112/files

Content of libnvidia-gl-535

dpkg -L libnvidia-gl-535 | xargs -I % sh -c '[ -f "%" ] && echo "%"'

x86_64:

/usr/bin/nvidia-ngx-updater
/usr/lib/x86_64-linux-gnu/libEGL_nvidia.so.535.86.05
/usr/lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.535.86.05
/usr/lib/x86_64-linux-gnu/libGLESv2_nvidia.so.535.86.05
/usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.535.86.05
/usr/lib/x86_64-linux-gnu/libnvidia-api.so.1
/usr/lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.1.1.0
/usr/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.86.05
/usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.535.86.05
/usr/lib/x86_64-linux-gnu/libnvidia-glsi.so.535.86.05
/usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.535.86.05
/usr/lib/x86_64-linux-gnu/libnvidia-ngx.so.535.86.05
/usr/lib/x86_64-linux-gnu/libnvidia-rtcore.so.535.86.05
/usr/lib/x86_64-linux-gnu/libnvidia-tls.so.535.86.05
/usr/lib/x86_64-linux-gnu/libnvidia-vulkan-producer.so.535.86.05
/usr/lib/x86_64-linux-gnu/libnvoptix.so.535.86.05
/usr/lib/x86_64-linux-gnu/nvidia/wine/_nvngx.dll
/usr/lib/x86_64-linux-gnu/nvidia/wine/nvngx.dll
/usr/lib/x86_64-linux-gnu/nvidia/xorg/libglxserver_nvidia.so.535.86.05
/usr/share/doc/libnvidia-gl-535/changelog.Debian.gz
/usr/share/doc/libnvidia-gl-535/copyright
/usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json
/usr/share/glvnd/egl_vendor.d/10_nvidia.json
/usr/share/lintian/overrides/libnvidia-gl-535
/usr/share/nvidia/nvoptix.bin
/usr/share/vulkan/icd.d/nvidia_icd.json
/usr/share/vulkan/implicit_layer.d/nvidia_layers.json
/usr/lib/x86_64-linux-gnu/libEGL_nvidia.so.0
/usr/lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.1
/usr/lib/x86_64-linux-gnu/libGLESv2_nvidia.so.2
/usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0
/usr/lib/x86_64-linux-gnu/libnvoptix.so.1
/usr/lib/x86_64-linux-gnu/nvidia/xorg/libglxserver_nvidia.so

aarch64:

/usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.535.86.10
/usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.535.86.10
/usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.535.86.10
/usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.535.86.10
/usr/lib/aarch64-linux-gnu/libnvidia-egl-gbm.so.1.1.0
/usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.535.86.10
/usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.535.86.10
/usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.535.86.10
/usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.535.86.10
/usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.535.86.10
/usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.535.86.10
/usr/lib/aarch64-linux-gnu/libnvidia-tls.so.535.86.10
/usr/lib/aarch64-linux-gnu/libnvoptix.so.535.86.10
/usr/lib/aarch64-linux-gnu/nvidia/xorg/libglxserver_nvidia.so.535.86.10
/usr/share/doc/libnvidia-gl-535/changelog.Debian.gz
/usr/share/doc/libnvidia-gl-535/copyright
/usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json
/usr/share/glvnd/egl_vendor.d/10_nvidia.json
/usr/share/nvidia/nvoptix.bin
/usr/share/vulkan/icd.d/nvidia_icd.json
/usr/share/vulkan/icd.d/nvidia_layers.json
/usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.0
/usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.1
/usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.2
/usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.0
/usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.1
/usr/lib/aarch64-linux-gnu/libnvoptix.so.1
/usr/lib/aarch64-linux-gnu/nvidia/xorg/libglxserver_nvidia.so

Files not mounted with nvidia runtime

Run this command to test:

nv_gl_files=$(dpkg -L libnvidia-gl-535 | xargs -I % sh -c '[ -f "%" ] && echo "%"')
docker run -it --rm \
  --runtime=nvidia -e NVIDIA_DRIVER_CAPABILITIES=all --gpus=all \
  -e FILES="$nv_gl_files" \
  ubuntu:22.04 \
  bash -c '
    for file in $FILES; do
      [ ! -f "$file" ] && echo "Missing: $file"
    done
'

x86_64:

Missing: /usr/bin/nvidia-ngx-updater
Missing: /usr/lib/x86_64-linux-gnu/libnvidia-api.so.1
Missing: /usr/lib/x86_64-linux-gnu/libnvidia-vulkan-producer.so.535.86.05
Missing: /usr/lib/x86_64-linux-gnu/nvidia/wine/_nvngx.dll
Missing: /usr/lib/x86_64-linux-gnu/nvidia/wine/nvngx.dll
Missing: /usr/share/doc/libnvidia-gl-535/changelog.Debian.gz
Missing: /usr/share/doc/libnvidia-gl-535/copyright
Missing: /usr/share/lintian/overrides/libnvidia-gl-535
Missing: /usr/share/nvidia/nvoptix.bin

aarch64:

Missing: /usr/lib/aarch64-linux-gnu/libnvidia-egl-gbm.so.1.1.0
Missing: /usr/share/doc/libnvidia-gl-535/changelog.Debian.gz
Missing: /usr/share/doc/libnvidia-gl-535/copyright
Missing: /usr/share/nvidia/nvoptix.bin
Missing: /usr/share/vulkan/icd.d/nvidia_layers.json

Observations

Why dll files on x86_64? /wine/nvngx.dll. Interestingly, there is no libnvidia-ngx.so.1 on x86_64 (vs aarch64).
The missing nvidia-ngx-updater, libnvidia-api.so.1 and libnvidia-vulkan-producer.so.535 only exist on x86_64. Expected ? Need mounting?
libnvidia-egl-gbm.so exist for both x86_64 and aarch64, but missing only in aarch64 containers.
nvidia_layers.json is in icd.d on aarch64, instead of implicit_layer.d in x86_64. The former isn't mounted, while the latter is.

agirault · 2023-10-23T14:52:40Z

agirault
Oct 23, 2023
Author

cc @AndreasHeumann @jjomier

0 replies

elezar · 2023-11-22T12:47:51Z

elezar
Nov 22, 2023
Maintainer

@agirault thanks for reporting this. Looking at the list of files, I think adding the following is relatively straightforward:

/usr/share/nvidia/nvoptix.bin

The following (for aarch64) is also not really a problem:

/usr/share/vulkan/icd.d/nvidia_layers.json
but it would be good to confirm that there is no conflicting file at this location for x86_64 systems. Handling such a conflict is possible though, we just need an indication as to whether the additional effort is required there.

With regards to the libnvidia-egl-gbm.so file. Since the file actually included in the driver installation is libnvidia-egl-gbm.so.1.1.0 it would be good to understand which symlinks on the host (in either case) point to this file.

The same is required for libnvidia-api.so.1. Here it's key to know what this points to on the host -- since it's expected to be a symbolic link.

0 replies

elezar · 2023-11-22T12:56:35Z

elezar
Nov 22, 2023
Maintainer

I have created https://gitlab.com/nvidia/container-toolkit/container-toolkit/-/merge_requests/501 to add the processing of these files. If we can settle on a final list of missing ones that should be included we can get that in to an upcoming release candidate.

0 replies

elezar · 2023-12-04T12:55:08Z

elezar
Dec 4, 2023
Maintainer

We have just released https://github.com/NVIDIA/nvidia-container-toolkit/releases/tag/v1.15.0-rc.1 that includes the injection of the nvoptix.bin file. The packages are available from our public experimental repositories.

Assuming these have been configured running:

sudo apt-get install -y \
    nvidia-container-toolkit=1.15.0~rc.1-1 \
    nvidia-container-toolkit-base=1.15.0~rc.1-1  \
    libnvidia-container-tools=1.15.0~rc.1-1 \
    libnvidia-container1=1.15.0~rc.1-1

should install the required packages.

0 replies

elezar · 2024-01-23T10:04:15Z

elezar
Jan 23, 2024
Maintainer

Note that we have backported these changes to the release-0.14 branch and they are included in the v1.14.4 release.

@agirault if you get a chance to validate what is still missing that would be great.

0 replies

turowicz · 2024-04-24T14:02:43Z

turowicz
Apr 24, 2024

I think the issue is back with latest

apt list --installed | grep nvidia-container-toolkit

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

nvidia-container-toolkit-base/unknown,now 1.15.0-1 amd64 [installed,automatic]
nvidia-container-toolkit/unknown,now 1.15.0-1 amd64 [installed,automatic]

I recently started getting in Omniverse docker container for Isaac Sim:

Could not open optix denoiser weights file "/usr/share/nvidia/nvoptix.bin"

0 replies

elezar · 2024-04-24T14:31:12Z

elezar
Apr 24, 2024
Maintainer

@turowicz first, could you confirm that the file exists on your host?

Then, which docker command are you running? Could you confirm that you are using the nvidia runtime and that the image has NVIDIA_DRIVER_CAPABILITIES=all set (alternatively add -e NVIDIA_DRIVER_CAPABILITIES=all to your docker command line).

The nvoptix.bin file is only injected if NVIDIA_DRIVER_CAPABILITIES include graphics or display.

0 replies

turowicz · 2024-04-24T14:33:24Z

turowicz
Apr 24, 2024

yes, to fix the error I had to -v /usr/share/nvidia/nvoptix.bin:/usr/share/nvidia/nvoptix.bin
I am using nvidia runtime through --gpus all
I don't use -e NVIDIA_DRIVER_CAPABILITIES=all and I have never used it. It used to work fine without it.

0 replies

turowicz · 2024-04-24T14:34:27Z

turowicz
Apr 24, 2024

updated my answer above

0 replies

turowicz · 2024-04-24T14:35:11Z

turowicz
Apr 24, 2024

addon: I am using nvcr.io/nvidia/isaac-sim:2023.1.1 and it used to work fine.

0 replies

turowicz · 2024-04-24T14:41:36Z

turowicz
Apr 24, 2024

I confirm the container nvcr.io/nvidia/isaac-sim:2023.1.1 has NVIDIA_DRIVER_CAPABILITIES=all

0 replies

elezar · 2024-04-24T15:20:10Z

elezar
Apr 24, 2024
Maintainer

@turowicz could you provide the full docker command you run?

0 replies

turowicz · 2024-04-25T06:31:46Z

turowicz
Apr 25, 2024

Here's the .devcontainer file:

// See https://aka.ms/vscode-remote/containers for the
// documentation about the devcontainer.json format
{
	"name": "surveily.omniverse",
	"build": {
		"dockerfile": "dockerfile"
	},
	"runArgs": [
		"--name",
		"surveily.omniverse",
		"-v",
		"${env:HOME}${env:USERPROFILE}/.ssh:/root/.ssh-localhost:ro",
		"-v",
		"/var/run/docker.sock:/var/run/docker.sock",
		"-v",
		"/usr/share/nvidia/nvoptix.bin:/usr/share/nvidia/nvoptix.bin",
		"--network",
		"host",
		"--gpus",
		"all",
		"-e",
		"ACCEPT_EULA=Y",
		"-e",
		"PRIVACY_CONSENT=N"
	],
	"postCreateCommand": "mkdir -p ~/.ssh && cp -r ~/.ssh-localhost/* ~/.ssh && chmod 700 ~/.ssh && chmod 600 ~/.ssh/*",
	"appPort": [
		"5003:5003"
	],
	"extensions": [
		"kosunix.guid",
		"redhat.vscode-yaml",
		"rogalmic.bash-debug",
		"mikeburgh.xml-format",
		"donjayamanne.githistory",
		"ms-azuretools.vscode-docker",
		"ms-azure-devops.azure-pipelines",
	],
	"settings": {
		"extensions.autoUpdate": false,
		"files.exclude": {
			"**/CVS": true,
			"**/bin": true,
			"**/obj": true,
			"**/.hg": true,
			"**/.svn": true,
			"**/.git": true,
			"**/.DS_Store": true,
			"**/BenchmarkDotNet.Artifacts": true
		}
	},
	"shutdownAction": "stopContainer",
}

and the dockerfile:

FROM nvcr.io/nvidia/isaac-sim:2023.1.1

# Install tools
RUN apt update && apt install git vim -y

# Remove ROS/2 Bridge
RUN sed -i 's/ros_bridge_extension = "omni.isaac.ros2_bridge"/ros_bridge_extension = ""/g' /isaac-sim/apps/omni.isaac.sim.base.kit

# Toggle Grid Off
RUN sed -i '17i import omni.kit.viewport' /isaac-sim/extscache/omni.replicator.replicator_yaml-2.0.4+lx64/omni/replicator/replicator_yaml/scripts/replicator_yaml_extension.py
RUN sed -i '100i \ \ \ \ \ \ \ \ omni.kit.viewport.actions.actions.toggle_global_visibility(visible=False)' /isaac-sim/extscache/omni.replicator.replicator_yaml-2.0.4+lx64/omni/replicator/replicator_yaml/scripts/replicator_yaml_extension.py

0 replies

turowicz · 2024-05-21T19:34:07Z

turowicz
May 21, 2024

my workaround works but you guys may want to fix the problem

0 replies

2025-11-05T19:37:28Z

github-actions[bot]
Bot Nov 5, 2025

This issue is stale because it has been open 90 days with no activity. This issue will be closed in 30 days unless new comments are made or the stale label is removed. To skip these checks, apply the "lifecycle/frozen" label.

0 replies

turowicz · 2025-11-06T09:37:16Z

turowicz
Nov 6, 2025

hey hey hey

0 replies

kevinfoley · 2025-11-19T18:09:15Z

kevinfoley
Nov 19, 2025

This issue seems to still exist in 5.1. We are testing Isaac Sim in headless mode in a Docker container on GCP with WebRTC livestreaming. We installed following the instructions here. When the Interactive renderer is selected, we get error spam every frame:

2025-11-19T18:06:51Z [207,753ms] [Error] [rtx.optixdenoising.plugin] [Optix] [DENOISER] Could not open optix denoiser weights file /usr/share/nvidia/nvoptix.bin
2025-11-19T18:06:51Z [207,753ms] [Error] [rtx.optixdenoising.plugin] optixDenoiserCreate(m_optixCtx, temporal ? OPTIX_DENOISER_MODEL_KIND_TEMPORAL_AOV : OPTIX_DENOISER_MODEL_KIND_AOV, &denoiserOptions, &denoiser) failed. Optix Error: OPTIX_ERROR_INTERNAL_ERROR.
Internal error

0 replies

Bam4d · 2025-12-10T16:59:07Z

Bam4d
Dec 10, 2025

Can confirm that this is still an issue. Using Isaac Sim also. bin file is missing.

0 replies

elezar · 2026-01-29T14:26:04Z

elezar
Jan 29, 2026
Maintainer

@turowicz, @kevinfoley , @Bam4d sorry for the silence from this side.

Could you please provide an update on the current state. This really SHOULD be addressed in v1.18.0 of the toolkit or later (by default), but could still depend on exactly how a container is being started.

Please note that as a prerequisite for these files being injected into a container, these MUST exist on the host. In the case of GCP, it may be that the files are at locations that we're not currently searching that that would have to be corrected on our end.

0 replies

kevinfoley · 2026-01-29T17:43:59Z

kevinfoley
Jan 29, 2026

I can't provide any further info, we weren't happy with Isaac Sim and have moved on to other software.

0 replies

turowicz · 2026-01-30T12:46:01Z

turowicz
Jan 30, 2026

Dear @kevinfoley can you please tell me what software have you picked? We are very irritated by IsaacSim and how its being supported.

0 replies

kevinfoley · 2026-01-30T17:27:53Z

kevinfoley
Jan 30, 2026

Dear @kevinfoley can you please tell me what software have you picked? We are very irritated by IsaacSim and how its being supported.

BlenderProc. It can't do many of the things that Isaac Sim can, but it installs and loads much faster, the API is more straightforward, and it doesn't have any serious bugs that I'm aware of.

0 replies

2026-05-01T05:36:16Z

github-actions[bot]
Bot May 1, 2026

This issue is stale because it has been open 90 days with no activity. This issue will be closed in 30 days unless new comments are made or the stale label is removed. To skip these checks, apply the "lifecycle/frozen" label.

0 replies

turowicz · 2026-05-04T18:05:28Z

turowicz
May 4, 2026

hey hey hey

0 replies

ssalagam · 2026-06-08T16:44:03Z

ssalagam
Jun 8, 2026
Maintainer

Keeping this open for active discussion.
Original issue should be addressed in v1.18.0 of the toolkit or later.

0 replies

Missing mount og nvoptix.bin from libnvidia-gl-535 #1872

Uh oh!

Uh oh!

Content of libnvidia-gl-535

Files not mounted with nvidia runtime

Observations

Replies: 25 comments

Uh oh!

agirault Oct 23, 2023 Author

Uh oh!

elezar Nov 22, 2023 Maintainer

Uh oh!

elezar Nov 22, 2023 Maintainer

Uh oh!

elezar Dec 4, 2023 Maintainer

Uh oh!

elezar Jan 23, 2024 Maintainer

Uh oh!

Uh oh!

Uh oh!

elezar Apr 24, 2024 Maintainer

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elezar Apr 24, 2024 Maintainer

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions[bot] Bot Nov 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elezar Jan 29, 2026 Maintainer

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions[bot] Bot May 1, 2026

Uh oh!

Uh oh!

ssalagam Jun 8, 2026 Maintainer

agirault
Oct 23, 2023
Author

elezar
Nov 22, 2023
Maintainer

elezar
Nov 22, 2023
Maintainer

elezar
Dec 4, 2023
Maintainer

elezar
Jan 23, 2024
Maintainer

elezar
Apr 24, 2024
Maintainer

elezar
Apr 24, 2024
Maintainer

github-actions[bot]
Bot Nov 5, 2025

elezar
Jan 29, 2026
Maintainer

github-actions[bot]
Bot May 1, 2026

ssalagam
Jun 8, 2026
Maintainer