Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inference [4.1.5] Inference nvidia-smi command exits but failed to execute #2098

Open
msalaz03 opened this issue Feb 6, 2025 · 0 comments
Open

Comments

@msalaz03
Copy link

msalaz03 commented Feb 6, 2025

Hello,

I ran the following command: nohup make generate_engines RUN_ARGS="--benchmarks=stable-diffusion-xl --scenarios=offline" 2>&1 | tee -a /Inference_Store/mlperf_4.1.5_inference_data/mlperf-logs/generate_engines_offline

And ran into the following errors.
[2025-02-06 15:29:21,626 systems.py:174 WARNING] nvidia-smi command exists but failed to execute - Ignoring NUMA detection.
[2025-02-06 15:29:21,870 main.py:227 INFO] Detected system did not match any known systems. Exiting. SystemConfiguration(host_cpu_conf=CPUConfiguration(layout={CPU(name='Cortex-A78AE', architecture=<CPUArchitecture.aarch64: AliasedName(name='aarch64', aliases=(), patterns=())>, core_count=4, threads_per_core=1): 1}), host_mem_conf=MemoryConfiguration(host_memory_capacity=Memory(quantity=64.348844, byte_suffix=<ByteSuffix.GB: (1000, 3)>, _num_bytes=64348844000), comparison_tolerance=0.05), accelerator_conf=AcceleratorConfiguration(layout=defaultdict(<class 'int'>, {GPU(name='CTI Forge + Orin AGX', accelerator_type=<AcceleratorType.Integrated: AliasedName(name='Integrated', aliases=(), patterns=())>, vram=None, max_power_limit=None, pci_id=None, compute_sm=87): 1})), numa_conf=None, system_id=None)
(mlperf) nvidia@mlperf-inference-nvidia-aarch64-1018:/closed/NVIDIA$ nvidia-smi
Thu Feb  6 15:34:15 2025

Any advice on how to resolve this issue? How can I ensure my system is detected and how can I check what my known systems are.

NVIDIA-SMI version: 540.4.0
Cuda version: 12.6
Driver version 540.4.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant