You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have faced the problem to use GGML_CUDA=ON.
If I install this with CMAKE_ARGS="GGML_CUDA=ON", The error is issued at cuda graphs when eval. (like /api/chat)
cudaGraphKernelNodeSetParams(cuda_ctx->cuda_graph->nodes[i], &cuda_ctx->cuda_graph->params[i])
CUDA error: invalid argument
My environment is that NVIDIA Graphic Driver version is 525.25 and CUDA is 12.0. Graphic card is A100.
I have searched similiar issues or discussions, but the solutions are not in anywhere.
So I use the CMAKE_ARGS+="GGML_CUDA_GRAPH=OFF", There is no error when eval.
I wonder that there is performance issue when use GGML_CUDA_GRAPH=OFF.
Thanks for your reply.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I have faced the problem to use
GGML_CUDA=ON
.If I install this with
CMAKE_ARGS="GGML_CUDA=ON"
, The error is issued at cuda graphs when eval. (like /api/chat)My environment is that NVIDIA Graphic Driver version is 525.25 and CUDA is 12.0. Graphic card is A100.
I have searched similiar issues or discussions, but the solutions are not in anywhere.
So I use the
CMAKE_ARGS+="GGML_CUDA_GRAPH=OFF"
, There is no error when eval.I wonder that there is performance issue when use
GGML_CUDA_GRAPH=OFF
.Thanks for your reply.
Beta Was this translation helpful? Give feedback.
All reactions