Description
Is your feature request related to a problem? Please describe.
The current Nvidia integration comprises a NvidiaTextEmbedder
, NvidiaDocumentEmbedder
, NvidiaRanker
andNvidiaGenerator
but no NvidiaChatGenerator
.
We should implement a NvidiaChatGenerator
component and add a code example to https://haystack.deepset.ai/integrations/nvidia
Describe the solution you'd like
nvidia/src/haystack_integrations/utils/nvidia/nim_backend.py
contains a generate
method that uses the /chat/completions endpoint.
We should implement a chat_generate
in nim_backend.py
. After that generate
can probably be refactored to call chat_generate
with a list containing a single ChatMessage where the "role"
is set to "user"
.
Note that we are implementing a run_async
in all ChatGenerators
too. An initial implementation of NvidiaChatGenerator
without run_async
would be okay but better if we add it too. #1379
Describe alternatives you've considered
Additional context
Adding this component was suggested on discord.