Skip to content

Add NvidiaChatGenerator component #1483

Closed
@julian-risch

Description

@julian-risch

Is your feature request related to a problem? Please describe.
The current Nvidia integration comprises a NvidiaTextEmbedder, NvidiaDocumentEmbedder, NvidiaRanker andNvidiaGenerator but no NvidiaChatGenerator.
We should implement a NvidiaChatGenerator component and add a code example to https://haystack.deepset.ai/integrations/nvidia

Describe the solution you'd like
nvidia/src/haystack_integrations/utils/nvidia/nim_backend.py contains a generate method that uses the /chat/completions endpoint.
We should implement a chat_generate in nim_backend.py. After that generate can probably be refactored to call chat_generate with a list containing a single ChatMessage where the "role" is set to "user".

Note that we are implementing a run_async in all ChatGenerators too. An initial implementation of NvidiaChatGenerator without run_async would be okay but better if we add it too. #1379

Describe alternatives you've considered

Additional context
Adding this component was suggested on discord.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions