-
Notifications
You must be signed in to change notification settings - Fork 483
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #3584 from opensearch-project/main
Push DeepSeek blog (and other staged changes) to prod
- Loading branch information
Showing
7 changed files
with
303 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,4 +20,4 @@ This code of conduct applies to all spaces provided by the OpenSource project in | |
|
|
||
| **Enforcement and Reporting Code of Conduct Issues:** | ||
|
|
||
| Instances of abusive, harassing, or otherwise unacceptable behavior may be reported. [Contact us](mailto:[email protected]). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. | ||
| Instances of abusive, harassing, or otherwise unacceptable behavior may be reported. [Contact us](mailto:[email protected]). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| --- | ||
| name: Nathalie Jonathan | ||
| short_name: nathhjo | ||
| photo: "/assets/media/community/members/nathhjo.jpg" | ||
| title: 'OpenSearch Community Member: Nathalie Jonathan' | ||
| primary_title: Nathalie Jonathan | ||
| breadcrumbs: | ||
| icon: community | ||
| items: | ||
| - title: Community | ||
| url: /community/index.html | ||
| - title: Members | ||
| url: /community/members/index.html | ||
| - title: 'Nathalie Jonathan's Profile' | ||
| url: '/community/members/nathalie-jonathan.html' | ||
| github: nathaliellenaa | ||
| job_title_and_company: 'Software engineer at Amazon Web Services' | ||
| personas: | ||
| - author | ||
| permalink: '/community/members/nathalie-jonathan.html' | ||
| redirect_from: '/authors/nathhjo/' | ||
| --- | ||
|
|
||
| **Nathalie Jonathan** is a Software Engineer at AWS working on the OpenSearch ML Commons team. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
270 changes: 270 additions & 0 deletions
270
_posts/2025-01-28-OpenSearch-Now-Supports-DeepSeek-Chat-Models.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,270 @@ | ||
| --- | ||
| layout: post | ||
| title: "OpenSearch now supports DeepSeek chat models" | ||
| authors: | ||
| - seanzheng | ||
| - ylwu | ||
| - nathhjo | ||
| - kolchfa | ||
| date: 2025-01-29 | ||
| categories: | ||
| - technical-posts | ||
| meta_keywords: OpenSearch DeepSeek integration, LLM integration, RAG, AI search, machine learning, natural language processing, open-source LLM | ||
| meta_description: Explore how OpenSearch's integration with DeepSeek R1 LLM models enables cost-effective Retrieval-Augmented Generation (RAG) while maintaining high performance comparable to leading LLMs. | ||
| --- | ||
|
|
||
| We're excited to announce that OpenSearch now supports DeepSeek integration, providing powerful and cost-effective AI capabilities. DeepSeek R1 is a recently released open-source large language model (LLM) that delivers **similar benchmarking performance** to leading LLMs like OpenAI O1 ([report](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf)) at a significantly **lower cost** ([DeepSeek API pricing](https://api-docs.deepseek.com/quick_start/pricing)). Because DeepSeek R1 is open source, you can download and deploy it to your preferred infrastructure. This enables you to build more cost-effective and sustainable retrieval-augmented generation (RAG) solutions. | ||
|
|
||
| OpenSearch gives you the flexibility to connect to any remote inference service, such as DeepSeek or OpenAI, using machine learning (ML) connectors. You can use [prebuilt connector blueprints](https://github.com/opensearch-project/ml-commons/tree/main/docs/remote_inference_blueprints) or customize connectors based on your requirements. For more information about connector blueprints, see [Blueprints](https://opensearch.org/docs/latest/ml-commons-plugin/remote-models/blueprints/). | ||
|
|
||
| We've added a new [connector blueprint](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/deepseek_connector_chat_blueprint.md) for the DeepSeek R1 model. This integration, combined with OpenSearch's built-in vector database capabilities, makes it easier and more cost effective to build [RAG applications](https://opensearch.org/docs/latest/search-plugins/conversational-search) in OpenSearch. | ||
|
|
||
| The following example shows you how to implement RAG with DeepSeek in OpenSearch's vector database. This example guides you through creating a connector for the [DeepSeek chat model](https://api-docs.deepseek.com/api/create-chat-completion) and setting up a [RAG pipeline](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rag-processor/) in OpenSearch. | ||
|
|
||
| ### 1. Create a connector for DeepSeek | ||
|
|
||
| First, create a connector for the DeepSeek chat model, providing your own DeepSeek API key: | ||
|
|
||
| ```json | ||
| POST /_plugins/_ml/connectors/_create | ||
| { | ||
| "name": "DeepSeek Chat", | ||
| "description": "Test connector for DeepSeek Chat", | ||
| "version": "1", | ||
| "protocol": "http", | ||
| "parameters": { | ||
| "endpoint": "api.deepseek.com", | ||
| "model": "deepseek-chat" | ||
| }, | ||
| "credential": { | ||
| "deepSeek_key": "<PLEASE ADD YOUR DEEPSEEK API KEY HERE>" | ||
| }, | ||
| "actions": [ | ||
| { | ||
| "action_type": "predict", | ||
| "method": "POST", | ||
| "url": "https://${parameters.endpoint}/v1/chat/completions", | ||
| "headers": { | ||
| "Content-Type": "application/json", | ||
| "Authorization": "Bearer ${credential.deepSeek_key}" | ||
| }, | ||
| "request_body": "{ \"model\": \"${parameters.model}\", \"messages\": ${parameters.messages} }" | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| The response contains a connector ID for the newly created connector: | ||
|
|
||
| ```json | ||
| { | ||
| "connector_id": "n0dOqZQBQwAL8-GO1pYI" | ||
| } | ||
| ``` | ||
|
|
||
| For more information, see [Connecting to externally hosted models](https://opensearch.org/docs/latest/ml-commons-plugin/remote-models/index/). | ||
|
|
||
| ### 2. Create a model group | ||
|
|
||
| Create a model group for the DeepSeek chat model: | ||
|
|
||
| ```json | ||
| POST /_plugins/_ml/model_groups/_register | ||
| { | ||
| "name": "remote_model_group_chat", | ||
| "description": "This is an example description" | ||
| } | ||
| ``` | ||
|
|
||
| The response contains a model group ID: | ||
|
|
||
| ```json | ||
| { | ||
| "model_group_id": "b0cjqZQBQwAL8-GOVJZ4", | ||
| "status": "CREATED" | ||
| } | ||
| ``` | ||
|
|
||
| For more information about model groups, see [Model access control](https://opensearch.org/docs/latest/ml-commons-plugin/model-access-control/). | ||
|
|
||
| ### 3. Register and deploy the model | ||
|
|
||
| Register the model to the model group and deploy the model using the model group ID and connector ID created in the previous steps: | ||
|
|
||
| ```json | ||
| POST /_plugins/_ml/models/_register?deploy=true | ||
| { | ||
| "name": "DeepSeek Chat model", | ||
| "function_name": "remote", | ||
| "model_group_id": "b0cjqZQBQwAL8-GOVJZ4", | ||
| "description": "DeepSeek Chat", | ||
| "connector_id": "n0dOqZQBQwAL8-GO1pYI" | ||
| } | ||
| ``` | ||
|
|
||
| The response contains the model ID: | ||
|
|
||
| ```json | ||
| { | ||
| "task_id": "oEdPqZQBQwAL8-GOCJbw", | ||
| "status": "CREATED", | ||
| "model_id": "oUdPqZQBQwAL8-GOCZYL" | ||
| } | ||
| ``` | ||
|
|
||
| To ensure that the connector is working as expected, test the model: | ||
|
|
||
| ```json | ||
| POST /_plugins/_ml/models/oUdPqZQBQwAL8-GOCZYL/_predict | ||
| { | ||
| "parameters": { | ||
| "messages": [ | ||
| { | ||
| "role": "system", | ||
| "content": "You are a helpful assistant." | ||
| }, | ||
| { | ||
| "role": "user", | ||
| "content": "Hello!" | ||
| } | ||
| ] | ||
| }, | ||
| "stream": false | ||
| } | ||
| ``` | ||
|
|
||
| The response verifies that the connector is working as expected: | ||
|
|
||
| ```json | ||
| { | ||
| "inference_results": [ | ||
| { | ||
| "output": [ | ||
| { | ||
| "name": "response", | ||
| "dataAsMap": { | ||
| "id": "9d9bd689-88a5-44b0-b73f-2daa92518761", | ||
| "object": "chat.completion", | ||
| "created": 1.738011126E9, | ||
| "model": "deepseek-chat", | ||
| "choices": [ | ||
| { | ||
| "index": 0.0, | ||
| "message": { | ||
| "role": "assistant", | ||
| "content": "Hello! How can I assist you today? 😊" | ||
| }, | ||
| "finish_reason": "stop" | ||
| } | ||
| ], | ||
| "usage": { | ||
| "prompt_tokens": 11.0, | ||
| "completion_tokens": 11.0, | ||
| "total_tokens": 22.0, | ||
| "prompt_tokens_details": { | ||
| "cached_tokens": 0.0 | ||
| }, | ||
| "prompt_cache_hit_tokens": 0.0, | ||
| "prompt_cache_miss_tokens": 11.0 | ||
| }, | ||
| "system_fingerprint": "fp_3a5770e1b4" | ||
| } | ||
| } | ||
| ], | ||
| "status_code": 200 | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| ### 4. Create a search pipeline | ||
|
|
||
| Create a search pipeline with a `retrieval_augmented_generation` processor: | ||
|
|
||
| ```json | ||
| PUT /_search/pipeline/rag_pipeline | ||
| { | ||
| "response_processors": [ | ||
| { | ||
| "retrieval_augmented_generation": { | ||
| "tag": "openai_pipeline_demo", | ||
| "description": "Demo pipeline Using OpenAI Connector", | ||
| "model_id": "oUdPqZQBQwAL8-GOCZY", | ||
| "context_field_list": ["text"], | ||
| "system_prompt": "You are a helpful assistant", | ||
| "user_instructions": "Generate a concise and informative answer in less than 100 words for the given question" | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| For more information, see [Conversational search](https://opensearch.org/docs/latest/search-plugins/conversational-search). | ||
|
|
||
| ### 5. Create a conversation memory | ||
|
|
||
| Assuming that you created a k-NN index and ingested the data, you can now create a conversation memory. For more information about creating a k-NN index, see [k-NN index](https://opensearch.org/docs/latest/search-plugins/knn/knn-index/). For more information about ingesting data, see [Ingest RAG data into an index](https://opensearch.org/docs/latest/search-plugins/conversational-search/#step-4-ingest-rag-data-into-an-index). | ||
|
|
||
| Create a conversation memory to store all messages from a conversation: | ||
|
|
||
| ```json | ||
| POST /_plugins/_ml/memory/ | ||
| { | ||
| "name": "Conversation about NYC population" | ||
| } | ||
| ``` | ||
|
|
||
| The response contains a memory ID for the created memory: | ||
|
|
||
| ```json | ||
| { | ||
| "memory_id": "znCqcI0BfUsSoeNTntd7" | ||
| } | ||
| ``` | ||
|
|
||
| ### 6. Use the pipeline for RAG | ||
|
|
||
| Send a query to OpenSearch and provide additional parameters in the `ext.generative_qa_parameters` object: | ||
|
|
||
| ```json | ||
| GET /my_rag_test_data/_search | ||
| { | ||
| "query": { | ||
| "match": { | ||
| "text": "What's the population of NYC metro area in 2023" | ||
| } | ||
| }, | ||
| "ext": { | ||
| "generative_qa_parameters": { | ||
| "llm_model": "gpt-3.5-turbo", | ||
| "llm_question": "What's the population of NYC metro area in 2023", | ||
| "memory_id": "znCqcI0BfUsSoeNTntd7", | ||
| "context_size": 5, | ||
| "message_size": 5, | ||
| "timeout": 15 | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| The response contains the model output: | ||
|
|
||
| ```json | ||
| { | ||
| ... | ||
| "ext": { | ||
| "retrieval_augmented_generation": { | ||
| "answer": "The population of the New York City metro area in 2022 was 18,867,000.", | ||
| "message_id": "p3CvcI0BfUsSoeNTj9iH" | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Wrapping up | ||
|
|
||
| By integrating DeepSeek R1, OpenSearch continues its mission to democratize AI-powered search and analytics—offering developers **more choice, greater flexibility, and lower costs**. | ||
|
|
||
| **Try DeepSeek R1 now!** | ||
|
|
||
| As always, we welcome your feedback, and we'd love to hear from you on the [OpenSearch forum](https://forum.opensearch.org/). |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters