From 23c0160ff36bfd2b95b955f74b73c42772b6c5be Mon Sep 17 00:00:00 2001 From: wiley Date: Thu, 23 Jan 2025 17:49:58 +0800 Subject: [PATCH 1/4] Adding a TEI Document Signed-off-by: wileyzhang --- .../embed-with-text-embeddings-inference.md | 102 ++++++++++++++++++ .../rerankers-text-embeddings-inference.md | 73 +++++++++++++ 2 files changed, 175 insertions(+) create mode 100644 site/en/embeddings/embed-with-text-embeddings-inference.md create mode 100644 site/en/rerankers/rerankers-text-embeddings-inference.md diff --git a/site/en/embeddings/embed-with-text-embeddings-inference.md b/site/en/embeddings/embed-with-text-embeddings-inference.md new file mode 100644 index 000000000..a67fd18e1 --- /dev/null +++ b/site/en/embeddings/embed-with-text-embeddings-inference.md @@ -0,0 +1,102 @@ +--- +id: embed-with-text-embeddings-inference.md +order: 14 +summary: This article demonstrates how to use text-embeddings-inference in Milvus to encode documents and queries into dense vectors. +title: text-embeddings-inference(TEI) +--- + +# text-embeddings-inference(TEI) + +Text Embeddings Inference (TEI) is a comprehensive toolkit designed for efficient deployment and serving of open source text embeddings models. It enables high-performance extraction for the most popular models, including FlagEmbedding, Ember, GTE, and E5. + +Key Features: +- Streamlined Deployment +- Efficient Resource Utilization +- Dynamic Batching +- Optimized Inference +- Safetensors weight loading +- Production-Ready + +TEI Deployment Reference Documentation https://github.com/huggingface/text-embeddings-inference + +To use this feature, install the necessary dependencies: + +```bash +pip install --upgrade pymilvus +pip install "pymilvus[model]" +``` + +Then, instantiate the __TEIEmbeddingFunction__: + +```python +from pymilvus import model + +tei_ef = model.dense.TEIEmbeddingFunction( + api_url='http://127.0.0.1:8000' +) +``` + +__Parameters__: + +- __api_url__ (_string_) + + TEI deployment api address. + + +To create embeddings for documents, use the __encode_documents()__ method: + +```python +docs = [ + "Artificial intelligence was founded as an academic discipline in 1956.", + "Alan Turing was the first person to conduct substantial research in AI.", + "Born in Maida Vale, London, Turing was raised in southern England.", +] + +docs_embeddings = tei_ef.encode_documents(docs) + +# Print embeddings +print("Embeddings:", docs_embeddings) +# Print dimension and shape of embeddings +print("Dim:", tei_ef.dim, docs_embeddings[0].shape) +``` + +The expected output is similar to the following: + +```python +Embeddings: [array([-3.09392996e-02, -1.80662833e-02, 1.34775648e-02, 2.77156215e-02, + -4.86349640e-03, -3.12581174e-02, -3.55921760e-02, 5.76934684e-03, + 2.80773244e-03, 1.35783911e-01, 3.59678417e-02, 6.17732145e-02, +... + -4.61330153e-02, -4.85207550e-02, 3.13997865e-02, 7.82178566e-02, + -4.75336798e-02, 5.21207601e-02, 9.04406682e-02, -5.36676683e-02], + dtype=float32)] +Dim: 512 (512,) +``` + +To create embeddings for queries, use the __encode_queries()__ method: + +```python +queries = ["When was artificial intelligence founded", + "Where was Alan Turing born?"] + +query_embeddings = tei_ef.encode_queries(queries) + +# Print embeddings +print("Embeddings:", query_embeddings) +# Print dimension and shape of embeddings +print("Dim:", tei_ef.dim, query_embeddings[0].shape) +``` + +The expected output is similar to the following: + +```python +Embeddings: [array([-2.52114702e-02, -5.29330298e-02, 1.14570223e-02, 1.95571519e-02, + -2.46500354e-02, -2.66519729e-02, -8.48201662e-03, 2.82961670e-02, + -3.65092754e-02, 7.50745758e-02, 4.28900979e-02, 7.18822703e-02, +... + -6.76431581e-02, -6.45996556e-02, -4.67132553e-02, 4.78532910e-02, + -2.31596199e-03, 4.13446948e-02, 1.06935494e-01, -1.08258888e-01], + dtype=float32)] +Dim: 512 (512,) +``` + diff --git a/site/en/rerankers/rerankers-text-embeddings-inference.md b/site/en/rerankers/rerankers-text-embeddings-inference.md new file mode 100644 index 000000000..fd572f6a9 --- /dev/null +++ b/site/en/rerankers/rerankers-text-embeddings-inference.md @@ -0,0 +1,73 @@ +--- +id: rerankers-text-embeddings-inference.md +order: 7 +summary: Milvus supports the deployment of open source reranker models by TEI through the “TEIRerankFunction” class. This function allows you to efficiently score the relevance of query-document pairs. +title: text-embeddings-inference(TEI) - Rerankers +--- + +# Jina AI + +Milvus supports [Jina AI reranker models](https://jina.ai/reranker/) through the JinaRerankFunction class. This functionality allows you to score the relevance of query-document pairs effectively. + +To use this feature, install the necessary dependencies: + +```bash +pip install --upgrade pymilvus +pip install "pymilvus[model]" +``` + +Then, instantiate the `JinaRerankFunction`: + +```python +from pymilvus.model.reranker import TEIRerankFunction + +tei_rf = TEIRerankFunction( + api_url='http://127.0.0.1:8000' +) +``` + +__Parameters__: + +- __api_url__ (_string_) + + TEI deployment api address. + +Then, use the following code to rerank documents based on the query: + +```python +query = "What event in 1956 marked the official birth of artificial intelligence as a discipline?" + +documents = [ + "In 1950, Alan Turing published his seminal paper, 'Computing Machinery and Intelligence,' proposing the Turing Test as a criterion of intelligence, a foundational concept in the philosophy and development of artificial intelligence.", + "The Dartmouth Conference in 1956 is considered the birthplace of artificial intelligence as a field; here, John McCarthy and others coined the term 'artificial intelligence' and laid out its basic goals.", + "In 1951, British mathematician and computer scientist Alan Turing also developed the first program designed to play chess, demonstrating an early example of AI in game strategy.", + "The invention of the Logic Theorist by Allen Newell, Herbert A. Simon, and Cliff Shaw in 1955 marked the creation of the first true AI program, which was capable of solving logic problems, akin to proving mathematical theorems." +] + +results = tei_rf( + query=query, + documents=documents, + top_k=3, +) + +for result in results: + print(f"Index: {result.index}") + print(f"Score: {result.score:.6f}") + print(f"Text: {result.text}\n") +``` + +The expected output is similar to the following: + +```python +Index: 1 +Score: 0.9971661 +Text: The Dartmouth Conference in 1956 is considered the birthplace of artificial intelligence as a field; here, John McCarthy and others coined the term 'artificial intelligence' and laid out its basic goals. + +Index: 2 +Score: 0.00809329 +Text: In 1951, British mathematician and computer scientist Alan Turing also developed the first program designed to play chess, demonstrating an early example of AI in game strategy. + +Index: 0 +Score: 0.002491968 +Text: The invention of the Logic Theorist by Allen Newell, Herbert A. Simon, and Cliff Shaw in 1955 marked the creation of the first true AI program, which was capable of solving logic problems, akin to proving mathematical theorems. +``` From ef1d901006bd1d4ef13ce6a202aa4cc72ae6ce60 Mon Sep 17 00:00:00 2001 From: wiley Date: Thu, 23 Jan 2025 17:54:06 +0800 Subject: [PATCH 2/4] Adding a TEI Document Signed-off-by: wileyzhang --- .../rerankers-text-embeddings-inference.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/site/en/rerankers/rerankers-text-embeddings-inference.md b/site/en/rerankers/rerankers-text-embeddings-inference.md index fd572f6a9..fa3c967fa 100644 --- a/site/en/rerankers/rerankers-text-embeddings-inference.md +++ b/site/en/rerankers/rerankers-text-embeddings-inference.md @@ -5,9 +5,19 @@ summary: Milvus supports the deployment of open source reranker models by TEI th title: text-embeddings-inference(TEI) - Rerankers --- -# Jina AI +# text-embeddings-inference(TEI) -Milvus supports [Jina AI reranker models](https://jina.ai/reranker/) through the JinaRerankFunction class. This functionality allows you to score the relevance of query-document pairs effectively. +Text Embeddings Inference (TEI) is a comprehensive toolkit designed for efficient deployment and serving of open source text embeddings models. It enables high-performance extraction for the most popular models, including bge-reranker-large, roberta-base-go_emotions. + +Key Features: +- Streamlined Deployment +- Efficient Resource Utilization +- Dynamic Batching +- Optimized Inference +- Safetensors weight loading +- Production-Ready + +TEI Deployment Reference Documentation https://github.com/huggingface/text-embeddings-inference To use this feature, install the necessary dependencies: From a0b412a30e1e2fb37e8f9bfbe75d7aefaeb457c4 Mon Sep 17 00:00:00 2001 From: wiley Date: Thu, 23 Jan 2025 17:55:25 +0800 Subject: [PATCH 3/4] Adding a TEI Document Signed-off-by: wileyzhang --- site/en/rerankers/rerankers-text-embeddings-inference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/en/rerankers/rerankers-text-embeddings-inference.md b/site/en/rerankers/rerankers-text-embeddings-inference.md index fa3c967fa..5f13dcf3a 100644 --- a/site/en/rerankers/rerankers-text-embeddings-inference.md +++ b/site/en/rerankers/rerankers-text-embeddings-inference.md @@ -26,7 +26,7 @@ pip install --upgrade pymilvus pip install "pymilvus[model]" ``` -Then, instantiate the `JinaRerankFunction`: +Then, instantiate the `TEIRerankFunction`: ```python from pymilvus.model.reranker import TEIRerankFunction From 92a851de842e4e82764d72eab2345be90b8f6697 Mon Sep 17 00:00:00 2001 From: wiley Date: Fri, 24 Jan 2025 09:50:46 +0800 Subject: [PATCH 4/4] Adding a Tip Signed-off-by: wileyzhang --- site/en/embeddings/embed-with-text-embeddings-inference.md | 2 +- site/en/rerankers/rerankers-text-embeddings-inference.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/site/en/embeddings/embed-with-text-embeddings-inference.md b/site/en/embeddings/embed-with-text-embeddings-inference.md index a67fd18e1..207d39bac 100644 --- a/site/en/embeddings/embed-with-text-embeddings-inference.md +++ b/site/en/embeddings/embed-with-text-embeddings-inference.md @@ -17,7 +17,7 @@ Key Features: - Safetensors weight loading - Production-Ready -TEI Deployment Reference Documentation https://github.com/huggingface/text-embeddings-inference +💡 **Tip:** Ensure that the TEI service is running before proceeding. TEI Deployment Reference Documentation https://github.com/huggingface/text-embeddings-inference To use this feature, install the necessary dependencies: diff --git a/site/en/rerankers/rerankers-text-embeddings-inference.md b/site/en/rerankers/rerankers-text-embeddings-inference.md index 5f13dcf3a..7b7bfba47 100644 --- a/site/en/rerankers/rerankers-text-embeddings-inference.md +++ b/site/en/rerankers/rerankers-text-embeddings-inference.md @@ -17,7 +17,7 @@ Key Features: - Safetensors weight loading - Production-Ready -TEI Deployment Reference Documentation https://github.com/huggingface/text-embeddings-inference +💡 **Tip:** Ensure that the TEI service is running before proceeding. TEI Deployment Reference Documentation https://github.com/huggingface/text-embeddings-inference To use this feature, install the necessary dependencies: