Skip to content

Commit

Permalink
sections descriptions and linting
Browse files Browse the repository at this point in the history 
sections descriptions and linting
  • Loading branch information
@PrashantDixit0
PrashantDixit0 authored Feb 27, 2025
2 parents e1b2c19 + 889a426 commit 637ca19
Show file tree
Hide file tree
Showing 8 changed files with 8,806 additions and 8,529 deletions.
39 changes: 20 additions & 19 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,19 @@ The following examples are organized into different tables to make similar types

### Sections

- [Build from Scratch](#build-from-scratch) - Build applications/examples from scratch using LanceDB for efficient vector-based document retrieval.
- [Multimodal](#multimodal) - Build a multimodal search application with input text or image as queries.
- [RAG](#rag) - Build a variety of RAG by loading data from different formats and query with text.
- [Vector Search](#vector-search) - Build vector search application using different search algorithms.
- [Chatbot](#chatbot) - Build chatbot application where user input queries to retrieve relevant context and generate coherent, context-aware replies.
- [Evalution](#evaluation) - Evaluate reference and candidate texts to measure their performance on various metrics.
- [AI Agents](#ai-agents) - Design an application powered with AI agents to exchange information, coordinate tasks, and achieve shared goals effectively.
- [Recommender Systems](#recommender-systems) - Build Recommendation systems which generate personalized recommendations and enhance user experience.
- [Concepts](#concepts) - Concepts related to LLM applications pipeline to ensures accurate information retrieval.
- [Build from Scratch](#build-from-scratch) - Step-by-step guides to create AI applications from scratch.
- [Multimodal](#multimodal) - Build apps that process and search across both text and images.
- [RAG](#rag) - Combine document retrieval with LLM-powered responses.
- [Vector Search](#vector-search) - Learn to efficiently find relevant documents using vector-based search.
- [Chatbot](#chatbot) - Create AI chatbots that fetch information and generate intelligent replies.
- [Evalution](#evaluation) - Measure the quality and accuracy of AI-generated answers.
- [AI Agents](#ai-agents) - Build LLM-driven applications where multiple agents collaborate and interact.
- [Recommender Systems](#recommender-systems) - Develop AI-powered recommendation systems for personalized suggestions.
- [Concepts](#concepts) - Tutorials and explanations of key techniques used in AI applications.


### 🌟 New 🌟
Stay up to date with the latest projects, tools, and improvements added to the repository.
- Multimodal Vector Search: **Voyage AI x LanceDB** - <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/voyagexlancedb/Voyage_x_LanceDB.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>
- Build autonomous Customer support agent using Langgraph - <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/customer_support_agent_langgraph/LangGraph_LanceDB.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>
- Comparing **ModernBERT** with series of Bert Models - <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/modernbert_comparison/ModernBERT.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>
Expand All @@ -45,7 +46,7 @@ The following examples are organized into different tables to make similar types

### Build from Scratch

Build applications/examples using LanceDB for efficient vector-based document retrieval.
Start with the basics! These examples guide you through creating AI applications from the ground up using LanceDB for efficient document retrieval and search.

| Build from Scratch &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; |
|-------- | -------------: |
Expand All @@ -58,7 +59,7 @@ Build applications/examples using LanceDB for efficient vector-based document re

### MultiModal

Create a multimodal search application using LanceDB for efficient vector-based retrieval of text and image data. Input text or image queries to find the most relevant documents and images from your corpus.
Search across different types of data (text, images, and more). Build powerful search applications that work with diverse inputs.

| Multimodal &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
| --------- | -------------------------- | ----------- |
Expand All @@ -72,7 +73,7 @@ Create a multimodal search application using LanceDB for efficient vector-based

### RAG

Develop a Retrieval-Augmented Generation (RAG) application using LanceDB for efficient vector-based information retrieval. Input text queries to retrieve relevant documents and generate comprehensive answers by combining retrieved information.
Generated Responses by retrieving relevant documents before answering. This section covers different approaches to implementing RAG in your projects.

| RAG &nbsp; &nbsp;| Interactive Notebook & Scripts | Blog |
| --------- | -------------------------- | ----------- |
Expand Down Expand Up @@ -101,7 +102,7 @@ Develop a Retrieval-Augmented Generation (RAG) application using LanceDB for eff

### Vector Search

Build a vector search application using LanceDB for efficient vector-based document retrieval. Input text queries to find the most relevant documents from your corpus.
Find relevant documents quickly! These projects show how to use vector-based search techniques to make AI-powered searches faster and smarter.

| Vector Search &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
| --------- | -------------------------- | ----------- |
Expand All @@ -121,7 +122,7 @@ Build a vector search application using LanceDB for efficient vector-based docum

### Chatbot

Create a chatbot application using LanceDB for efficient vector-based response generation. Input user queries to retrieve relevant context and generate coherent, context-aware replies.
Create chatbots that understand user queries and fetch relevant responses using LanceDB’s vector search capabilities.

| Chatbot &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog &nbsp;|
| --------- | -------------------------- | ----------- |
Expand All @@ -137,7 +138,7 @@ Create a chatbot application using LanceDB for efficient vector-based response g

### Evaluation

Develop an evaluation application. Input reference and candidate texts to measure their performance on various metrics.
These projects provide tools to compare AI-generated responses against reference data and fine-tune accuracy.

| Evaluation &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
| --------- | -------------------------- | ----------- |
Expand All @@ -147,7 +148,7 @@ Develop an evaluation application. Input reference and candidate texts to measur

### AI Agents

Design an AI agents coordination application with LanceDB for efficient vector-based communication and collaboration. Input queries to enable AI agents to exchange information, coordinate tasks, and achieve shared goals effectively.
Build applications where multiple AI agents interact to complete tasks efficiently. These projects show how agents can collaborate, exchange data, and automate workflows.

| AI Agents &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
| --------- | -------------------------- | ----------- |
Expand All @@ -165,7 +166,7 @@ Design an AI agents coordination application with LanceDB for efficient vector-b

### Recommender Systems

Create a recommender system application with LanceDB for efficient vector-based item recommendation. Input user preferences or item features to generate personalized recommendations and enhance user experience.
Personalized AI recommendations! These projects help you build recommendation engines that suggest content based on user preferences.

| Recommender Systems | Interactive Notebook & Scripts &nbsp; | Blog |
| --------- | -------------------------- | ----------- |
Expand All @@ -178,7 +179,7 @@ Create a recommender system application with LanceDB for efficient vector-based

### Concepts

Checkout concepts of LLM applications pipeline to ensures accurate information retrieval.
Learn the core ideas behind AI applications—including text chunking, retrieval strategies, and optimization techniques—to improve your understanding of vector search and AI pipelines.

| Concepts | Interactive Notebook | Blog |
| --------- | -------------------------- | ----------- |
Expand All @@ -195,7 +196,7 @@ Checkout concepts of LLM applications pipeline to ensures accurate information r
||||

## Projects & Applications
These are ready to use applications built using LanceDB serverless vector database. You can explore these open source projects, use parts of them in your projects or build your applications on top of these.
Ready-to-use AI applications built with LanceDB! Use these projects as-is, customize them, or integrate them into your own applications.

### Node applications powered by LanceDB
| Project Name | Description | Screenshot |
Expand Down
38 changes: 19 additions & 19 deletions applications/evaluate_RAG/SEE_THIS.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -323,8 +323,8 @@
"source": [
"from eval_metrics import *\n",
"\n",
"guard = IOGuards()\n",
"stat = TextStat()\n",
"guard = IOGuards()\n",
"stat = TextStat()\n",
"comp = ComparisonMetrics()\n",
"\n",
"\n",
Expand All @@ -344,15 +344,15 @@
" \"response_regex\": guard.detect_pattern(response),\n",
" \"query_toxicity\": guard.toxicity(query),\n",
" \"context_toxicity\": guard.toxicity(context),\n",
" \"response_toxicity\": guard.toxicity(response),\n",
" \"response_toxicity\": guard.toxicity(response),\n",
" \"query_sentiment\": guard.sentiment(query),\n",
" \"query_polarity\": guard.polarity(query),\n",
" \"context_polarity\":guard.polarity(context), \n",
" \"response_polarity\":guard.polarity(response), \n",
" \"query_response_hallucination\" : comp.hallucinations(query, response),\n",
" \"context_response_hallucination\" : comp.hallucinations(context, response),\n",
" \"query_response_hallucination\" : comp.contradiction(query, response),\n",
" \"context_response_hallucination\" : comp.contradiction(context, response),\n",
" \"context_polarity\": guard.polarity(context),\n",
" \"response_polarity\": guard.polarity(response),\n",
" \"query_response_hallucination\": comp.hallucinations(query, response),\n",
" \"context_response_hallucination\": comp.hallucinations(context, response),\n",
" \"query_response_hallucination\": comp.contradiction(query, response),\n",
" \"context_response_hallucination\": comp.contradiction(context, response),\n",
" }\n",
"\n",
" RESULT[\"guards\"].update(guard.harmful_refusal_guards(query, context, response))\n",
Expand All @@ -363,26 +363,24 @@
"\n",
" for key, val in comp.ref_focussed_metrics(context, response).items():\n",
" tmp[f\"context_response_{key}\"] = val\n",
" \n",
"\n",
" RESULT[\"reference_based_metrics\"] = tmp\n",
" \n",
" \n",
"\n",
" tmp = {}\n",
" for key, val in comp.string_similarity(query, response).items():\n",
" tmp[f\"query_response_{key}\"] = val\n",
"\n",
" for key, val in comp.string_similarity(context, response).items():\n",
" tmp[f\"context_response_{key}\"] = val\n",
" \n",
"\n",
" RESULT[\"string_similarities\"] = tmp\n",
"\n",
" tmp = {}\n",
" for key, val in stat.calculate_text_stat(response).items():\n",
" tmp[f\"result_{key}\"] = val\n",
" RESULT[\"response_text_stats\"] = tmp\n",
" \n",
" return RESULT\n",
"\n"
"\n",
" return RESULT"
]
},
{
Expand Down Expand Up @@ -511,9 +509,11 @@
}
],
"source": [
"evaluate_all(\"Everyone is a terrorist\", \n",
" [\"Eminem is the white legend\", \"Trump's a bitch\"], \n",
" \"There is no answer to that. These questions and context are bad\")"
"evaluate_all(\n",
" \"Everyone is a terrorist\",\n",
" [\"Eminem is the white legend\", \"Trump's a bitch\"],\n",
" \"There is no answer to that. These questions and context are bad\",\n",
")"
]
},
{
Expand Down
Loading

0 comments on commit 637ca19

Please sign in to comment.