Skip to content

Commit ae67e14

Browse files
dantelmomsftdantelmomsft
committed
integrate java sk 0.2.9 and cognitivesearch memory store
1 parent 446c935 commit ae67e14

24 files changed

+578
-267
lines changed

‎README.md

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -14,20 +14,23 @@ The repo includes sample data so it's ready to try end to end. In this sample ap
1414
* Chat and Q&A interfaces
1515
* Explores various options to help users evaluate the trustworthiness of responses with citations, tracking of source content, etc.
1616
* Shows possible approaches for data preparation, prompt construction, and orchestration of interaction between model (ChatGPT) and retriever (Cognitive Search)
17+
* Shows possible AI orchestration implementation using the plain Java Open AI sdk or the Java Semantic Kernel sdk
1718
* Settings directly in the UX to tweak the behavior and experiment with options
19+
*
1820

1921
![Chat screen](docs/chatscreen.png)
2022

21-
## Python Conversion Status
22-
This repo is focused to showcase different options to implement semantic search using RAG patterns with Java, Azure OpenAI and Semantic Kernel.
23-
It is still under active development. Below you can find the status of the python original repo conversion and the planned features.
24-
25-
| Python RAG Approach | Java RAG Approach | Description | Java Open AI SDK | Java Semantic Kernel |
26-
|:----------------------------|:------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------|:---------------------|
27-
| Ask - RetrieveThenRead | PlainJavaAskApproach | Use Cognitive Search and OpenAI APIs directly. It first retrieves top documents from search and use them to build a prompt. Then, it uses OpenAI to generate an answer (completion) for the user question | :white_check_mark: | :x: |
28-
| Chat - ChatReadRetrieveRead | PlainJavaChatApproach | Use Cognitive Search and OpenAI APIs directly. It first uses OpenAI to generate a search keyword for the chat history and then answer to the last chat question replicating RetrieveThenRead same steps. | :white_check_mark: | :x: |
29-
| Ask - ReadRetrieveRead | JavaSemanticKernelAskApproach | Use java Semantic Kernel framework to orchestrate Cognitive Search and OpenAI as native and semantic functions respectively. A sequential planner is used to generate steps orchestation for answering the user question. | :x: | :white_check_mark: |
30-
| ASK - ReadDecomposeAsk | TBD | Like the above but using Java Semantic Kernel built-in vector storage and search capabilities to simplify RAG implementation when vector similarity search is used to retrieve relevant documents to answer a question. | :x: | :soon: |
23+
## RAG options
24+
This repo is focused to showcase different options to implement semantic search on private documents using RAG patterns with Java, Azure OpenAI and Semantic Kernel.
25+
Below you can find the list of available implementations.
26+
27+
| RAG Approach | Description | Java Open AI SDK | Java Semantic Kernel |
28+
|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------|:----------------------|
29+
| [PlainJavaAskApproach](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/java/com/microsoft/openai/samples/rag/ask/approaches/PlainJavaAskApproach.java) | Use Cognitive Search and Java OpenAI APIs. It first retrieves top documents from search and use them to build a prompt. Then, it uses OpenAI to generate an answer for the user question.Several cognitive search retrieval options are available: Text, Vector, Hybrid. When Hybrid and Vector are selected and additional call to OpenAi is required to generate embeddings vector for the question. | :white_check_mark: | :x: |
30+
| [PlainJavaChatApproach](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/java/com/microsoft/openai/samples/rag/chat/approaches/PlainJavaChatApproach.java) | Use Cognitive Search and Java OpenAI APIs. It first calls OpenAI to generate a search keyword for the chat history and then answer to the last chat question.Several cognitive search retrieval options are available: Text, Vector, Hybrid. When Hybrid and Vector are selected and additional call to OpenAi is required to generate embeddings vector for the chat extracted keywords. | :white_check_mark: | :x: |
31+
| [JavaSemanticKernelWithMemoryApproach](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/java/com/microsoft/openai/samples/rag/ask/approaches/semantickernel/JavaSemanticKernelWithMemoryApproach.java) | Use Java Semantic Kernel framework with built-in MemoryStore for embeddings similarity search. A semantic function [RAG.AnswerQuestion](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/resources/semantickernel/Plugins/RAG/AnswerQuestion/config.json) is defined to build the prompt using Memory Store vector search results.A customized version of SK built-in CognitiveSearchMemoryStore is used to map index fields populated by the documents ingestion process. | :x: | :white_check_mark: |
32+
| [JavaSemanticKernelChainsApproach](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/java/com/microsoft/openai/samples/rag/ask/approaches/semantickernel/JavaSemanticKernelChainsApproach.java) | Use Java Semantic Kernel framework with semantic and native functions chaining. It uses an imperative style for AI orchestration through semantic kernel functions chaining. [InformationFinder.Search](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/java/com/microsoft/openai/samples/rag/ask/approaches/semantickernel/CognitiveSearchPlugin.java) native function and [RAG.AnswerQuestion](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/resources/semantickernel/Plugins/RAG/AnswerQuestion/config.json) semantic function are called sequentially. Several cognitive search retrieval options are available: Text, Vector, Hybrid. | :x: | :white_check_mark: |
33+
| [JavaSemanticKernelPlannerApproach](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/java/com/microsoft/openai/samples/rag/ask/approaches/semantickernel/JavaSemanticKernelPlannerApproach.java) | Use Java Semantic Kernel framework with built-in Planner for functions orchestration. It uses a declarative style for AI orchestration through the built-in SequentialPlanner. SequentialPlanner call OpenAI to generate a plan for answering a question using available skills/plugins: [InformationFinder](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/java/com/microsoft/openai/samples/rag/ask/approaches/semantickernel/CognitiveSearchPlugin.java) and [RAG](https://github.com/Azure-Samples/azure-search-openai-demo-java/blob/main/app/backend/src/main/resources/semantickernel/Plugins/RAG/AnswerQuestion/config.json). Several cognitive search retrieval options are available: Text, Vector, Hybrid. | :x: | :white_check_mark: |
3134

3235
## Getting Started
3336

‎app/backend/pom.xml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
<spring-cloud-azure.version>4.9.0</spring-cloud-azure.version>
1919
<azure-search.version>11.6.0-beta.8</azure-search.version>
2020
<azure-openai.version>1.0.0-beta.2</azure-openai.version>
21-
<semantic-kernel.version>0.2.8-alpha</semantic-kernel.version>
21+
<semantic-kernel.version>0.2.9-alpha</semantic-kernel.version>
2222
<mockito-inline.version>4.5.1</mockito-inline.version>
2323
</properties>
2424

@@ -101,6 +101,10 @@
101101
<groupId>com.microsoft.semantic-kernel</groupId>
102102
<artifactId>semantickernel-planners</artifactId>
103103
</dependency>
104+
<dependency>
105+
<groupId>com.microsoft.semantic-kernel</groupId>
106+
<artifactId>semantickernel-connectors-memory-azurecognitivesearch</artifactId>
107+
</dependency>
104108
<!-- Semantic Kernel end -->
105109
</dependencies>
106110

‎app/backend/src/main/java/com/microsoft/openai/samples/rag/approaches/RAGApproachFactory.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,6 @@
22

33
public interface RAGApproachFactory<I, O> {
44

5-
RAGApproach<I, O> createApproach(String approachName, RAGType ragType);
5+
RAGApproach<I, O> createApproach(String approachName, RAGType ragType, RAGOptions options);
66

77
}

‎app/backend/src/main/java/com/microsoft/openai/samples/rag/approaches/RAGApproachFactorySpringBootImpl.java

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
package com.microsoft.openai.samples.rag.approaches;
22

33
import com.microsoft.openai.samples.rag.ask.approaches.PlainJavaAskApproach;
4-
import com.microsoft.openai.samples.rag.ask.approaches.semantickernel.JavaSemanticKernelAskApproach;
4+
import com.microsoft.openai.samples.rag.ask.approaches.semantickernel.JavaSemanticKernelChainsApproach;
5+
import com.microsoft.openai.samples.rag.ask.approaches.semantickernel.JavaSemanticKernelWithMemoryApproach;
6+
import com.microsoft.openai.samples.rag.ask.approaches.semantickernel.JavaSemanticKernelPlannerApproach;
57
import com.microsoft.openai.samples.rag.chat.approaches.PlainJavaChatApproach;
68
import org.springframework.context.ApplicationContext;
79
import org.springframework.context.ApplicationContextAware;
@@ -13,7 +15,7 @@ public class RAGApproachFactorySpringBootImpl implements RAGApproachFactory, App
1315
private static final String JAVA_OPENAI_SDK = "jos";
1416
private static final String JAVA_SEMANTIC_KERNEL = "jsk";
1517

16-
private static final String JAVA_SEMANTIC_KERNEL_VECTORS = "jskv";
18+
private static final String JAVA_SEMANTIC_KERNEL_PLANNER = "jskp";
1719
private ApplicationContext applicationContext;
1820

1921
/**
@@ -23,7 +25,7 @@ public class RAGApproachFactorySpringBootImpl implements RAGApproachFactory, App
2325
* @return
2426
*/
2527
@Override
26-
public RAGApproach createApproach(String approachName, RAGType ragType) {
28+
public RAGApproach createApproach(String approachName, RAGType ragType, RAGOptions ragOptions) {
2729

2830
if (ragType.equals(RAGType.CHAT) && JAVA_OPENAI_SDK.equals(approachName)) {
2931
return applicationContext.getBean(PlainJavaChatApproach.class);
@@ -32,7 +34,12 @@ public RAGApproach createApproach(String approachName, RAGType ragType) {
3234
if (JAVA_OPENAI_SDK.equals(approachName))
3335
return applicationContext.getBean(PlainJavaAskApproach.class);
3436
else if (JAVA_SEMANTIC_KERNEL.equals(approachName))
35-
return applicationContext.getBean(JavaSemanticKernelAskApproach.class);
37+
return applicationContext.getBean(JavaSemanticKernelWithMemoryApproach.class);
38+
else if (JAVA_SEMANTIC_KERNEL_PLANNER.equals(approachName) && ragOptions.getSemantickKernelMode() != null && ragOptions.getSemantickKernelMode() == SemanticKernelMode.planner)
39+
return applicationContext.getBean(JavaSemanticKernelPlannerApproach.class);
40+
else if(JAVA_SEMANTIC_KERNEL_PLANNER.equals(approachName) && ragOptions != null && ragOptions.getSemantickKernelMode() != null && ragOptions.getSemantickKernelMode() == SemanticKernelMode.chains)
41+
return applicationContext.getBean(JavaSemanticKernelChainsApproach.class);
42+
3643
}
3744
//if this point is reached then the combination of approach and rag type is not supported
3845
throw new IllegalArgumentException("Invalid combination for approach[%s] and rag type[%s]: ".formatted(approachName, ragType));

‎app/backend/src/main/java/com/microsoft/openai/samples/rag/approaches/RAGOptions.java

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
public class RAGOptions {
44

55
private RetrievalMode retrievalMode;
6-
6+
private SemanticKernelMode semantickKernelMode;
77
private boolean semanticRanker;
88
private boolean semanticCaptions;
99
private boolean suggestFollowupQuestions;
@@ -17,6 +17,9 @@ private RAGOptions() {
1717
public RetrievalMode getRetrievalMode() {
1818
return retrievalMode;
1919
}
20+
public SemanticKernelMode getSemantickKernelMode() {
21+
return semantickKernelMode;
22+
}
2023
public boolean isSemanticRanker() {
2124
return semanticRanker;
2225
}
@@ -43,6 +46,8 @@ public boolean isSuggestFollowupQuestions() {
4346

4447
public static class Builder {
4548
private RetrievalMode retrievalMode;
49+
50+
private SemanticKernelMode semanticKernelMode;
4651
private boolean semanticRanker;
4752
private boolean semanticCaptions;
4853
private String excludeCategory;
@@ -55,6 +60,10 @@ public Builder retrievialMode(String retrievialMode) {
5560
this.retrievalMode = RetrievalMode.valueOf(retrievialMode);
5661
return this;
5762
}
63+
public Builder semanticKernelMode(String semanticKernelMode) {
64+
this.semanticKernelMode = SemanticKernelMode.valueOf(semanticKernelMode);
65+
return this;
66+
}
5867
public Builder semanticRanker(boolean semanticRanker) {
5968
this.semanticRanker = semanticRanker;
6069
return this;
@@ -88,6 +97,7 @@ public Builder top(Integer top) {
8897
public RAGOptions build() {
8998
RAGOptions ragOptions = new RAGOptions();
9099
ragOptions.retrievalMode = this.retrievalMode;
100+
ragOptions.semantickKernelMode = this.semanticKernelMode;
91101
ragOptions.semanticRanker = this.semanticRanker;
92102
ragOptions.semanticCaptions = this.semanticCaptions;
93103
ragOptions.suggestFollowupQuestions = this.suggestFollowupQuestions;

‎app/backend/src/main/java/com/microsoft/openai/samples/rag/approaches/SemanticKernelMode.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
package com.microsoft.openai.samples.rag.approaches;
2+
3+
public enum SemanticKernelMode {
4+
chains, planner;
5+
6+
}

‎app/backend/src/main/java/com/microsoft/openai/samples/rag/ask/approaches/PlainJavaAskApproach.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ public PlainJavaAskApproach(FactsRetrieverProvider factsRetrieverProvider, OpenA
4141
public RAGResponse run(String question, RAGOptions options) {
4242
//TODO exception handling
4343

44-
//Get instance of retriever based on the retrieval mode: hybryd, text, vectors
44+
//Get instance of retriever based on the retrieval mode: hybryd, text, vectors.
4545
Retriever factsRetriever = factsRetrieverProvider.getFactsRetriever(options);
4646
List<ContentSource> sources = factsRetriever.retrieveFromQuestion(question, options);
4747
LOGGER.info("Total {} sources found in cognitive search for keyword search query[{}]", sources.size(),

0 commit comments

Comments
 (0)