Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lacks support for annotations field in OpenAI's chat completion object #2449

Open
minhrongcon2000 opened this issue Mar 12, 2025 · 3 comments · May be fixed by #2460
Open

Lacks support for annotations field in OpenAI's chat completion object #2449

minhrongcon2000 opened this issue Mar 12, 2025 · 3 comments · May be fixed by #2460

Comments

@minhrongcon2000
Copy link

minhrongcon2000 commented Mar 12, 2025 •
edited
Loading

Please do a quick search on GitHub issues first, there might be already a duplicate issue for the one you are about to create.
If the bug is trivial, just go ahead and create the issue. Otherwise, please take a few moments and fill in the following sections:

Bug description
Lacks support for annotations field in OpenAI's chat completion object (under chat completion object > message > annotations)
https://platform.openai.com/docs/api-reference/chat/object

Environment
org.springframework.ai:spring-ai-bom:1.0.0-M6

Steps to reproduce

Expected behavior

Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "annotations" (class org.springframework.ai.openai.api.OpenAiApi$ChatCompletionMessage), not marked as ignorable (7 known properties: "tool_call_id", "role", "content", "name", "refusal", "tool_calls", "audio"])
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 14, column: 8] (through reference chain: org.springframework.ai.openai.api.OpenAiApi$ChatCompletion["choices"]->java.util.ArrayList[0]->org.springframework.ai.openai.api.OpenAiApi$ChatCompletion$Choice["message"]->org.springframework.ai.openai.api.OpenAiApi$ChatCompletionMessage["annotations"])
	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:1153) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:2241) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1821) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperties(BeanDeserializerBase.java:1771) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:561) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:543) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:585) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:447) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:246) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:543) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:585) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:447) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2125) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1501) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:398) ~[spring-web-6.2.2.jar!/:6.2.2]
	... 85 common frames omitted

Minimal Complete Reproducible example
N/A

Ref:
@zmwei666
Copy link

The problem is very serious.

apappascs added a commit to apappascs/spring-ai that referenced this issue Mar 13, 2025
@apappascs
feat: OpenAI Web Search Annotations
72cba5a 
This PR adds support for retrieving web search annotations from the OpenAI API, as described in their [web search documentation](https://platform.openai.com/docs/guides/web-search).  This allows us to access citation URLs and their context within generated responses when using models like `gpt-4o-search-preview`.

**Changes:**

*   Added `annotations` (with `Annotation` and `UrlCitation` records) to `ChatCompletionMessage` in `OpenAiApi.java`.
*   Updated `OpenAiChatModel` to populate the `annotations` field (via metadata) for both regular and streaming responses.
*   Added integration tests (`webSearchAnnotationsTest`, `streamWebSearchAnnotationsTest`) to `OpenAiChatModelIT.java`.
*   Added `GPT_4_O_SEARCH_PREVIEW` and `GPT_4_O_MINI_SEARCH_PREVIEW` to `OpenAiApi.ChatModel`.
*   Added `WebSearchOptions` and related records to `OpenAiApi`.
*   Minor updates to `ChatCompletionRequest` and its `Builder`.

Resolves spring-projects#2449

Signed-off-by: Alexandros Pappas <[email protected]>
@apappascs apappascs linked a pull request Mar 13, 2025 that will close this issue
Open
@zainlateef
Copy link

zainlateef commented Mar 16, 2025 •
edited
Loading

As a temporary workaround, add this objectMapper to your main class

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Bean
    public ObjectMapper objectMapper() {
        ObjectMapper mapper = new ObjectMapper();
        mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        return mapper;
    }
}

@minhrongcon2000
Copy link
Author

@zainlateef yeah! I have done that in my project, but I just happened to see it, so I reported it here 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: OpenAI Web Search Annotations apappascs/spring-ai
3 participants
@zainlateef @minhrongcon2000 @zmwei666