Skip to content

[RHIDP-12414] Fix file-id leaking into query response - #3

Merged
Jdubrick merged 1 commit into
redhat-ai-dev:mainfrom
Jdubrick:add-vector-store-annotation
Mar 9, 2026
Merged

[RHIDP-12414] Fix file-id leaking into query response#3
Jdubrick merged 1 commit into
redhat-ai-dev:mainfrom
Jdubrick:add-vector-store-annotation

Conversation

@Jdubrick

@Jdubrick Jdubrick commented Mar 6, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?:

  • Adds the annotation prompt template to override the default in Llama Stack. This template is what was causing the <| file -id xyz |> that was making its way into our responses.

Which issue(s) this PR fixes:

https://issues.redhat.com/browse/RHIDP-12414

PR acceptance criteria:

Testing and documentation do not need to be complete in order for this PR to be approved. We just need to ensure tracking issues are opened and linked to this PR, if they are not in the PR scope due to various constraints.

  • Tested and Verified

  • Documentation (READMEs, Product Docs, Blogs, Education Modules, etc.)

How to test changes / Special notes to the reviewer:

…to response

Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>
@JslYoon

JslYoon commented Mar 6, 2026

Copy link
Copy Markdown
Contributor

Just a question @Jdubrick would this address the reference error? My understanding was that this <| file -id xyz |> was coming from the vectordb configuration side

@Jdubrick

Jdubrick commented Mar 6, 2026

Copy link
Copy Markdown
Contributor Author

Which reference error are you talking about @JslYoon? This PR fixes the reported issue here: https://redhat-internal.slack.com/archives/C081KVDV6LV/p1771989582617069?thread_ts=1771367096.585949&cid=C081KVDV6LV where its citing sources it used during the RAG tool calls, but incorrectly uses the internal file id because the default prompt template in Llama Stack is

annotation_instruction_template: Cite sources immediately at the end of sentences
      before punctuation, using `<|file-id|>` format like 'This is a fact <|file-Cn3MSNn72ENTiiq11Qda4A|>.'.
      Do not add extra punctuation. Use only the file IDs provided, do not invent
      new ones.

This updates that so these internal file ids don't get "cited" in the response

@Jdubrick
Jdubrick merged commit 6d716f6 into redhat-ai-dev:main Mar 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants