RHIDP-14077: migrate RAG and vector stores to LCORE config

[Jdubrick]

What does this PR do?:

• Migrates (temporarily) RAG from being defined in Llama Stack to using the LCORE BYOK RAG
• Migrates pre-defined vector stores (such as Notebooks) to the appropriate LCORE config definition
• Updates the LCORE image to a newer version that contains the vector store contents
• Updates the Gitops script to use the new pgvector definition via LCORE config (fyi @JslYoon)

Which issue(s) this PR fixes:

https://redhat.atlassian.net/browse/RHIDP-14077

PR acceptance criteria:

Testing and documentation do not need to be complete in order for this PR to be approved. We just need to ensure tracking issues are opened and linked to this PR, if they are not in the PR scope due to various constraints.

• Tested and Verified

• Documentation (READMEs, Product Docs, Blogs, Education Modules, etc.)

How to test changes / Special notes to the reviewer:

Pull the changes from this PR and add the lightspeed-stack.yaml contents to your lightspeed-stack.local.yaml file and run make local-up. You should be able to query /v1/rags, /v1/models and /v1/streaming_query and obtain the expected results.

For testing Notebooks it is trickier but doable. I took the changes in this PR and added them to RHDH Local and then started it up from there locally so I could test notebooks.

[qodo-code-review]

PR Summary by Qodo

Migrate BYOK RAG and vector stores to LCORE config (pgvector GitOps rewrite)

✨ Enhancement ⚙️ Configuration changes 📝 Documentation 🕐 40+ Minutes

AI Description

• Move product-docs RAG and vector store definitions into LCORE lightspeed-stack.yaml.
• Remove FAISS/vector-store wiring from the Llama Stack profile config.
• Update GitOps manifest generation to rewrite notebooks FAISS to pgvector; bump LCORE image.

Diagram

graph TD
  A["lightspeed-stack.yaml (LCORE)"] --> B["Lightspeed Core (container)"] --> C["BYOK RAG / vector_store"]
  D["llama-stack config.yaml"] --> B
  E["rag-content image/volume"] --> C
  F["generate-gitops-manifests.sh"] --> G["GitOps ConfigMap (cluster)"] --> B
  G --> H[("pgvector DB")]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Use yq (or a YAML-aware tool) instead of awk rewrites

• ➕ More robust against formatting/key-order changes in lightspeed-stack.yaml
• ➕ Easier to validate/extend transformations (e.g., multiple providers, nested configs)
• ➕ Lower risk of generating invalid YAML in edge cases
• ➖ Adds/strengthens tooling dependency and may require more scripting work now
• ➖ Potentially slower to implement if the repo avoids non-POSIX tooling in scripts

2. Maintain separate cluster overlay config (no rewrite)

• ➕ Eliminates transformation logic in generate-gitops-manifests.sh
• ➕ Makes the FAISS-vs-pgvector difference explicit and reviewable as YAML
• ➕ Simplifies debugging by reducing “generated differs from source” surprises
• ➖ Introduces another config artifact to keep in sync
• ➖ Requires deciding/standardizing overlay/merge mechanism (kustomize/helm values/etc.)

3. Make notebooks provider pgvector in config with env-driven local fallback

• ➕ Single config source; avoids GitOps-only behavior divergence
• ➕ Local vs cluster selection can be explicit via env vars
• ➖ More conditional logic in config; local setup becomes more complex
• ➖ May not be viable if LCORE expects fixed provider types per environment

Recommendation: The PR’s direction (making LCORE the source of truth for BYOK RAG/vector stores) is sound and aligns configuration with the owning runtime. The main risk is the GitOps awk-based YAML transformation: it’s workable short-term, but consider migrating to a YAML-aware transform (yq) or a dedicated cluster overlay to reduce brittleness as the config evolves.

Files changed (6) +46 / -67

Enhancement (1) +27 / -0

lightspeed-stack.yamlDefine BYOK RAG corpus and vector_store providers in LCORE config +27/-0

Define BYOK RAG corpus and vector_store providers in LCORE config

• Adds 'byok_rag' for product docs (FAISS) including 'vector_db_id' and 'db_path', and wires it into 'rag.tool'. Introduces 'vector_store' with a default notebooks provider (FAISS locally) and keeps a temporary 'native_override' for vector store annotation prompt params.

lightspeed-core-configs/lightspeed-stack.yaml

Documentation (1) +6 / -4

CONTRIBUTING.mdDocument BYOK RAG/vector_store config flow and vector_store_id update steps +6/-4

Document BYOK RAG/vector_store config flow and vector_store_id update steps

• Updates the RAG content setup instructions to point to 'byok_rag' in 'lightspeed-stack.yaml' as the place to paste 'vector_store_id'. Clarifies that notebooks is configured under 'vector_store' and is rewritten to pgvector by GitOps for cluster deployments.

docs/CONTRIBUTING.md

Other (4) +13 / -63

default-values.envBump default Lightspeed Core image tag +1/-1

Bump default Lightspeed Core image tag

• Updates 'LIGHTSPEED_CORE_IMAGE' to the newer dev build tag to pick up updated vector store contents.

env/default-values.env

images.yamlUpdate lightspeed-core image reference for sprint images +1/-1

Update lightspeed-core image reference for sprint images

• Aligns the 'lightspeed-core.image' entry with the new dev image tag so image sync tooling stays consistent.

images.yaml

config.yamlRemove vector store/provider wiring from Llama Stack profile +0/-42

Remove vector store/provider wiring from Llama Stack profile

• Deletes the 'vector_io' FAISS providers, kv store backends, and registered vector store resources that previously defined RAG/notebooks at the Llama Stack level. Leaves the remainder of the profile intact (agents, safety shield, storage defaults, server).

llama-stack-configs/config.yaml

generate-gitops-manifests.shRewrite notebooks vector_store provider to pgvector in generated GitOps ConfigMap +11/-19

Rewrite notebooks vector_store provider to pgvector in generated GitOps ConfigMap

• Replaces the old low-level rewrite with a new function that rewrites 'vector_store.providers' for notebooks from FAISS to pgvector, including connection env vars. Applies the rewrite when generating the 'lightspeed-stack' ConfigMap and removes the prior config.yaml pipeline rewrite.

scripts/generate-gitops-manifests.sh

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Context used

⚠️ Tickets: not configured — ticket URL found in PR but could not be fetched — check ticket provider credentials

1. Embedding provider removed 🐞 Bug ≡ Correctness

Description

The new byok_rag corpus references sentence-transformers//rag-content/embeddings_model, but the
GitOps generator replaces the only sentence_transformers inference provider with
vLLM/OpenAI/VertexAI providers. The generated cluster config will therefore have no provider backing
the sentence-transformers model prefix, causing RAG queries/vector-store operations that need
embeddings to fail at runtime.

Code

lightspeed-core-configs/lightspeed-stack.yaml[R39-45]

+byok_rag:
+  - rag_id: rhdh-docs
+    rag_type: inline::faiss
+    embedding_model: sentence-transformers//rag-content/embeddings_model
+    embedding_dimension: 768
+    vector_db_id: vs_757285d9-b657-4bed-b18c-3359844e8c0d
+    db_path: /rag-content/vector_db/rhdh_product_docs/1.10/faiss_store.db

Relevance

● Weak

Very similar GitOps fix (preserve sentence_transformers provider) was explicitly rejected; drop
appears intentional.

PR-#36

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

byok_rag explicitly uses a sentence-transformers//... embedding model, while the GitOps
generator removes the sentence_transformers provider from inference.providers by replacing it
with other providers; repo docs also show sentence_transformers is expected to coexist with other
providers.

lightspeed-core-configs/lightspeed-stack.yaml[36-45]
scripts/generate-gitops-manifests.sh[92-126]
docs/PROVIDERS.md[126-163]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`byok_rag.embedding_model` is set to `sentence-transformers//rag-content/embeddings_model`, which requires the `sentence_transformers` provider to remain configured. However, `scripts/generate-gitops-manifests.sh` currently *replaces* the `sentence_transformers` provider line with vLLM/OpenAI/VertexAI providers, so the generated GitOps config loses the embeddings provider.

## Issue Context
- Local/dev configs include `sentence_transformers` as the embeddings provider.
- The GitOps generator should *add* remote LLM providers for cluster usage, but must not remove the embeddings provider when RAG relies on it.

## Fix Focus Areas
- scripts/generate-gitops-manifests.sh[92-127]
- lightspeed-core-configs/lightspeed-stack.yaml[36-45]

### Suggested approach
Update the `awk` transform so it preserves the original `- type: sentence_transformers` line and appends the vLLM/OpenAI/VertexAI providers after it (or otherwise ensures a `sentence_transformers` provider remains present in the generated ConfigMap).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

_{To customize comments, go to the Qodo configuration screen, or learn more in the docs.}

[Jdubrick]

For the Qodo review, when defining it via the LCORE config it will register the model for you. Therefore the path provided will be present at runtime.

[yangcao77]

changes look good to me