Skip to content

SETUP_GUIDE.md

Latest commit

 

History

History
264 lines (201 loc) · 15.8 KB

SETUP_GUIDE.md

File metadata and controls

264 lines (201 loc) · 15.8 KB
Raw

Rolling demo setup

Here are the steps required to setup an instance of the rolling demo on your own:

Pre-requisites

Cluster requirements

Two install paths are available depending on your cluster size:

Install command Minimum node type GPU nodes RHOAI required
make install g5.2xlarge or bigger Yes Yes
make install-no-rhoai Any OCP node No No

For make install (full stack), you need an OpenShift cluster (version 4.19+) with GPU-capable nodes (g5.2xlarge or bigger), and you must also clone the odh-kubeflow-model-registry-setup repo locally.

For make install-no-rhoai (lightweight), any OCP 4.19+ cluster works — no GPU nodes, no RHOAI, and no odh-kubeflow-model-registry-setup repo is needed. Set ODH_SETUP_DIR to any non-empty placeholder value (e.g., "n/a") in your private-env since the script validates it is set but will not use it when SKIP_RHOAI_SETUP=true.

Dependencies

In order to be able to set everything up you need to have installed:

Setup the private-env file

  • Create a file called private-env inside the scripts/ directory and copy all the contents of the scripts/env file there.

  • Your scripts/private-env should look like:

# GitOps repo settings
# GITOPS_REPO_URL: the URL of the gitops repository. If you are working
# from a fork, set this to your fork's URL.
export GITOPS_REPO_URL="https://github.com/your-org/ai-rolling-demo-gitops.git"
# GITOPS_TARGET_REVISION: the git branch ArgoCD will track.
export GITOPS_TARGET_REVISION="dev"
# ODH_SETUP_DIR: the local path to the cloned odh-kubeflow-model-registry-setup repo.
export ODH_SETUP_DIR="/path/to/odh-kubeflow-model-registry-setup"
# RHDH_NAMESPACE: the namespace where RHDH and related resources are deployed.
export RHDH_NAMESPACE="rolling-demo-ns"
# ARGOCD_APP_NAME: the argoCD application name.
export ARGOCD_APP_NAME="rolling-demo"

# Github secrets
# For more information on how to setup your github app
# you can take a look here:
# https://github.com/redhat-ai-dev/ai-rhdh-installer/blob/main/docs/APP-SETUP.md
export GITHUB_APP_APP_ID="your-github-apps-app-id"
export GITOPS_GIT_TOKEN="your-github-token"
# GITOPS_GIT_ORG: the GitHub organization used for catalog discovery in RHDH.
export GITOPS_GIT_ORG="your-github-org-name"
export GITHUB_APP_CLIENT_ID="your-github-app-client-id"
export GITHUB_APP_CLIENT_SECRET="your-github-app-client-secret"
export GITHUB_APP_WEBHOOK_URL="your-github-app-webhook-url"
export GITHUB_APP_WEBHOOK_SECRET="your-github-app-webhook-secret"
export GITHUB_APP_PRIVATE_KEY="your-github-app-private-key"

# ArgoCD secrets
export ARGOCD_USER="your-user" # default value is "admin"
# NOTE: To be able to run all tests locally you'll need
# to export some dummy values for ARGOCD_PASSWORD, ARGOCD_HOSTNAME,
# ARGOCD_API_TOKEN.

# RHDH secrets
export BACKEND_SECRET="a-randomly-generated-string"
# RHDH_CLUSTER_ROUTER_BASE: the domain suffix of all OCP routes.
# This is used to derive the RHDH URL and is injected as an ArgoCD
# Helm parameter override for the clusterRouterBase chart value.
export RHDH_CLUSTER_ROUTER_BASE="apps.mycluster.openshift.com"

# RHDH Postgres Secrets - any valid password strings for these two env vars will do
export POSTGRESQL_POSTGRES_PASSWORD="your-preffered-postgres-pass"
export POSTGRESQL_USER_PASSWORD="your-preffered-user-pass"

# Quay.io secrets
# For more information on how to setup quay.io you check the doc
# here: https://github.com/redhat-ai-dev/ai-rhdh-installer/blob/main/docs/APP-SETUP.md#quay-setup
export QUAY_DOCKERCONFIGJSON="your-quay.io-dockerconfig.json"

# KeyCloak (RH SSO) secrets
# In this area you can point to an already existing keycloak instance.
#
# Hint:: As per RHDHPAI use case You could replace the values below with the ones in
# the Dev Instance: https://console-openshift-console.apps.rosa.redhat-ai-dev.m6no.p3.openshiftapps.com/
# Namespace: rolling-demo-ns and Secret: keycloak-secrets.
# Remember to Base64 Decode the values for the various environment variables stored in the `keycloak-secrets`.
export KEYCLOAK_CLIENT_ID="your-client-id"
export KEYCLOAK_CLIENT_SECRET="your-secret"
# KEYCLOAK_REALM: The realm you want to use for your deployment.
# Check more info for realms here: https://www.keycloak.org/docs/latest/server_admin/index.html#_configuring-realms
export KEYCLOAK_REALM="your-realm"
export KEYCLOAK_LOGIN_REALM=${KEYCLOAK_REALM}
export KEYCLOAK_METADATA_URL="https://your-keycloak-host/auth/realms/${KEYCLOAK_REALM}"
export KEYCLOAK_BASE_URL="https://your-keycloak-host/auth"

# Llama Stack secrets (for Lightspeed Core Service)
# Hint:: Per RHDHPAI case the Llama Stack tokens can be found in https://console-openshift-console.apps.rosa.redhat-ai-dev.m6no.p3.openshiftapps.com/
# Secret: llama-stack-secrets (keys are identical with the env var names below).
# Remember to Base64 Decode the values for the various environment variables stored in the `llama-stack-secrets`.
export VLLM_URL="https://meta-llama-31-8b-3scale-apicast-production.your-3scale-host:443/v1"
export VLLM_API_KEY="your-llama-stack-token"
# VALIDATION_PROVIDER: The LLM provider type - must be one of: vllm, ollama, openai
export VALIDATION_PROVIDER="vllm"
# VALIDATION_MODEL_NAME: The name of the model to use for validation
export VALIDATION_MODEL_NAME="llama-31-8b-version1"
# NOTEBOOKS_QUERY_PROVIDER_ID: The Llama Stack provider ID used to route notebook queries to the
# inference backend. Its set of values is similar to VALIDATION_PROVIDER.
export NOTEBOOKS_QUERY_PROVIDER_ID="vllm"
# NOTEBOOKS_QUERY_MODEL: The model identifier as exposed by Llama Stack. Note this may differ
# from the kserve deployment name or vLLM model path. It takes values similar to VALIDATION_MODEL_NAME.
export NOTEBOOKS_QUERY_MODEL="llama-31-8b-version1"

# Postgres secrets
export LIGHTSPEED_POSTGRES_PASSWORD="your-preffered-lightspeed-psql-password"
export LIGHTSPEED_POSTGRES_USER="your-preffered-lightspeed-psql-username"
export LIGHTSPEED_POSTGRES_DB="your-preffered-lightspeed-psql-dbname"

Installation

After configuring your scripts/private-env file, run the setup from the repository root:

# Full install (GPU + RHOAI + Model Catalog)
make install

# Lightweight install — no GPU, no RHOAI, no Model Catalog Bridge
make install-no-rhoai

make install-no-rhoai — lightweight install for smaller clusters

make install-no-rhoai sets SKIP_RHOAI_SETUP=true and runs the same setup.sh entry point as the full install. It is the right choice when your cluster has no GPU nodes or you do not need the Model Catalog Bridge.

What is skipped:

  • Node Feature Discovery (NFD) operator and instance
  • NVIDIA GPU operator and ClusterPolicy
  • ODH Kubeflow Model Registry setup (setup-rhoai.sh)
  • Model Catalog sidecars (location, storage-rest, rhoai-normalizer) and their RBAC in the deployed Helm chart

What is still installed:

  • OpenShift GitOps operator (ArgoCD)
  • OpenShift Pipelines operator and Pipelines-as-Code
  • RHDH (via ArgoCD + the rhdh-chart)
  • Developer Lightspeed and its PostgreSQL instance
  • AI Software Templates
  • All Kubernetes secrets, service accounts, and namespaces

private-env note for make install-no-rhoai:

ODH_SETUP_DIR is validated as a required variable even when SKIP_RHOAI_SETUP=true. Set it to any non-empty placeholder so the check passes:

export ODH_SETUP_DIR="n/a"

The setup.sh script automates the entire setup process by calling focused subscripts in order:

  1. install-operators.sh — installs the required operators (OpenShift GitOps, OpenShift Pipelines, Node Feature Discovery, NVIDIA GPU) and creates the NFD instance and NVIDIA ClusterPolicy. Skipped for NFD and GPU when SKIP_RHOAI_SETUP=true.
  2. setup-rhoai.sh — applies the ODH Kubeflow Model Registry kustomize and waits for the setup job to complete.
  3. setup-argocd.sh — retrieves ArgoCD admin credentials and generates an API token.
  4. setup-namespaces.sh — creates the RHDH and LightSpeed Postgres namespaces.
  5. setup-sa-tokens.sh — creates service accounts and generates their tokens.
  6. setup-secrets.sh — creates all required Kubernetes secrets in the target namespaces.
  7. setup-pipelines.sh — configures the Cosign signing secret and runs the Tekton pipeline setup.
  8. apply-argocd-application.sh — applies the ArgoCD Application with your configured GITOPS_REPO_URL, GITOPS_TARGET_REVISION, and RHDH_CLUSTER_ROUTER_BASE values.

Skipping steps

You can skip earlier steps if they have already been completed on your cluster:

  • SKIP_INSTALL_DEPS=true — skips all operator and instance installation.
  • SKIP_RHOAI_SETUP=true — skips NFD + GPU operator installation, RHOAI setup, and disables Model Catalog sidecars (location, storage-rest, rhoai-normalizer) and RBAC in the deployed chart. This is what make install-no-rhoai sets.

For example, to jump straight to the rolling demo preparation:

SKIP_INSTALL_DEPS=true SKIP_RHOAI_SETUP=true make install

Secondary instance

If there's an instance of RHDH Rolling Demo already existing on your cluster (with Cosign keys, TektonConfig, and Pipelines-as-Code secrets already configured), you can deploy an additional instance by providing a different namespace and ArgoCD application name:

RHDH_NAMESPACE=my-secondary-ns \
ARGOCD_APP_NAME=my-secondary-app-name \
IS_SECONDARY_INSTANCE=true \
SKIP_INSTALL_DEPS=true \
SKIP_RHOAI_SETUP=true \
make install

setup.sh will automatically compute RHDH_BASE_URL and RHDH_CALLBACK_URL from ARGOCD_APP_NAME, RHDH_NAMESPACE, and RHDH_CLUSTER_ROUTER_BASE.

When IS_SECONDARY_INSTANCE=true:

  • The Cosign signing secret is not regenerated.
  • The TektonConfig transparency.url is not patched.
  • The pipelines-as-code-secret and the LightSpeed Postgres secret in the PAC_NAMESPACE are not created.
  • The ArgoCD Application is deployed with global.isSecondaryInstance=true.

Optional overrides

The following variables have built-in defaults and do not need to be set in private-env unless you want to change them. All operator channel and CSV defaults are based on OCP 4.20:

Variable Default Description
ARGOCD_NAMESPACE openshift-gitops Namespace where ArgoCD (OpenShift GitOps) is installed.
PAC_NAMESPACE openshift-pipelines Namespace where OpenShift Pipelines and Pipelines-as-Code run.
LIGHTSPEED_POSTGRES_NAMESPACE lightspeed-postgres Namespace where the LightSpeed PostgreSQL instance is deployed.
GITOPS_OPERATOR_CHANNEL latest Subscription channel for the OpenShift GitOps operator.
GITOPS_STARTING_CSV openshift-gitops-operator.v1.19.1 Starting CSV for the OpenShift GitOps operator.
PIPELINES_OPERATOR_CHANNEL latest Subscription channel for the OpenShift Pipelines operator.
PIPELINES_STARTING_CSV openshift-pipelines-operator-rh.v1.21.0 Starting CSV for the OpenShift Pipelines operator.
NFD_OPERATOR_CHANNEL stable Subscription channel for the Node Feature Discovery operator.
NFD_STARTING_CSV auto-detected Starting CSV for the Node Feature Discovery operator. Auto-detected from the cluster catalog for the active NFD_OPERATOR_CHANNEL. Set explicitly to pin to a specific version.
GPU_OPERATOR_CHANNEL v25.10 Subscription channel for the NVIDIA GPU operator.
GPU_STARTING_CSV gpu-operator-certified.v25.10.1 Starting CSV for the NVIDIA GPU operator.

These can be set in private-env or passed directly on the command line:

PAC_NAMESPACE=my-pipelines-ns make install

Customize operator channels and CSV versions

To see the available subscription channels and current CSV for any operator, query the PackageManifest from your cluster:

# List all available channels for an operator
oc get packagemanifest <package-name> -n openshift-marketplace \
  -o jsonpath='{range .status.channels[*]}{.name}{"\n"}{end}'

# Get the current CSV for a specific channel
oc get packagemanifest <package-name> -n openshift-marketplace \
  -o jsonpath='{.status.channels[?(@.name=="<channel-name>")].currentCSV}'

Working from a fork

If you are working from a fork of this repository, set these env vars in your private-env:

export GITOPS_REPO_URL="https://github.com/your-user/ai-rolling-demo-gitops.git"
export GITOPS_TARGET_REVISION="main"

The setup.sh script uses yq to inject these values into gitops/application.yaml at apply time, so the file in git is not modified.

How clusterRouterBase is configured

The clusterRouterBase value in charts/rhdh/values.yaml is a Helm chart value and cannot use ${ENV_VAR} substitution at runtime. Instead, setup.sh injects it as an ArgoCD Helm parameter override when applying gitops/application.yaml. This means the value from RHDH_CLUSTER_ROUTER_BASE in your private-env is used without modifying any files in git.