You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/modules/ROOT/pages/lm-eval-tutorial.adoc
+15-11Lines changed: 15 additions & 11 deletions
Original file line number
Diff line number
Diff line change
@@ -250,10 +250,11 @@ spec:
250
250
logSamples: true
251
251
pod:
252
252
container:
253
-
env:
253
+
env: <1>
254
254
- name: HF_TOKEN
255
255
value: "My HuggingFace token"
256
256
----
257
+
<1> `spec.pod.env` fields are passed directly to the LMEvalJob's container as environment variables.
257
258
258
259
Or you can create a secret to store the token and refer the key from the secret object using the reference syntax:
259
260
@@ -465,23 +466,26 @@ spec:
465
466
value: "False"
466
467
- name: tokenizer
467
468
value: ibm-granite/granite-7b-instruct
468
-
env:
469
-
- name: OPENAI_TOKEN
470
-
valueFrom:
471
-
secretKeyRef: <2>
472
-
name: <secret-name> <3>
473
-
key: token <4>
469
+
pod:
470
+
container:
471
+
env:
472
+
- name: OPENAI_API_KEY <2>
473
+
valueFrom:
474
+
secretKeyRef: <3>
475
+
name: <secret-name> <4>
476
+
key: token <5>
474
477
----
475
478
<1> `base_url` should be set to the route/service URL of your model. Make sure to include the `/v1/completions` endpoint in the URL.
476
-
<2> `env.valueFrom.secretKeyRef.name` should point to a secret that contains a token that can authenticate to your model. `secretRef.name` should be the secret's name in the namespace, while `secretRef.key` should point at the token's key within the secret.
477
-
<3> `secretKeyRef.name` can equal the output of
479
+
<2> `OPENAI_API_KEY` values are passed directly to remote model servers, so they can also be used as general authentication bearer tokens.
480
+
<3> `env.valueFrom.secretKeyRef.name` should point to a secret that contains a token that can authenticate to your model. `secretRef.name` should be the secret's name in the namespace, while `secretRef.key` should point at the token's key within the secret.
481
+
<4> `secretKeyRef.name` can equal the output of
478
482
+
479
483
[source,shell]
480
484
----
481
485
oc get secrets -o custom-columns=SECRET:.metadata.name --no-headers | grep user-one-token
482
486
----
483
487
+
484
-
<4> `secretKeyRef.key` should equal `token`
488
+
<5> `secretKeyRef.key` should equal field name holding the token value, in this example `token`
485
489
486
490
487
491
Then, apply this CR into the same namespace as your model. You should see a pod spin up in your
@@ -497,7 +501,7 @@ To enable Kueue on Open Data Hub, add the following to your `DataScienceCluster`
0 commit comments