Skip to content

Commit 4ed2f6f

Browse files
Allow for evals with no args (#1517)
As raised in #1515, the `args` field of `EvalSpec` is optional. Therefore it is possible for evals with no args to exist. Here `args` is `None`. However, currently our [arg overriding code](https://github.com/openai/evals/blame/main/evals/cli/oaieval.py#L158) mistakingly does not support this API, since it assumes `args` is not `None`. This PR addresses the issue with an if statement. Fixes #1515
1 parent 20de8c5 commit 4ed2f6f

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

evals/cli/oaieval.py

+5-1
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,11 @@ def to_number(x: str) -> Union[int, float, str]:
155155
return {k: to_number(v) for k, v in str_dict.items()}
156156

157157
extra_eval_params = parse_extra_eval_params(args.extra_eval_params)
158-
eval_spec.args.update(extra_eval_params)
158+
159+
if eval_spec.args is None:
160+
eval_spec.args = extra_eval_params
161+
else:
162+
eval_spec.args.update(extra_eval_params)
159163

160164
# If the user provided an argument to --completion_args, parse it into a dict here, to be passed to the completion_fn creation **kwargs
161165
completion_args = args.completion_args.split(",")

0 commit comments

Comments
 (0)