Skip to content

Commit cd543a1

Browse files
Add GPT 4.1 judge (#448)
Co-authored-by: Saurabh Shah <saurabhs@allenai.org>
1 parent f19c323 commit cd543a1

1 file changed

Lines changed: 16 additions & 0 deletions

File tree

  • src/alpaca_eval/evaluators_configs/weighted_alpaca_eval_gpt4.1
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
weighted_alpaca_eval_gpt4.1:
2+
prompt_template: "alpaca_eval_clf_gpt4_turbo/alpaca_eval_clf.txt"
3+
fn_completions: "openai_completions"
4+
completions_kwargs:
5+
model_name: "gpt-4.1-2025-04-14"
6+
max_tokens: 1
7+
temperature: 1 # temperature should be applied for sampling, so that should make no effect.
8+
logprobs: true
9+
top_logprobs: 5
10+
fn_completion_parser: "logprob_parser"
11+
completion_parser_kwargs:
12+
numerator_token: "m"
13+
denominator_tokens: ["m", "M"]
14+
is_binarize: false
15+
completion_key: "completions_all"
16+
batch_size: 1

0 commit comments

Comments
 (0)