Search results

Per-category #1 rankings are within noise on their own samples (paired McNemar) — consider showing CIs / significance groups

Status: Open.

#269 In allenai/reward-bench;

· ipezygj opened on Jul 19, 2026

Focus subset is length-confounded: a content-blind "pick the shortest completion" baseline scores 52.7% (2.1× chance, z=14.2)

Status: Open.

#268 In allenai/reward-bench;

· ipezygj opened on Jul 17, 2026

alpacaeval-easy and alpacaeval-hard are length-degenerate: a 0-parameter "pick the longer answer" baseline scores 100%

Status: Open.

#267 In allenai/reward-bench;

· ipezygj opened on Jul 17, 2026

Related benchmark: REFUTE (scientific critique / calibration)

Status: Open.

#266 In allenai/reward-bench;

· connerlambden opened on Jul 14, 2026

[BOUNTY CLAIM] Calibration diagnostics #263

Status: Open.

#265 In allenai/reward-bench;

· nikos1005 opened on Jun 7, 2026

Adding calibration diagnostics for reward model evaluation.

Status: Open.

#263 In allenai/reward-bench;

· akshan-main opened on Jun 3, 2026

Add Unsloth Fast Inference option

Status: Open.

#251 In allenai/reward-bench;

· natolambert opened on Jun 12, 2025