[Question] How to make a submission?

[LC-John]

Hi, there! I find this benchmark quite valuable for front end coding agent products.

I am wondering how to make a submission from the community?

Is submission viable through trajectories, just like SWE-bench?

Or is a production package necessary? Like... submitting runner.ts and cli.ts through PRs?

[LC-John]

Also, I have another question...

I am not sure that what is the correct metric for evaluation, since there are cases where I have building failures but passed tests...

Take problem 005-react-use-api for instance, I failed in building and linting stages, but I have got a passed test... I am not sure what is going on here...

| 005-react-use-api         | ❌ FAIL     | ❌    | ❌   | ✅   | 27.0s    |

005-react-use-api:
  Build: /next-evals-oss_benchmark/evals/005-react-use-api/output-pyagent/pnpm-lock.yaml
   * /PATH/TO/BENCHMARK/next-evals-oss_benchmark/evals/pnpm-lock.yaml
   * /PATH/TO/BENCHMARK/next-evals-oss_benchmark/pnpm-lock.yaml


Failed to compile.

./app/ClientComponent.tsx
5:67  Error: Unexpected any. Specify a different type.  @typescript-eslint/no-explicit-any

info  - Need to disable some ESLint rules? Learn more here: https://nextjs.org/docs/app/api-reference/config/eslint#disabling-rules

  Lint: 
`next lint` is deprecated and will be removed in Next.js 16.
For new projects, use create-next-app to choose your preferred linter.
For existing projects, migrate to the ESLint CLI:
npx @next/codemod@canary next-lint-to-eslint-cli .


./app/ClientComponent.tsx
5:67  Error: Unexpected any. Specify a different type.  @typescript-eslint/no-explicit-any

info  - Need to disable some ESLint rules? Learn more here: https://nextjs.org/docs/app/api-reference/config/eslint#disabling-rules