Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It seems that the Uint Tests are not match with the code #2

Open
old-kai opened this issue Aug 24, 2023 · 1 comment
Open

It seems that the Uint Tests are not match with the code #2

old-kai opened this issue Aug 24, 2023 · 1 comment

Comments

@old-kai
Copy link

old-kai commented Aug 24, 2023

For example, see the python test split example in python_test0.jsonl:

{"id":6584,"buggy_submission_id":3590,"fixed_submission_id":3591,"problem_id":"p00000","user_id":"u009980501","buggy_code":"for i in range(1, 10):\n for j in range(1, 10):\n print(str(i) +"" +str(j) +"=" + str(ij))","fixed_code":"for i in range(1, 10):\n for j in range(1, 10):\n print(str(i) +"x" +str(j) +"=" + str(i*j))","labels":["literal.string.change","call.arguments.change","expression.operation.binary.change","io.output.change"],"change_count":1,"line_hunks":1}

the Unit Test in tests_all.jsonl file is :

{"id":6584,"problem_id":"p02860","input":"1\nz","output":"No"}

But obviously the Unit Test can not suit for the python code example.
Is there any misunderstanding for the dataset?

@furunkel
Copy link
Contributor

Hi, to match bugs with test cases the problem_id field should be used (as you can see, they don't match).
Please note that I'm currently revising the whole dataset, so the split is likely to change in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants