Skip to content

Commit 000ccfa

Browse files
danesherbsdanesherbs
committed
Add eval spec and data
1 parent 2420c62 commit 000ccfa

File tree

523 files changed

+4096
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

523 files changed

+4096
-0
lines changed

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_a_digit.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:d4fec43a2a263077d155234768ea1d0c343bbc0e3481c9731852ad397a51d53c
3+
size 120510

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_a_french_word.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:57ea672108cab1c041d6a06f54ea7e45156c6883a5fd93255e7aa38da59f0def
3+
size 125583

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:040357e6052c43675c2958fd86d4a57590729ec9d77e7af6bdb4374fc1c821d9
3+
size 126035

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_word_w_and_a_digit.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:08c020dd0077216c8d00343eb706147d8700b8ccadbb9543ac46ef5782b68237
3+
size 119645

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_word_w_and_a_french_word.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:a7bf728474e25597255771a245e58b44c3b37d06ebf2b6597f3185caaf23ca9e
3+
size 126062

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_word_w_and_is_in_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:518f37b74214565bf084afbf7ff3740e546999fadae884d9a31f3b56086f3843
3+
size 126985

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_word_w_and_the_word_w_prime_in_sorted_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:14458f5dd2ba1eb41066fabb9a74092f7f9f8ed0ae35c4a1485ca4fe805d2749
3+
size 127497

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_word_w_at_least_k_times.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:21ada192939b7842509f0f3caf21a8609fb3971db3b68426cb80aad8a44a4fd0
3+
size 126436

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_word_w_between_k_and_k_prime_times.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:056f567542b54e7949306cf58dd8f6142f77ccc48fe5695f55265522b15f2854
3+
size 126936

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_word_w_followed_by_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:95c72b3614c576279a88bb8a8756a0ccab419bda0c86a6ecbda2a09925eb71ad
3+
size 127104

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_word_w_within_x_positions_of_the_word_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:88154d66fc12283b907dcd048c2d2e5e3d8ce32cb37c937dfb7e3e7ab919c948
3+
size 127413

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_words_w_and_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:7dad8a29fad9a5461cda8ff72301bfd6821b10bd3898d78c6e3c406ef78a9bab
3+
size 126195

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_words_w_or_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:28353a371eec50a940e85ea143ce84378389429714b2adcb0f756c190f9694f3
3+
size 126197

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/contains_the_words_w_or_w_prime_but_not_both.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:dc320821a7233d47bbcf07e0d3d76c06414480f3c43ddf5d0f54de076f9ef8e1
3+
size 126343

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/does_not_contain_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:e5973fe4af38d74191ceb9c7ebfd6866eb25993be4ec9bf777845e2c35af532e
3+
size 127327

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/does_not_start_with_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:76a4334e5a6d7658429e534074eee407b1a9d21a2fc0a26aea9f6a13d305c2e1
3+
size 125639

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/ends_with_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:f0b40f09bd12b85121f2c2751190d024e9892188033b4dee544ca818dba3b002
3+
size 127860

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/ends_with_the_word_w_or_contains_the_word_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:e88d3da7f44fed42cc4f9fcb61877c2da30f5f178f356f7f88ec932af1eb54e7
3+
size 128790

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/repeats_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:6ea2f7c69f03009f98a162d2fa48b1058e03dcc991f816875737d1ebee450378
3+
size 126244

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/repeats_the_word_w_and_contains_a_digit.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:cb82f2a9270d60c274720c88852e8a70937b92570e4607fc6be93ce1f291a5c7
3+
size 119878

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/repeats_the_word_w_and_contains_a_french_word.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:a07476cfd8bfef4c88ebac2f8235c86f58317832aebd5fd982c09494a2c6e684
3+
size 125736

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/repeats_the_word_w_and_contains_the_word_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:ca67f775619c06922449de794df12917e11b905db32cc19ef4b07cda0b04a3e7
3+
size 126492

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/repeats_the_word_w_and_is_in_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:2633e9db18be85448768f54c2189c7bb68298aa4151dd1058a79bf5c33101adb
3+
size 126636

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/repeats_the_word_w_and_starts_with_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:a1a61648d222b86ab272693a2fc4fe71f94362bf9f795bc950a1d20adaba2aa6
3+
size 126768

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/starts_with_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:44e3f61b36f77fc25ef950c5460dec0a81c8e72b063995d3496a5d3bdec03943
3+
size 125968

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/starts_with_the_word_w_or_repeats_the_word_w_prime_but_not_both.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:2a5335027e75eba4da5ecc9817f8596b12940271ffd434a7e274620eac9a92a8
3+
size 126626

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/word_w_appears_in_position_i.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:0c8f1f3e0a8d186c565b3dfc9c4b835d7631d4a47a38001c2b2f3e29d9e70fee
3+
size 126398

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/words_are_in_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:6d2ac176bdbba9c12ff3c5923c6f50c24197df9e9ecfb63e6581583f0459741b
3+
size 126623

‎evals/registry/data/articulate_rules/articulation/in-context/freeform/words_are_in_reverse_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:8c9f52d26f9f143a829aecb9603cd6903b0d53c9f7ef56e0aa1ccfc3b25e0e2a
3+
size 127602

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_a_digit.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:fb1b0f5ff585c0ad503e8ddee7ea34193386c296e031766afb8a7c5dbc450ed7
3+
size 122617

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_a_french_word.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:b0f7e5b75f1d0166b261c5c08d8fe348a9b3d778a3ecc0a68feb54b9c4c9cdb0
3+
size 127612

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:8484d792778c60122e5faf37d8a6d0c1871a8efb4403af73c22d12254eec4ec1
3+
size 128202

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_word_w_and_a_digit.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:10b491131733aaafae084f479a75a0ecc63b2678242a5bdd1f7a18522e3d83e2
3+
size 122166

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_word_w_and_a_french_word.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:2b81723982dd34c058e9d33ea6f077bb6f6cfe22d170df5b263cf65bf0900678
3+
size 128364

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_word_w_and_is_in_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:daa37092fa53284e969c9b421fb9573334796347ee19fc89eb6f3911483b0336
3+
size 129076

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_word_w_and_the_word_w_prime_in_sorted_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:e9b3ce605417928913e9c5c5f3b8a26319726fbdf838bce307b208588044fb25
3+
size 129651

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_word_w_at_least_k_times.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:8318f843a16fb85a168f8bd019a13da6e868a313ba580910ddd13c2fee5ad5d0
3+
size 129303

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_word_w_between_k_and_k_prime_times.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:d79bccfd3465a82ef893d93cff3248e228f0281345656a283df3c106c0e0a9a3
3+
size 129516

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_word_w_followed_by_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:c78d8595baa93851b3f511906242e4d31d66927e1e8776f1a3d84fe1bef4e0a2
3+
size 129352

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_word_w_within_x_positions_of_the_word_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:a21d840761e8bf2d2c3b9a92eae7c11c3802a6ba16d3767541617d0cf0e0451e
3+
size 129705

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_words_w_and_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:dfe01a9c91e7a22a7cabc2fa1189c7f9c6df0fc0df669c9274caff30549caead
3+
size 128904

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_words_w_or_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:71bbc202a00b91f3b898c55dbedef1bcdf3d6f751d2e78645791be2ce1521aa5
3+
size 128678

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/contains_the_words_w_or_w_prime_but_not_both.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:c55544592f826a0a996aff0ded31878e15021b109dfc1002257143e43ed15da7
3+
size 128454

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/does_not_contain_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:d60aaff4960b35c8c86a4ebc0ad5abd534672de51d473f6abd3b1d53a4d67e66
3+
size 129777

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/does_not_start_with_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:a44b29c0b86841f11e3394c9701738ec9c9836b907930879a6b37dc49325740a
3+
size 128205

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/ends_with_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:0ffb30f245094281d0bf8bcad0e77bcf055142e7532d02d23636719de8ace891
3+
size 129821

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/ends_with_the_word_w_or_contains_the_word_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:7e0964514d617dc6719407bbf4bfad098395bd6d0178a3469a11ef6061150591
3+
size 131767

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/repeats_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:1c7b413ec0e7bbeaac0351885f11840f2341c73b1674e91b1a10791b35507cbd
3+
size 128113

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/repeats_the_word_w_and_contains_a_digit.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:8d563b109eda8dfe1e1d3737ac94a14f7cd277aa2149235ca28a51678d5d7f39
3+
size 121953

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/repeats_the_word_w_and_contains_a_french_word.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:83fe69d022643e76d6769f0e74785867ad34a7044bcfd12d3e9dd825fde78029
3+
size 128390

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/repeats_the_word_w_and_contains_the_word_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:34c5649d2ffb20b121c1a0393d2d771143517b3e939c9c85de61612fb305d53b
3+
size 129371

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/repeats_the_word_w_and_is_in_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:ba68d3996aa43c61cd77c35f1f6d48c385f86eb81a033f48b253decdc47220e9
3+
size 129282

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/repeats_the_word_w_and_starts_with_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:f23c4df5670cd99ba33c64a8e32deb513b8e49e35e82dd93bc60a71385192c35
3+
size 129532

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/starts_with_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:76033037e02e23183468deccd2c5c80e02b75765dcf6ea4b599fb50aacbb66b8
3+
size 128404

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/starts_with_the_word_w_or_repeats_the_word_w_prime_but_not_both.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:02001a6e7b12696d8eb6b9ff05dc4be4423c41160cf6fc5ce9bf5af97df07037
3+
size 129060

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/word_w_appears_in_position_i.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:635d2316c2839893b269330a2f6dc0bf14e3284433cdfcccb9ecc60cace7c871
3+
size 128903

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/words_are_in_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:ac5d2f7a0addd5aecfc0addbf618d779dcefa34482d5bdcfac621115620d4a21
3+
size 129074

‎evals/registry/data/articulate_rules/articulation/in-context/multiple-choice/words_are_in_reverse_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:8ce40f914d43fafb46c1c01648bb549b425d5c62a81d73e4fde0fdd7ace9e1bc
3+
size 129751

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_a_digit.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:9b3406f320d737b20f72142ec8c60935273347bc28c7217342e2204b0d8b9e1e
3+
size 116431

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_a_french_word.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:ca1add7f1cfd922b263bd74e3f30e3e1a29ffe94c2372eb29806322e3e689308
3+
size 121583

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:5a05395d2d4013fb0842f1df44f7df0c6177c71025a3008ff6a02a0ec56d161c
3+
size 121671

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_word_w_and_a_digit.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:e1c34ab1afde1888bd132135668578af70d929ca2fca798c55d2a54d51bc3978
3+
size 115647

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_word_w_and_a_french_word.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:deb4bc614b27b68eaa49aa65875bdda2ec8307ea964293e08f92317cff552e50
3+
size 121081

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_word_w_and_is_in_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:979b8fbb74c4eb3000cb5849a982da9e1b52fa23e67fe8d864293bc132015921
3+
size 122135

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_word_w_and_the_word_w_prime_in_sorted_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:57a3b939192a47d3c05f4a56e62fa99c191d450df3e23500aee1a44047a49bc3
3+
size 122049

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_word_w_at_least_k_times.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:3c6812322a8cfa59c10ef743c44f9f9a7e2b7ba418cb4b71800ccaa99b23e100
3+
size 121901

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_word_w_between_k_and_k_prime_times.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:da1c1529034e2c4c40cfffbdb10dbb4e2af607d3e9f8fa6d7d45acb5c3d53b0b
3+
size 123051

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_word_w_followed_by_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:45ce44b4c357e5ea07689ee68f8393d1a7dfac47378af299d754406fd9c758f8
3+
size 122415

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_word_w_within_x_positions_of_the_word_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:8780ee273f62bd7f3c164f6c326182235786c6bb3a9d16a3b8a418f14d2e8cce
3+
size 121899

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_words_w_and_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:8ec6cf3cca60c0391478e5a9092315b86a533bf47e90c0a15b3cae87af48e0ef
3+
size 121720

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_words_w_or_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:8e03907ebbf913fc768b5968e3b012499d387cbaface5b32f67144fcadd01cda
3+
size 121411

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/contains_the_words_w_or_w_prime_but_not_both.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:4340123e19ab91375449aabbfaac06cba18bfc9686bd25302c34f9b0d1ca8bf3
3+
size 121691

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/does_not_contain_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:e180fbdaf3cb4fce6b6197a04b41168d8cf0e26cfaa6f2c2ce33a3b4ba5f6252
3+
size 122619

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/does_not_start_with_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:f0db4922b43c352ad77913d6e1469ab8de9c5b15434f0a620078e2be774b210e
3+
size 121309

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/ends_with_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:a1d48669b18beba505346e94c1fcc29afd7291d3e6bcd66fdeaf8502da8e65b4
3+
size 123862

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/ends_with_the_word_w_or_contains_the_word_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:2d325cad63ade27aa6defc29c266973bea9aaee8a96626ba29390117d23fee8f
3+
size 124033

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/repeats_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:d55788e0ae4e42ec2e14db3e5d04ffae25faab73fe63c3dbf132a0211917a299
3+
size 121954

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/repeats_the_word_w_and_contains_a_digit.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:124e0f364759629afb835aa9548cd0d036a9f7d3419488daddf47fd0e63e759b
3+
size 115515

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/repeats_the_word_w_and_contains_a_french_word.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:f51be4a9331e9ae264a8bafb8caa5de01e3e137dd51763ebeb515f2d87e7eec2
3+
size 121254

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/repeats_the_word_w_and_contains_the_word_w_prime.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:b8475270851fea7833a2ee6bb91c8a847249a4191414ca45f56f2356c8e52201
3+
size 122048

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/repeats_the_word_w_and_is_in_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:ce53357422cf39626d8ca3f02f548f1e6d3111ea62b5daf7f6de5a6df240d7d4
3+
size 122245

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/repeats_the_word_w_and_starts_with_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:0ab72f71f5b037c0ea1bfb36447991c82d6c2f5299980dbe91f762b7598667e7
3+
size 121962

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/starts_with_the_word_w.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:040858dc0f1bd6a339378ead7ef94019c2976b679c7cb5e51ef164b10c755f6e
3+
size 122116

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/starts_with_the_word_w_or_repeats_the_word_w_prime_but_not_both.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:b445f8fe40ca9aea835c8471eba0cc902aff6ab70972a9df96bfe7b03fdbc13e
3+
size 121121

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/word_w_appears_in_position_i.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:e35fa2e4ad03bc5f91b469ecbc092012bcf14729c3ae845efbc6c25112f898c7
3+
size 122406

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/words_are_in_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:d09a2bc9c5e7fec7aa31b1f3c05f6f0470aea3b5e730bcd9eab5468801db2631
3+
size 122383

‎evals/registry/data/articulate_rules/classification/in-context/in-distribution/words_are_in_reverse_alphabetical_order.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:25e11a46c0c9c8bdd6353df52d67f15195abda0dae0bf6f2ea71b35f71ac22d3
3+
size 122448

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/ChangeCaseAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/ChangeLanguageOfWordAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/ChangePositionOfWordAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/CommonMisspellingsOfWordAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/DecreaseNumberOfWordsAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:19df46aad596e341e4d50225244bac69adbd00e76f43da85ad4e7237f13b53db
3+
size 120390

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/DifferentPartOfSpeechAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/IncreaseNumberOfWordsAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:0a824fc6f719e7432c11df144b04aaab9657fe4c566528f564dcaa4c79b880f2
3+
size 115029

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/InfillTrickAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:2c48fb6f056365f3987e09a6702a5611a342dd7cbe32c4c4ad98dc5a6e71ae9a
3+
size 117989

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/InsertRandomCharactersAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:14fa35236d8987496c008df0edc4b69f0144a1de5389d82a7f54924e5acaf9a2
3+
size 117253

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/InsertSpacesAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/InstructionInjectionAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:cb8f17f674163880870707a079a09bf7fa517e1aa62e749ef2567c2c0a8f353b
3+
size 113743

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/JumbleLettersAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/MarkdownStylingAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/RemoveSpacesAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:a84801411353ec134cfeae4ecd49105dfae923fd3c00476fd2e6595d5b0cff4a
3+
size 118456

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_digit/SynonymAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/ChangeCaseAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/ChangeLanguageOfWordAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/ChangePositionOfWordAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/CommonMisspellingsOfWordAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/DecreaseNumberOfWordsAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:7e296184ee5d10511005588d357dbe5c4180617e806935ff5a5128dcbf6abaef
3+
size 125220

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/DifferentPartOfSpeechAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/IncreaseNumberOfWordsAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:7fb934e49612634ffb443f800134b296fcfcf16c92f877ede4285cc2afe5ccd4
3+
size 120287

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/InfillTrickAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:88006dbb39da35548741b15b9a19c0eeb13c1aed2e6139f4f273bec84e1bb7ed
3+
size 122612

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/InsertRandomCharactersAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:dc1260241675012d6c6aa57a030358e37a802af9079c3f1dc5201c31d80bc5fe
3+
size 122703

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/InsertSpacesAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/InstructionInjectionAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:5ea979888b6252f63dfb5c653f98be2d1579c9995a4959e220fa3f68154c3579
3+
size 119149

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/JumbleLettersAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/MarkdownStylingAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/RemoveSpacesAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:116fb40db0f6db6fcf2923e4924dfaa083f3a1d34f4808deb59a3f55deddb134
3+
size 123526

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_a_french_word/SynonymAttackBehaviour.jsonl

Whitespace-only changes.

‎evals/registry/data/articulate_rules/classification/in-context/out-of-distribution/contains_the_word_w/ChangeCaseAttackBehaviour.jsonl

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:7b764e6fa2de347cc8dc7fa0e562f42bca58ae83fbb746858348bf970f4e605a
3+
size 122246

0 commit comments

Comments
 (0)