Tokenizer v0.2.6

meili-bot released this 10 Nov 14:02

· 699 commits to main since this release

a69bb0c

Changes

Test Meilisearch issue 1714 (#58) @ManyTheFish
Please exclude Hangul from is_cjk. (#60) @datamaker
Add mapping between bytes in original word and normalized word (#59) @Samyak2

Thanks again to @ManyTheFish, @Samyak2, @datamaker and JB! 🎉

Contributors

ManyTheFish, datamaker, and Samyak2

Assets 2