v0.2.0

What's Changed

• build: upgrade scaffolding by @lsorber in #3
• feat: avoid unnecessary __len__ check by using is None for tokenizer by @lsorber in #6
• feat: support local tokenizers by @lsorber in #7
• fix: fix loading of local models by @lsorber in #8
• chore: copier update by @lsorber in #9

Full Changelog: v0.1.0...v0.2.0

v0.1.0

Initial release:

• Accelerated ONNX inference of wtpsplit's Segment any Text (SaT) models with minimal dependencies.
• Set treat_newline_as_space=True by default (segment-any-text/wtpsplit#146).
• Add improved hat weighting (segment-any-text/wtpsplit#147).
• Type annotations for the top-level interface.
• Tests to verify that wtpsplit-lite's output exactly matches wtpsplit's output.

Full Changelog: https://github.com/superlinear-ai/wtpsplit-lite/commits/v0.1.0