Skip to content

Commit

Permalink
Update charabia/src/segmenter/utils.rs
Browse files Browse the repository at this point in the history
Co-authored-by: Many the fish <[email protected]>
  • Loading branch information
luflow and ManyTheFish authored Sep 9, 2024
1 parent 61634c9 commit d65941f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion charabia/src/segmenter/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ impl<'fst> FstSegmenter<'fst> {
// ensure the length is a valid character boundary
length = to_segment
.char_indices()
.nth(length)
.find(|(idx, _)| idx >= length)
.map(|(idx, _)| idx)
.unwrap_or(to_segment.len());

Expand Down

0 comments on commit d65941f

Please sign in to comment.