Skip to content

Update unicode_data to Unicode version 18.0.0 - #162929

Open
Kmeakin wants to merge 1 commit into
rust-lang:mainfrom
Kmeakin:km/unicode-18
Open

Kmeakin wants to merge 1 commit into
rust-lang:mainfrom
Kmeakin:km/unicode-18

Conversation

@Kmeakin

@Kmeakin Kmeakin commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Had to add an extra array to L2Lut to account for characters that change plane when case-mapped (so far, only U+1DF95 LATIN SMALL LIGATURE LONG S WITH DESCENDER S).

Also made version() defer to ucd_parse::ucd_directory_version(), which is more robust to minor changes in the ReadMe.txt format.

@rustbot

rustbot commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

library/core/src/unicode/unicode_data.rs is generated by the src/tools/unicode-table-generator tool.

If you want to modify unicode_data.rs, please modify the tool then regenerate the library source file via ./x run src/tools/unicode-table-generator instead of editing unicode_data.rs manually.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Sep 17, 2026
@rustbot

rustbot commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

r? @JohnTitor

rustbot has assigned @JohnTitor.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: libs
  • libs expanded to 12 candidates
  • Random selection from Darksonn, JohnTitor, Mark-Simulacrum, clarfonthey, jhpratt

Had to add an extra array to `L2Lut` to account for characters that change
plane when case-mapped (so far, only U+1DF95 LATIN SMALL LIGATURE LONG S WITH
DESCENDER S).

Also made `version()` defer to `ucd_parse::ucd_directory_version()`,
which is more robust to minor changes in the ReadMe.txt format.
Comment on lines +412 to +417
if let Ok(idx) = l2_lut.exceptions.binary_search_by_key(&input_low, |&(p, _)| p) {
// SAFETY: binary search guarantees that the index is in bounds.
let &(_, output) = unsafe { l2_lut.exceptions.get_unchecked(idx) };
return Some(output);
}

@clarfonthey clarfonthey Sep 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to binary search through a list this small? It generates to only 0 or 1 item from what I can see.

View changes since the review

@clarfonthey

Copy link
Copy Markdown
Contributor

r? clarfonthey

I'll take this over since I already reviewed a similar PR that was closed. Knew about the exceptions, wasn't sure how exactly they should be handled, but I like this method.

@rustbot rustbot assigned clarfonthey and unassigned JohnTitor Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants