Breaking grammar when upgrading to 1.24.0

[julio4]

Validations

• Follow our Code of Conduct
• Read the Contributing Guide.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

Describe the bug

Without any changes in the grammar, it seems upgrading from 1.23 to ^1.24 cause unexpected breaking changes.
In my case it breaks the cairo highlighting.

It can be easily reproduced with tm-grammars previews: https://deploy-preview-100--textmate-grammars-themes.netlify.app/?theme=github-dark&grammar=cairo (from shikijs/textmate-grammars-themes#100) (shiki 1.20.0, but it works in 1.23.1 as well)

Compared to the current version:

Reproduction

textmate grammars preview

Contributes

• I am willing to submit a PR to fix this issue
• I am willing to submit a PR with failing tests

[slevithan]

Note: This is affecting both the Oniguruma and JavaScript engine.

Without any changes in the grammar, it seems upgrading from 1.23 to ^1.24 cause unexpected breaking changes. In my case it breaks the cairo highlighting.

But the Cairo grammar did change significantly (see shikijs/textmate-grammars-themes@71926b1), and based on the dates, it looks like this update shipped in Shiki 1.24.1.