Description
When an aside tag contains text that the parser transformed — smart punctuation or an entity reference — the source range of the aside's remaining text points at the wrong characters.
BlockQuote.parseAsideTag(tagRequirement:) computes how far to move the start of the text's range with
let shiftCount = kindTag.utf8.count + 1 + ...
That counts UTF-8 bytes of the parsed string, but SourceLocation.column counts UTF-8 bytes of the source. The two differ whenever cmark rewrites the text on the way in:
| source |
parsed |
delta |
-- (2 bytes) |
– (3 bytes) |
+1 |
" (1 byte) |
“ (3 bytes) |
+2 |
& (5 bytes) |
& (1 byte) |
−4 |
Reproduction
Parsing each of these as an Aside and inspecting the range of the first Text node of the aside's content (swift-markdown at 27b7fc1):
| source |
text |
range |
source at that range |
> Note: text after |
"text after" |
1:9..<1:19 |
"text after" ✅ |
> Before -- after: text |
"text" |
1:21..<1:24 |
"ext" ❌ |
> "Note": text after |
"text after" |
1:15..<1:21 |
" after" ❌ |
> Note&more: text after |
"text after" |
1:14..<1:28 |
"re: text after" ❌ |
Expected behavior
The range should cover the text it belongs to, so that consumers which map ranges back to the source (diagnostics, for example) point at the right characters.
Notes
This is the same arithmetic that used to trap with Range requires lowerBound <= upperBound (#231, and the crash fixed in #272). The clamp added in #272 keeps the range valid, so this is no longer a crash — the remaining problem is only that the range can be inaccurate.
Two things that came up while looking at this, in case they're useful:
- An exact fix doesn't look possible from inside
parseAsideTag, because the markup tree keeps only parsedRange and not the source text, and the transformations can't be reversed unambiguously (a parsed & may come from &, &, & or &).
- Anchoring the new start at the end instead —
upperBound.column - trimmedText.utf8.count — fixes all of the rows above and keeps the empty-content case exact, but it moves the error to asides whose content was transformed, such as > Note: "quoted", which is likely the more common shape. So it doesn't look like a clear win over the current heuristic.
Environment
swift-markdown main at 27b7fc1, macOS, Swift 6.3.3.
Description
When an aside tag contains text that the parser transformed — smart punctuation or an entity reference — the source range of the aside's remaining text points at the wrong characters.
BlockQuote.parseAsideTag(tagRequirement:)computes how far to move the start of the text's range withThat counts UTF-8 bytes of the parsed string, but
SourceLocation.columncounts UTF-8 bytes of the source. The two differ whenever cmark rewrites the text on the way in:--(2 bytes)–(3 bytes)"(1 byte)“(3 bytes)&(5 bytes)&(1 byte)Reproduction
Parsing each of these as an
Asideand inspecting the range of the firstTextnode of the aside's content (swift-markdown at 27b7fc1):> Note: text after"text after"1:9..<1:19"text after"✅> Before -- after: text"text"1:21..<1:24"ext"❌> "Note": text after"text after"1:15..<1:21" after"❌> Note&more: text after"text after"1:14..<1:28"re: text after"❌Expected behavior
The range should cover the text it belongs to, so that consumers which map ranges back to the source (diagnostics, for example) point at the right characters.
Notes
This is the same arithmetic that used to trap with
Range requires lowerBound <= upperBound(#231, and the crash fixed in #272). The clamp added in #272 keeps the range valid, so this is no longer a crash — the remaining problem is only that the range can be inaccurate.Two things that came up while looking at this, in case they're useful:
parseAsideTag, because the markup tree keeps onlyparsedRangeand not the source text, and the transformations can't be reversed unambiguously (a parsed&may come from&,&,&or&).upperBound.column - trimmedText.utf8.count— fixes all of the rows above and keeps the empty-content case exact, but it moves the error to asides whose content was transformed, such as> Note: "quoted", which is likely the more common shape. So it doesn't look like a clear win over the current heuristic.Environment
swift-markdown main at 27b7fc1, macOS, Swift 6.3.3.