Skip to content

Aside content source ranges are inaccurate when the tag contains transformed text #285

Description

@maxches99

Description

When an aside tag contains text that the parser transformed — smart punctuation or an entity reference — the source range of the aside's remaining text points at the wrong characters.

BlockQuote.parseAsideTag(tagRequirement:) computes how far to move the start of the text's range with

let shiftCount = kindTag.utf8.count + 1 + ...

That counts UTF-8 bytes of the parsed string, but SourceLocation.column counts UTF-8 bytes of the source. The two differ whenever cmark rewrites the text on the way in:

source parsed delta
-- (2 bytes) (3 bytes) +1
" (1 byte) (3 bytes) +2
& (5 bytes) & (1 byte) −4

Reproduction

Parsing each of these as an Aside and inspecting the range of the first Text node of the aside's content (swift-markdown at 27b7fc1):

source text range source at that range
> Note: text after "text after" 1:9..<1:19 "text after"
> Before -- after: text "text" 1:21..<1:24 "ext"
> "Note": text after "text after" 1:15..<1:21 " after"
> Note&amp;more: text after "text after" 1:14..<1:28 "re: text after"

Expected behavior

The range should cover the text it belongs to, so that consumers which map ranges back to the source (diagnostics, for example) point at the right characters.

Notes

This is the same arithmetic that used to trap with Range requires lowerBound <= upperBound (#231, and the crash fixed in #272). The clamp added in #272 keeps the range valid, so this is no longer a crash — the remaining problem is only that the range can be inaccurate.

Two things that came up while looking at this, in case they're useful:

  • An exact fix doesn't look possible from inside parseAsideTag, because the markup tree keeps only parsedRange and not the source text, and the transformations can't be reversed unambiguously (a parsed & may come from &, &amp;, &#38; or &#x26;).
  • Anchoring the new start at the end instead — upperBound.column - trimmedText.utf8.count — fixes all of the rows above and keeps the empty-content case exact, but it moves the error to asides whose content was transformed, such as > Note: "quoted", which is likely the more common shape. So it doesn't look like a clear win over the current heuristic.

Environment

swift-markdown main at 27b7fc1, macOS, Swift 6.3.3.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions