Skip to content

Fix ASCII case-fold collision between distinct tchar punctuation bytes - #3713

Open
shoemoney wants to merge 1 commit into
apple:mainfrom
shoemoney:fix/header-name-case-fold-punctuation-collision
Open

Fix ASCII case-fold collision between distinct tchar punctuation bytes#3713
shoemoney wants to merge 1 commit into
apple:mainfrom
shoemoney:fix/header-name-case-fold-punctuation-collision

Conversation

@shoemoney

Copy link
Copy Markdown

Motivation:

ByteCollectionUtils.swift's compareCaseInsensitiveASCIIBytes folds case by
masking every byte with & 0xdf (clearing bit 0x20). That bit is what
distinguishes an ASCII lowercase letter from its uppercase form, but it is
also the only bit separating several unrelated punctuation bytes from one
another:

byte hex & 0xdf byte hex & 0xdf
^ 0x5E 0x5E ~ 0x7E 0x5E
[ 0x5B 0x5B { 0x7B 0x5B
] 0x5D 0x5D } 0x7D 0x5D
\ 0x5C 0x5C | 0x7C 0x5C
@ 0x40 0x40 ` 0x60 0x40

Every byte in the right-hand column masks down to the same value as its
left-hand partner, so the current implementation treats them as identical.
All ten of these bytes are legal tchar characters in an HTTP header field
name per RFC 7230 §3.2.6, and are accepted as such by NIO's own header-name
validator (HTTPHeaders+Validation.swift). The result: two genuinely
different header names collide.

var headers = HTTPHeaders()
headers.add(name: "X-Foo^Bar", value: "caret-value")

headers.first(name: "X-Foo~Bar")   // returns "caret-value" -- wrong header!
headers.contains(name: "X-Foo~Bar") // returns true -- wrong header!

Because compareCaseInsensitiveASCIIBytes backs String.isEqualCaseInsensitiveASCIIBytes,
which backs HTTPHeaders's name comparisons, this collision propagates to:

  • HTTPHeaders.first(name:)
  • HTTPHeaders.subscript(_:) / subscript(canonicalForm:)
  • HTTPHeaders.contains(name:)
  • HTTPHeaders.remove(name:)

Modifications:

Replace the unconditional & 0xdf mask with a helper that only clears bit
0x20 when the byte is actually an ASCII lowercase letter (a...z),
leaving every other byte — punctuation included — untouched:

@inline(__always)
private func uppercaseASCIILetter(_ byte: UInt8) -> UInt8 {
    switch byte {
    case UInt8(ascii: "a")...UInt8(ascii: "z"):
        return byte & 0xdf
    default:
        return byte
    }
}

This is a single range check (matches the UInt8(ascii:) range-switch idiom
already used elsewhere in NIOHTTP1, e.g. HTTPHeaderValidator.swift and
HTTPHeaders+Validation.swift), so it stays branch-light and does not
regress the hot path both the fast (withContiguousStorageIfAvailable) and
fallback (elementsEqual) comparison loops sit on.

Result:

Header names that differ only by one of these five punctuation pairs are no
longer treated as equal, while ASCII letter case-folding (the actual purpose
of the function) is unaffected.

Tests:

Added a test to Tests/NIOHTTP1Tests/ByteBufferUtilsTest.swift exercising
compareCaseInsensitiveASCIIBytes directly against all five colliding pairs
(plus a same-byte and letter-case sanity check), and a test to
Tests/NIOHTTP1Tests/HTTPHeadersTest.swift exercising the bug through the
public HTTPHeaders API (first(name:), contains(name:),
subscript(canonicalForm:), remove(name:)) with the ^/~ pair.

Confirmed both new tests fail against the old implementation (5 assertion
failures in the byte-level test, 4 in the HTTPHeaders-level test) and pass
after the fix, with the full NIOHTTP1Tests suite green.

Motivation:

compareCaseInsensitiveASCIIBytes(to:) in ByteCollectionUtils.swift folds
case by masking every byte with & 0xdf, clearing bit 0x20. That bit is
what separates an ASCII lowercase letter from its uppercase form, but it
is also the only bit separating five unrelated punctuation byte pairs
from one another: ^(0x5e)/~(0x7e), [(0x5b)/{(0x7b), ](0x5d)/}(0x7d),
\(0x5c)/|(0x7c), and @(0x40)/`(0x60). Masking a byte from any of the
right-hand column produces the same value as its left-hand partner, so
the comparison treats them as identical even though every one of these
ten bytes is a legal tchar character in an HTTP header field name per
RFC 7230 3.2.6, and is accepted as such by NIOHTTP1's own header name
validator. In practice this means two genuinely distinct header names,
e.g. X-Foo^Bar and X-Foo~Bar, compare equal, and the collision reaches
every HTTPHeaders API built on this comparison: first(name:), the
subscript family (including canonicalForm:), contains(name:), and
remove(name:).

Modifications:

Replace the unconditional & 0xdf mask with a small helper that only
clears bit 0x20 when the byte is actually an ASCII lowercase letter,
leaving every other byte untouched. The helper is a single UInt8(ascii:)
range switch, matching the idiom already used elsewhere in NIOHTTP1
(HTTPHeaderValidator.swift, HTTPHeaders+Validation.swift), so both the
fast withContiguousStorageIfAvailable loop and the elementsEqual
fallback stay branch-light.

Result:

Header names differing only by one of these five punctuation pairs are
no longer treated as equal, while ASCII letter case-folding is
unaffected. Added a byte-level test covering all five colliding pairs
plus same-byte and letter-case sanity checks, and an HTTPHeaders-level
test exercising the bug through first(name:), contains(name:),
subscript(canonicalForm:), and remove(name:). Both fail against the old
implementation and pass with the fix; the full NIOHTTP1Tests suite
passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant