Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[connect-tcp] Remove bare TCP payload mode #2943

Merged
merged 7 commits into from
Nov 18, 2024
Merged

[connect-tcp] Remove bare TCP payload mode #2943

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
98e6ef6
[connect-tcp] Remove bare TCP payload mode
bemasc Nov 8, 2024
9da3ff7
Rephrase forward-looking language
bemasc Nov 11, 2024
2bf3e3b
Clarifications for Piotr Sikora
bemasc Nov 12, 2024
57e8093
Be even more explicit on splitting and merging
bemasc Nov 12, 2024
1f82e24
Add FINAL_DATA
bemasc Nov 13, 2024
a89c5b5
Tweaks to FINAL_DATA
bemasc Nov 13, 2024
5aa262b
Remove FINAL_DATA
bemasc Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 20 additions & 17 deletions draft-ietf-httpbis-connect-tcp.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ HTTP clients can be configured to use proxies by selecting a proxy hostname, a p

The absence of an explicit origin for the proxy also rules out the usual defenses against server port misdirection attacks (see {{Section 7.4 of ?RFC9110}}) and creates ambiguity about the use of origin-scoped response header fields (e.g., "Alt-Svc" {{?RFC7838}}, "Strict-Transport-Security" {{?RFC6797}}).

Classic HTTP CONNECT requests cannot carry in-stream metadata. For example, the WRAP_UP capsule {{?I-D.schinazi-httpbis-wrap-up}} cannot be used with Classic HTTP CONNECT.

## Overview

This specification describes an alternative mechanism for proxying TCP in HTTP. Like {{?CONNECT-UDP}} and {{?CONNECT-IP}}, the proxy service is identified by a URI Template. Proxy interactions reuse standard HTTP components and semantics, avoiding changes to the core HTTP protocol.
Expand All @@ -62,41 +64,45 @@ This specification describes an alternative mechanism for proxying TCP in HTTP.

# Specification

A template-driven TCP transport proxy for HTTP is identified by a URI Template {{!RFC6570}} containing variables named "target_host" and "target_port". This URI Template and its variable values MUST meet all the same requirements as for UDP proxying ({{!RFC9298, Section 2}}), and are subject to the same validation rules. The client MUST substitute the destination host and port number into this template to produce the request URI.
A template-driven TCP transport proxy for HTTP is identified by a URI Template {{!RFC6570}} containing variables named "target_host" and "target_port". This URI Template and its variable values MUST meet all the same requirements as for UDP proxying ({{!RFC9298, Section 2}}), and are subject to the same validation rules. The client MUST substitute the destination host and port number into this template to produce the request URI. The derived URI serves as the destination of a Capsule Protocol connection using the Upgrade Token "connect-tcp" (see registration in {#new-upgrade-token}).

When using "connect-tcp", TCP payload data is sent in the payload of a new Capsule Type named DATA (see registration in {{data-capsule}}). The ordered concatenation of DATA capsule payloads represents the main payload data stream in any protocol where this is well-defined. Intermediaries MAY split or merge DATA capsules.

## In HTTP/1.1

In HTTP/1.1, the client uses the proxy by issuing a request as follows:

* The method SHALL be "GET".
* The request's target SHALL correspond to the URI derived from expansion of the proxy's URI Template.
* The request SHALL include a single "Host" header field containing the origin of the proxy.
* The request SHALL include a "Connection" header field with the value "Upgrade". (Note that this requirement is case-insensitive as per {{Section 7.6.1 of !RFC9110}}.)
* The request SHALL include an "Upgrade" header field with the value "connect-tcp".
* The request's target SHALL correspond to the URI derived from expansion of the proxy's URI Template.
* The request SHOULD include a "Capsule-Protocol: ?1" header.

If the request is well-formed and permissible, the proxy MUST attempt to establish the TCP connection before sending any response status code other than "100 (Continue)" (see {{conveying-metadata}}). If the TCP connection is successful, the response SHALL be as follows:

* The HTTP status code SHALL be "101 (Switching Protocols)".
* The response SHALL include a "Connection" header field with the value "Upgrade".
* The response SHALL include a single "Upgrade" header field with the value "connect-tcp".
* The response SHOULD include a "Capsule-Protocol: ?1" header.

If the request is malformed or impermissible, the proxy MUST return a 4XX error code. If a TCP connection was not established, the proxy MUST NOT switch protocols to "connect-tcp", and the client MAY reuse this connection for additional HTTP requests.

After a success response is returned, the connection SHALL conform to all the usual requirements for classic CONNECT proxies in HTTP/1.1 ({{!RFC9110, Section 9.3.6}}). Additionally, if the proxy observes a connection error from the client (e.g., a TCP RST, TCP timeout, or TLS error), it SHOULD send a TCP RST to the target. If the proxy observes a connection error from the target, it SHOULD send a TLS "internal_error" alert to the client, or set the TCP RST bit if TLS is not in use. These behaviors avoid truncation of transfers between the client and the target on vulnerable protocols (e.g., HTTP/1.1 without TLS) while preserving the confidentiality and integrity guarantees of the "https" scheme.

~~~
Client Proxy

GET /proxy?target_host=192.0.2.1&target_port=443 HTTP/1.1
Host: example.com
Connection: Upgrade
Upgrade: connect-tcp
Capsule-Protocol: ?1

** Proxy establishes a TCP connection to 192.0.2.1:443 **

HTTP/1.1 101 Switching Protocols
Connection: Upgrade
Upgrade: connect-tcp
Capsule-Protocol: ?1
~~~
{: title="Templated TCP proxy example in HTTP/1.1"}

Expand All @@ -109,8 +115,6 @@ In HTTP/2 and HTTP/3, the proxy MUST include SETTINGS_ENABLE_CONNECT_PROTOCOL in
* The :authority pseudo-header field SHALL contain the authority of the proxy.
* The :path and :scheme pseudo-header fields SHALL contain the path and scheme of the request URI derived from the proxy's URI Template.

From this point on, the request and response SHALL conform to all the usual requirements for classic CONNECT proxies in this HTTP version (see {{Section 8.5 of !RFC9113}} and {{Section 4.4 of !RFC9114}}).

A templated TCP proxying request that does not conform to all of these requirements represents a client error (see {{!RFC9110, Section 15.5}}) and may be malformed (see {{Section 8.1.1 of !RFC9113}} and {{Section 4.1.2 of !RFC9114}}).

~~~
Expand All @@ -120,11 +124,12 @@ HEADERS
:authority = request-proxy.example
:path = /proxy?target_host=2001%3Adb8%3A%3A1&target_port=443
:protocol = connect-tcp
capsule-protocol = ?1
...
~~~
{: title="Templated TCP proxy example in HTTP/2"}

## Use of Relevant Headers
## Use of Other Relevant Headers

### Origin-scoped Headers

Expand All @@ -145,13 +150,12 @@ Authentication to a templated TCP proxy normally uses ordinary HTTP authenticati

Clients SHOULD assume that all proxy resources generated by a single template share a protection space (i.e., a realm) ({{?RFC9110, Section 11.5}}). For many authentication schemes, this will allow the client to avoid waiting for a "401 (Unauthorized)" response before each new connection through the proxy.

## Use of the Capsule Protocol {#capsule-protocol}

When using the "connect-tcp" Upgrade Token, templated TCP proxies do not use the Capsule Protocol {{!RFC9297}}. Clients MAY request use of the Capsule Protocol by offering the Upgrade Token "connect-tcp-capsule" instead. When offering or accepting the "connect-tcp-capsule" Upgrade Token, clients and servers SHOULD include a "Capsule-Protocol: ?1" header to facilitate processing by intermediaries.
## Closing Connections

Clients of this specification MAY implement "connect-tcp", "connect-tcp-capsule", or both. Accordingly, a templated TCP proxy server MUST implement both Upgrade Tokens unless its use is restricted to a subset of compatible clients.
In each HTTP version, any requirements related to closing connections in Classic HTTP CONNECT also apply to "connect-tcp", with the following modifications:

When using "connect-tcp-capsule", TCP payload data is sent in the payload of a new Capsule Type named DATA ({{data-capsule}}). The ordered concatenation of DATA capsule payloads represents the main payload data stream in any protocol where this is well-defined. Intermediaries MAY split or merge DATA capsules. Endpoints MAY indicate a TCP connection error by sending an incomplete DATA capsule, as an alternative to using TCP, TLS, or HTTP stream errors.
* In HTTP/1.1, endpoints SHOULD send an incomplete DATA capsule to indicate receipt of a TCP connection error (e.g., a TCP RST or timeout). When a connection is terminated with any incomplete capsule, or with an error in the underlying connection (e.g. a TLS "internal_error" alert or TCP RST), the endpoint SHOULD send a TCP RST if the underlying TCP implementation permits it.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's "incomplete DATA capsule"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A DATA capsule is incomplete if it is truncated by the end of the underlying data stream. This concept is defined in RFC 9297, so I've added a reference here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if there is no data in-flight (i.e. an idle connection)? Should intermediaries inject malformed DATA frame?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the recommendation is to inject a truncated DATA capsule by sending a DATA capsule header indicating nonzero length, then close the stream without sending any payload.

An alternative would be to define a new capsule type that serves as an explicit error indication.

An explicit error indication is much more powerful, may support easier debugging of connection failures, and may be easier to implement on the sender side. However, it is more work to implement on the receiver side, and opens up a major design bikeshed. The parsimony of having fewer ways to fail also might increase the odds of those failures being handled correctly.

I'm open to either approach.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're now always using capsules, I think there's value in making this an explicit signal. How about adding a DATA_END capsule of length_zero and requiring to send that? If the stream closes without receiving that, then it means it was aborted.

Alternate encoding: is to have two capsule types for DATA and a FIN bit, like we do for STREAM frames in QUIC.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a bunch of different ways a TCP connection can end. At a minimum, there's the distinction between FIN and RST, but there's also timeout (dead air), interface going down, ICMP Unreachable, proxy-injected errors like "quota exceeded", etc. We have to decide how much detail to encode, and how to encode it.

We have to be able to emit those back to TCP endpoints, so the amount of details and bike-sheding that we can do here is limited.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DID SOMEONE SAY BIKESHED??? I'M ENTHUSIASTIC

I never liked the fact that reading 0 on a BSD socket means "end of stream" for TCP. It's odd, and inconsistent with other uses (e.g., for UDP it means "I received a UDP packet with no payload").

We should definitely separate between "clean termination" (TCP FIN) and "abortive termination" (TCP RST, timeout, etc.) but I don't think we need to separate RST from timeout. That would match what classic CONNECT does.

So, +1 for DATA capsule with a FIN bit :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A FIN bit adds overhead to every DATA capsule, which seems unappealing to me. A zero-length DATA capsule seems fine to me (AFAIK we don't have any other use for length=0), but otherwise I would define a new capsule type ... probably a type meaning "clean shutdown", so that underlying transport failures naturally are treated as "unclean".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I meant putting the FIN bit in the capsule type. That can be described as two separate capsules (DATA = 0xTBD, DATA_END=0xTBD+1) or as a bit (DATA = 0xTBD..0xTBD+1, lowest bit of capsule type means FIN) but they're the same thing on the wire

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I've written this in as a new FINAL_DATA capsule type.

* In HTTP/2 and HTTP/3, senders MAY use an incomplete DATA capsule to indicate a TCP connection error, instead of (or in addition to) the signals defined for TCP connection errors in Classic HTTP CONNECT. Recipients MUST recognize an incomplete DATA capsule as a TCP connection error.

# Additional Connection Setup Behaviors

Expand Down Expand Up @@ -216,16 +220,15 @@ Templated TCP proxies can make use of standard HTTP gateways and path-routing to

# IANA Considerations

## New Upgrade Tokens
## New Upgrade Token

IF APPROVED, IANA is requested to add the following entries to the HTTP Upgrade Token Registry:
IF APPROVED, IANA is requested to add the following entry to the HTTP Upgrade Token Registry:

| Value | Description | Reference |
| ----- | ----------- | --------- |
| "connect-tcp" | Proxying of TCP payloads | (This document) |
| "connect-tcp-capsule" | Proxying of TCP payloads in HTTP Capsules | (This document) |

For interoperability testing of this draft version, implementations SHALL use the values "connect-tcp-06" and "connect-tcp-capsule-06.
For interoperability testing of this draft version, implementations SHALL use the value "connect-tcp-07".

## New MASQUE Default Template {#iana-template}

Expand All @@ -241,7 +244,7 @@ IF APPROVED, IANA is requested to add the following entry to the "HTTP Capsule T

| ----- | ------------ | --------- | ------------------------------------- | ----------------- | ------- |
| Value | Capsule Type | Status | Reference | Change Controller | Contact |
| (TBD) | DATA | permanent | (This document), {{capsule-protocol}} | IETF | HTTPBIS |
| (TBD) | DATA | permanent | (This document), {{specification}} | IETF | HTTPBIS |

For this draft version of the protocol, the Capsule Type value `0xb739a6d0` shall be used provisionally for testing, under the name "DATA-05".

Expand Down