Similar to Content-Type, this probably needs to be restricted to content that is CORS-enabled or same-origin. That's not a limitation for the dictionary compression encodings (as they have similar restrictions). This might be a limitation for non-dictionary ztd/brotli.

Discussed on the Feb 29, 2024 W3C WebPerf call:

Summary:

• General agreement this would be useful for RUM providers
• This is similar to work that has been merged in for Content-Type (behind a flag in Chromium)
• Chrome net team will be picking up the spec work (in Fetch and ResourceTiming) to complete this and get it into Chromium

Filed chromium side bug. https://crbug.com/327941462

+CC: @Jxck who is interested in this.

Thanks @horo-t I'm happy to work on it !
Let me check.

Updated explainer can be found here:
https://github.com/guohuideng2024/resource-timing/blob/gh-pages/Explainers/Content_Encoding.md

I plan to filter the contentEncoding value before exposing it in resource timing. ( The unfiltered value in the response header remains for other uses), but how exactly the value should be filtered?

Continuing the discussing in whatwg/fetch#1742.

Below are the possible values that are registered. I am putting them in the following categories:

-- definitely allowed (confirmed that Chromium supports in WPT tests)
br
dcb
dcz
deflate
gzip
zstd

-- maybe allowed?
compress

-- What about these? Maybe not allow? why?

exi
identity
pack200-gzip
aes128gcm

-- The below are deprecated. Since we are building new feature, should we just disallow them( forcing the app to change to the up-to-date name)? Or should be be forgiving and transfer them to compress and gzip?

x-compress Deprecated (alias for compress)
x-gzip Deprecated (alias for gzip)

-- What about these? They are not registered contendEncoding values, should we disallow them?
br-d
zstd-d

++++++++++++++++
A disallowed value will be exposed as unknown
++++++++++++++++
Thanks!

Based on what Patrick said above and some digging, I think the following could be filtered out:

aes128gcm: looks like this is only used for "pushing messages", not for receiving resources via fetch
compress, exi, pack200-gzip: not supported in chromium
br-d, zstd-d, x-compress and x-zip: they are deprecated name
identity: it's just a reserved name

And the following will be supported:
br, dcb, dcz, deflate, gzip, zstd

(last updated 2024/12/11)

I think we must also specify what would be the value in case there is no content encoding (i.e. uncompressed payload).
It should be differentiated from the unknown case so we can track these separately. An example use case is when a CDN process decides to no compress the payload due to some issue (a real scenario I had). It is important to being able to identify these cases.

Seems easiest to just not set the value in that case so that when the property is read it will be JS undefined. However, this will harm supportability discovery since it will not be possible to determine if there was no encoding or the feature is not supported / or allowed by the UA.

My suggestion is for a specific string value in that case:
undefined / uncompressed / no-encoding

Anyone has a preference or other suggestions?

"identity" seems appropriate IMO.

Thanks for pointing out the "identity" value meaning Yoav. I missed it. :)

And I think I could mention the "identity" value too in the fetch standard modification.
I am going to modify the CLs and PRs accordingly, and I will soon start reviewing process after that.

Do we need to decipher between "identity" and "no content-encoding header" (an empty string in contentEncoding)? Unlike unknown, those sound like the same thing to me from a client perspective?

@yoavweiss

Noam point out this spec:
https://httpwg.org/specs/rfc9110.html#rfc.section.8.4
actually says that identity shouldn't be allowed in response header:

"Note that the coding named "identity" is reserved for its special role in Accept-Encoding and thus SHOULD NOT be included."

If we cannot report identify value in resource timing, the client is not able to tell between:
a) no compression is used, and the server really likes to tell the client that no compression is used
(in this case, returning identify value is idea)
b) the server didn't update to new fetch standard so although it's using some compression, it doesn't tell the client.
(in this case, resourceTiming returning empty string)

So it appears to me that it's desirable that:
(This is actually what submitted to chromium)

1. identity is allowed in fetch response, and such value is reported in resourceTiming
2. if server doesn't send contentEncoding value, such field is empty in resourceTiming. It means the value is missing, and CDN statistics and analysis will treat them accordingly.

==>
Then we will need to make change to #https://httpwg.org/specs/rfc9110.html#rfc.section.8.4?

@yoavweiss

Noam point out this spec: https://httpwg.org/specs/rfc9110.html#rfc.section.8.4 actually says that identity shouldn't be allowed in response header:

"Note that the coding named "identity" is reserved for its special role in Accept-Encoding and thus SHOULD NOT be included."

If we cannot report identify value in resource timing, the client is not able to tell between: a) no compression is used, and the server really likes to tell the client that no compression is used (in this case, returning identify value is idea) b) the server didn't update to new fetch standard so although it's using some compression, it doesn't tell the client. (in this case, resourceTiming returning empty string)

What does it mean "The server didn't update to the new fetch standard"? Fetch is a client-side standards that works on top of HTTP. If the server didn't send content-encoding doesn't it effective means there is no compression, as in "it's sending the identity content"?

Im working on other things related to HTTP content-encoding. The feedback I've received is to NOT use identity as label for message content that does not have a coding.

RFC 9110 reflects the reality of use in the wild, that implementations SHOULD NOT include "identity" in Content-Encoding but they might.

Therefore, my suggestion would be to use a label like unencoded (or no-encoding or empty string, but NOT uncompressed l) when there is no Content-Encoding response header. Use identity if there really is a Content-Encoding: identiy response header field.

I believe empty-string is consistent with other no-value cases in the resource timing spec (e.g. deliveryType, nextHopProtocol). Perhaps that's good enough here? unknown can still be used as a catch-all for an encoding type that is not in the supported list of encodings by the UA.

Thanks for the input! So, the reality is that most of the servers just send empty string, instead of a literal string identity.

Let me list the possible situation here and could you guys please verify:

1. if no header is sent at all --> return empty string
2. if empty header is sent --> return empty string
3. Otherwise, at least one coding is sent. A coding would be transferred to unknown if it's not recognized.

{gzip, apple, identity, pear } will be filtered to
{gzip, unknown, identity, unknown}
(because @LPardue said if the server did send identity we can keep it?)

or we filter it to
{gzip, unknown, unknown, unknown} with identity filtered too?

Thanks for the input! So, the reality is that most of the servers just send empty string, instead of a literal string identity.

Let me list the possible situation here and could you guys please verify:

1. if no header is sent at all --> return empty string
2. if empty header is sent --> return empty string
3. Otherwise, at least one coding is sent. A coding would be transferred to unknown if it's not recognized.

{gzip, apple, identity, pear } will be filtered to {gzip, unknown, identity, unknown} (because @LPardue said if the server did send identity we can keep it?)

or we filter it to {gzip, unknown, unknown, unknown} with identity filtered too?

We don't want this to become a little side-channel where servers can encode information by sending mutltiple encodings over a 1-byte fetch or so.

So I think it should be:

• limited to maximum 2 codings. more than that becomes multiple. (Can be 3 if there is a common use case?)
• string separated rather than an array
• if one of them is unknown, the whole string become unknown
• identity turns to unknown
• No-header or empty-string turns into the empty string

Examples: gzip, gzip deflate, br, unknown, multiple, "" etc.

Adding to what Noam said above:

1. the filter results in a string nicely formatted, separated by a comma and a space, examples:

""
"gzip" // only one coding
"gzip, deflate" // two codings
"multiple" // 3 or more
"unknown" // there is something unrecognized

2. empty string sent from server along with other coding values will be ignored

below is the conversion from the raw string from server, to the string reported in resourceTiming:

", gzip" ==> "gzip"
", gzip, deflate" ==> "gzip, deflate"

3 What if duplicated values from server?
(the server shouldn't do that but I guess we still need to define the rule here?)

"gzip, Gzip" => "gzip"?
"gzip, Gzip, deflate" => "gzip, deflate"?
"gzip, deflate, gzip" => "gzip, deflate" or "multiple"?

Adding to what Noam said above:
3 What if duplicated values from server? (the server shouldn't do that but I guess we still need to define the rule here?)

"gzip, Gzip" => "gzip"? "gzip, Gzip, deflate" => "gzip, deflate"? "gzip, deflate, gzip" => "gzip, deflate" or "multiple"?

To be more concrete, these are th values used in this header in November 2024 in more than 100 responses (HTTP archive, mobile, out of ~1.5M):

COUNT | encoding
-- | --
819,032,384 | <empty>
369,685,911 | gzip
359,144,061 | br
29,541,044 | zstd
107,020 | deflate
75,470 | none
29,740 | base64
19,778 | UTF-8
13,586 | identity
11,715 | utf-8
8,247 | webp
7,788 | dcb
3,907 | text
1,955 | GZIP
1,768 | 7bit
1,323 | nosniff
925 | ISO-8859-1
873 | x-gzip
732 | null
689 | utf8
639 | None
488 | binary
454 | image/webp
408 | gzip, gzip
295 | GP5
192 | br, gzip
187 | image/jpeg
134 | application/javascript
134 | compress
107 | Stream

Seems like using multiple values is very rare in practice. Perhaps supplying just unknoen, one value or multiple?

I can't think of a valid use case for multiple values. I actually now wonder what does the browser do with e.g. br, gzip.
The content is actually encoded with either one or the other.

Ideally, we'd report the value that was actually respected by the browser. Is that the first?

I can't think of a valid use case for multiple values. I actually now wonder what does the browser do with e.g. br, gzip. The content is actually encoded with either one or the other.

Doesn't it mean - compress with br, and then compress the result with gzip?

You're right! TIL

I still don't think it's a valid use case. A single value of "multiple" is sufficient.

That would be much easier to define for now.
And I will work on a CL to revise the current Chromium implementation to it.

One the other hand, I could speculate that in future "two compressions" could become more common, like a very specific compression used with a general compression technique (Maybe some dictionary compress followed by gzip or br) could be beneficial.
I am curious, when putting this to resourceTiming specification, is it possible that we leave some room so it's possible to extend in future? If yes, do we just change the specification in future (making it compatible with older version), or we just leave some room in the text we write right now?

One the other hand, I could speculate that in future "two compressions" could become more common, like a very specific compression used with a general compression technique (Maybe some dictionary compress followed by gzip or br) could be beneficial.

I think even that would be an extreme edge case. For example, the current dictionary compressions are built directly into brotli and zstd (using dcb and dcz content encodings). Specialized compression is likely to be complete and layering them is usually a mistake or at least not helpful.

I'd prefer to keep it simple and just use multiple for that case, even going forward. That would provide enough information from the RUM case where someone could further dig-in and test if they aren't expecting to see it.

Thanks, and I think got the principle. And I think this "multiple" rule can take the highest precedence?
i.e., as long as there is a "comma" the result is "multiple".

Examples below are all to converted to "multiple"
", " not ==> ""
"gzip, gzip" not to ==> "gzip"
"unrecognized_compression, gzip" not to "unknown"

(Because this is the simplest and the very unlikely cases don't matter much.)

Agreed. The comma means there are multiple passes at encoding, even if all of the passes are using the same encoding.

is it possible that we leave some room so it's possible to extend in future?

If in some unforeseeable future we'd find ourselves with a genuine case where it makes sense to apply multiple encodings, we can always decide then to add a value that represents those multiple encodings. I don't think that deciding to go with "multiple" now would have negative future-compat implications.