[css-color-hdr] CSS syntax for HDR colors parameterized by headroom

[svgeesus]

@ccameron-chromium suggested a mechanism for allowing CSS colors to be displayed in HDR, which copes with displays having different amounts of HDR headroom.

It also establishes, in a backwards-and-forwards compatible way, the convention that CSS colors are to be gamut mapped to the SDR gamut of the display, unless they explicitly opt-in to a potentially-higher-headroom gamut via the new syntax.

[svgeesus]

Agenda+ to see if we can resolve to adopt the proposal.

[weinig]

Really glad to see this being discussed!

One thing I can't figure out from the proposal on first reading is if this allows matching the colors of an HDR image or HDR video. (Partially, this is likely a problem of me not knowing enough about how HDR images / videos are represented/ encoded).

That is, if a user has an HDR image, and for the sake of simplicity, it is all one color, is there a way to create CSS Color values that match that color in all headrooms?

[ccameron-chromium]

That is, if a user has an HDR image, and for the sake of simplicity, it is all one color, is there a way to create CSS Color values that match that color in all headrooms?

Yes, that's the goal -- to be able to create a CSS Color that will match an HDR image color in all headrooms.

In order to handle all possible ISO 21496-1 images, the "it may be worth adding those as parameters to be able to color match with gain map images" would need some elaboration.

If there's consensus around this general approach, then I can flesh out those details.

[weinig]

Great. Ok, after reading a bit more about ISO 21496-1 images I think I understand the idea (I do wish those ISO specs were not behind paywalls, but I digress and should just fork over some cash).

Generally, and for matching images that use ISO 21496-1 gain map, this seems like a great way forward to me.

One thing that might make sense in addition to adding parameterization (primaries of the space, esp value, etc.) of the interpolation space would be a way to explicitly use an image's metadata. For example:

  color-hdr(matching url(myImage.png),
      color(rec2100-linear 0.9 1.0 0.8) 1,
      color(rec2100-linear 1.8 2.0 1.5) 2);

What about HDR images that don't use ISO 21496-1 gain maps? (I think one can make an HDR image that has no gain map, presumably getting some default tone-mapping from the system?)

Any thoughts on HDR video?

[ccameron-chromium]

What about HDR images that don't use ISO 21496-1 gain maps?

There is a similar proposal in ICC that defines this mapping as a global tone map. I don't think it's share-able yet, unfortunately. It is also representable by this scheme.

Also if there is no ICC global tone map, and no other metadata, then ISO 22028-5 is working to define how this would be tone mapped "by default".

(BTW, I'd like for us to eventually have WPT tests for all 3 of these cases: gain map tone mapping, global profile tone mapping, and default tone mapping ... hopefully by the end of the year?).

color-hdr(matching url(myImage.png)

I suspect that we'll want to provide something like the ICC global tone map as metadata also for canvases. And so I think something more like that metadata would be a better "match". There are also some parameters in the image metadata to account for the fact that image data is in [0,1] (the min/max/gamma parameters), which don't apply to CSS colors that can specify any value.

Any thoughts on HDR video?

There is a proposal for gainmap support in AV1 video. I would like for there to be a uniform approach on global tone mapping between ICC and video as well.

[svgeesus]

(BTW, I'd like for us to eventually have WPT tests for all 3 of these cases: gain map tone mapping, global profile tone mapping, and default tone mapping ... hopefully by the end of the year?).

Yes, definitely!

Great. Ok, after reading a bit more about ISO 21496-1 images I think I understand the idea (I do wish those ISO specs were not behind paywalls, but I digress and should just fork over some cash).

I agree that these paywalled specs do hinder development. In your case @weinig I suggest an email to Nicolas Bonnier at Apple would be helpful.

[css-meeting-bot]

The CSS Working Group just discussed [css-color-hdr] CSS syntax for HDR colors parameterized by headroom, and agreed to the following:

• RESOLVED: add current proposal as a Note into the spec

The full IRC log of that discussion

<ccameron> hello!
<joshtumath> ccameron: Taking some ideas in ICC, and also colour matching with 8 map images in ISO
<joshtumath> ... A good way to deal with HDR colours
<joshtumath> ... We're not going to expose headroom directly
<joshtumath> ... recomputing every time there are changes is painful
<ChrisL> q+
<joshtumath> ... want to say headroom for this and that, and interoplate between the two in UA
<joshtumath> ... the UA and device can limit it
<smfr> q+
<joshtumath> ... wanted to present the idea. not the definitive way to do it
<joshtumath> ... good to wait until ICC stuff is in a good place
<joshtumath> ... but that's the idea. wanted to see if anyone has thoughts
<astearns> ack ChrisL
<joshtumath> ChrisL: I really like the idea
<weinig> q+
<joshtumath> ... for SDR, everything relative to medium white
<cpn> s/8 map/gain map/
<joshtumath> ... but HDR there is no single value to use but we can't expose the headroom
<joshtumath> ... in the original proposal, you can say what the headroom is for two colours
<joshtumath> ... but when would it not be 0?
<joshtumath> ... it's only defined in between. what if headroom is 2 and 4 on an SDR display?
<joshtumath> ccameron: it needs to be defined down to 0
<joshtumath> ... I'm still working on proposal for what to do with different headroom values
<joshtumath> ... maybe we should build that into the syntax
<joshtumath> ... for images, you can say-- Oh. There is a reason why not to have 0
<joshtumath> ... suppose you had content that you don't want to become HDR
<joshtumath> ... is that the most useful thing, I don't know
<joshtumath> ChrisL: we have to define what happens when headroom is 0
<joshtumath> ... could make the headroom 0 the default value. removes verbosity
<joshtumath> ... could have n values
<joshtumath> ... I don't want to make this too complicated. I want it to be good enough
<ChrisL> q?
<joshtumath> ccameron: I like that idea. I also think that if we workshop this, having an initial version with only two values, one headroom 0 sounds good
<joshtumath> ... maybe we do want the whole p?? wise thing
<joshtumath> ... it adds complexity, but I'd be happy to leave that out for v1
<joshtumath> ... make sure the common case is super simple
<joshtumath> smfr: The colour that you render, the comutation of that colour takes into account screen brightness?
<joshtumath> ccameron: yes
<joshtumath> smfr: need to treat it like accent colour and not paint it into canvas?
<joshtumath> ccameron: I think canvas right now, it's a snapshot at a fixed headroom. not possible to realise values at rendering
<pal> q+
<joshtumath> ... so I haven't written this up. but 2d canvas has a headroom that's 0 SDR
<joshtumath> ... so maybe render canvas by 2, but you need to tell canvas what the effective headroom is
<joshtumath> smfr: so there's no way of using the colour painted as a proxy for screen brightness?
<joshtumath> ccameron: there better not be!
<joshtumath> ... the goal is to make it so you can't exfoltrate that
<joshtumath> ... you have to re-render your entire page when the headroom changes, so we're aligned on that
<joshtumath> smfr: the screen brightness things are ramped
<joshtumath> ... I really want to avoid re-rendering on brightness change
<joshtumath> ChrisL: what is the alternative?
<joshtumath> ???: let the rendering change deal with the viewing environment
<astearns> s/???/pal/
<joshtumath> weinig: you can't write arbitrary PQ values
<joshtumath> pal: as long as we allow that, that's fine right?
<joshtumath> ... you're just writing HDR values and let the renderer deal with it
<joshtumath> ... as long as that's possible, everything else is optional convinience
<smfr> q+
<ccameron> q+ (for the topic of churning the WindowServer during ramp-up)
<joshtumath> ChrisL: Understand that will be possible but that's not topic
<joshtumath> pal: I think it's important. apps that will want additional control, may want this, but otherwise should be able to let renderer do its thing
<pal> q-
<joshtumath> weinig: I was thinking about this. if getting to parity with what images and video can do... if images have to recompute each pixel value...
<joshtumath> ... then this will have to, too
<astearns> q+ ccameron
<ChrisL> Agree that this is CSS parity for images/video
<joshtumath> ... it's as if you have lots of tiny images that you've loaded that have metadata with 'between these ranges'
<joshtumath> ... comfortable for implementation if it mirrors what images and video can already do
<joshtumath> ... so I think that should quell smfr's concerns
<joshtumath> ... it's something that engines already have to deal with
<joshtumath> ... we should discuss alternative of this proposal. new property with CSS colour values that should be treated as if they are HDR values and the system can do whatever tone mapping it wants
<joshtumath> ... I think there are some formats with no colour map. system decides how to tone map that
<joshtumath> ChrisL: yes PNG and ???? has that
<joshtumath> ... it's just the values
<joshtumath> weinig: so I think benefit to having the gain map, matching something that uses PQ values directly
<astearns> ack ccameron
<joshtumath> ccameron: yes absolutely. right now it's going on in ICC where you say 'here's my headroom dependent ICC profile'
<joshtumath> ... here's the transformation to get to headroom 3 1 0
<joshtumath> ... my goal with that is that can be something you can attach to canvas
<joshtumath> ... 'here's the tone mapping for different headrooms'
<joshtumath> ... and that packet, I'd like to have the same packet between video, images, CSS, canvas
<joshtumath> ... so that's another option
<smfr> q-
<joshtumath> ... perhaps that's better than the suggestion I had
<joshtumath> weinig: we just want to get the model of what's happening in the system. something like: you can attach to px some metadata about how to tone map
<joshtumath> ... and the compositor does it
<joshtumath> smfr: I think both. you map at paint time and ????. I'm not sure whether we redraw in the first case but I think we do
<joshtumath> ccameron: for us, if something is in a layer, we can delegate to the OS
<joshtumath> ... otherwise we rewrite every pixel when headroom changes
<joshtumath> ... we throttle it
<ChrisL> s/???/re-render the whole buffer/
<joshtumath> ... interopolate it out or in
<joshtumath> weinig: but for systems with display lists, etc, they don't have to block paint
<joshtumath> ... a concern is, if we're privacy minded, we don't leak this info through perf characteristics
<joshtumath> ccameron: on some platforms it's observable
<joshtumath> ... but the goal is to push as far down pipeline as possible
<joshtumath> weinig: but if we parallel what images can do, simplifies model
<joshtumath> astearns: where are we at. can we resolve to adopt?
<joshtumath> ... or the idea of a different packet to send around something to explore?
<ChrisL> q+
<joshtumath> ccameron: I want something written down and give ICC something public to point at
<joshtumath> ... it's an idea worth getting out to marinate
<joshtumath> ... get an idea of concerns. but I like the packet idea.
<joshtumath> ... we should try to incorporate that
<joshtumath> weinig: agreed needs to marinate. let engines experiment to see what is feasable
<astearns> ack ChrisL
<joshtumath> ... makes a lot more sense to figure out how for images and video it will start playing before we tackle this
<joshtumath> ChrisL: would like to see this in spec so people can see what we're looking at
<joshtumath> ... can't currently make a HDR colour
<joshtumath> ... rather people get to see something
<joshtumath> ... am rep to ICC. I know they tend to hide things until they're public.
<joshtumath> ... these are things to look at, and we need experimentation that's coordinated. get a stick in the ground to kick around
<joshtumath> weinig: maybe we could say in the spec that this is changing
<joshtumath> ChrisL: yes
<joshtumath> ... an explicit 'DO NOT SHIP!!!'
<joshtumath> astearns: we can point to the issue and put in a competing proposal too
<joshtumath> ccameron: everyone same page about: on chrome, you specify the colour and you get something bright
<joshtumath> ... the goal is for that to not be the case. you opt into a brighter colour is the goal
<joshtumath> ... otherwise you get gamut mapped to SDR
<joshtumath> PROPOSED RESOLUTION: put current proposal in spec with a note that we need to solve this and where discussion is happening
<joshtumath> RESOLVED: add current proposal as a Note into the spec