Task: VideoDecoderStreamConfig + InputBufferMetadata overrides — atomic stream-config batching + per-buffer container-derived overrides (from discussion #484)

[Ulrond]

Background

Tracks the agreed design from Discussion #484 (raised by @shafi12). Multiple Binder calls to set VideoDecoder properties (setMasteringDisplayInfo, setContentLightLevel, setColorimetry, setStreamResolution, setFrameRate, setDolbyVisionLayerFlags, setPixelAspectRatio, plus dynamic-range setters) cause:

• IPC overhead — 7+ separate transactions per reconfiguration
• Partial-state risk during reconfiguration — failure mid-batch leaves the decoder in an inconsistent state
• No path for container-derived metadata changes mid-stream (ABR HDR profile change per DASH Period, SSAI, live→VOD)

Three weeks of discussion converged on a two-surface solution scoped by decoder lifecycle.

Proposed Solution — two surfaces

Surface 1 — VideoDecoderStreamConfig parcelable (READY-state, atomic setup batching)

A single all-optional parcelable that replaces the multi-call setup pattern. null field = "no change" / "use existing default".

@VintfStability
parcelable VideoDecoderStreamConfig {
    @nullable Resolution            resolution;           // {width, height}
    @nullable Fraction              frameRate;            // {numerator, denominator}
    @nullable Fraction              pixelAspectRatio;     // per #441 / #487
    @nullable Colorimetry           colorimetry;
    @nullable MasteringDisplayInfo  masteringDisplayInfo;
    @nullable ContentLightLevel     contentLightLevel;
    @nullable DolbyVisionLayerFlags dolbyVisionLayerFlags;
}

// On IVideoDecoderController:
void setStreamConfig(in VideoDecoderStreamConfig config);  // @pre State::READY

Properties:

• Atomic — all fields land in one binder transaction
• Partial — clients send only what changed (other fields stay null)
• Type-safe — structured parcelables, no PropertyValue expansion
• Discoverable — full set of stream-description fields in one place
• Per-instance — lives on the controller; no VideoDecoderResourceIndex needed (the controller IS the per-instance handle)

The existing typed setters (setStreamResolution, setColorimetry, etc.) can either stay as convenience aliases or be deprecated and removed in the next major version (pre-restructure timing makes either viable).

Surface 2 — Extended InputBufferMetadata (STARTED-state, per-buffer overrides)

For container-derived metadata that changes mid-stream (ABR per-Period HDR, SSAI ad insertion, live→VOD), extend InputBufferMetadata with optional override fields:

@VintfStability
parcelable InputBufferMetadata {
    long nsPresentationTime;
    boolean endOfStream;
    boolean discontinuity;

    // NEW — optional overrides applied from this buffer onward.
    // null = no change. Bitstream-derived values still take precedence.
    @nullable MasteringDisplayInfo  masteringDisplayInfo;
    @nullable ContentLightLevel     contentLightLevel;
    @nullable Colorimetry           colorimetry;
    @nullable DolbyVisionLayerFlags dolbyVisionLayerFlags;
    @nullable Fraction              pixelAspectRatio;
    // Resolution / frameRate intentionally omitted — these are bitstream-driven
    // (SPS) on every modern codec; middleware shouldn't override them mid-stream.
}

Why per-buffer rather than a separate updateStreamConfig():

• Atomic with the frame the override applies from — no state-machine race between call ordering and the buffer queue
• Naturally aligns with discontinuity boundaries where you'd send these anyway in DASH/HLS
• Reuses existing decodeBufferWithMetadata() plumbing — no new method
• Bitstream-derived precedence still applies — the override is itself a hint; the bitstream wins if the new SPS/SEI say something different (matches the rule in To Propagate Pixel Aspect Ratio from GstCaps to HAL and expose new setPixelAspectRatio() API #441 / feat(videodecoder): add setPixelAspectRatio() and clarify hint precedence + persistence (#486) #487)

Coverage matrix

Scenario	Solved by
Atomic initial config (the original #484 problem)	Surface 1 — setStreamConfig()
ABR resolution / framerate change	Already works — bitstream-derived (SPS), no API needed
ABR HDR profile change per DASH Period	Surface 2 — per-buffer overrides
SSAI ad insertion (same codec, same encryption)	Surface 2 — per-buffer overrides on the ad boundary frame
SSAI with codec or encryption-mode change	Out of scope — decoder restart by middleware policy
Live→VOD transition	Surface 2 — per-buffer overrides
In-stream HDR SEI changes	Already works — decoder picks up from bitstream

Out of scope

• Codec changes mid-stream — middleware orchestrates a teardown / IVideoDecoder.open() cycle.
• Encryption-mode changes mid-stream (CENC → clear, etc.) — same. Decoder restart.
• Audio-side equivalent — the same pattern likely applies for audiodecoder (ad with different audio codec / channel count) and audiomixer (input format changes). Worth raising as a parallel issue if there's interest — left out of this issue's scope to keep it focused on video.

Dependencies

• feat(videodecoder): add setPixelAspectRatio() and clarify hint precedence + persistence (#486) #487 must land first. Establishes the precedence (bitstream wins over middleware hint) and persistence rules for the typed setters; Surface 1 and Surface 2 inherit those rules.

Acceptance

Surface 1

• VideoDecoderStreamConfig.aidl parcelable added under videodecoder/current/.
• setStreamConfig(in VideoDecoderStreamConfig config) added to IVideoDecoderController, @pre State::READY.
• Decision recorded on the existing 6 typed setters: keep as convenience aliases, or deprecate-and-remove. Documented in the AIDL doc-comments either way.
• Regenerated C++ matches.
• Build clean.
• videodecoder/metadata.yaml version bumped per the subsume rule (likely generation since this is a structural addition + possible removal of the 6 setters; breaking).

Surface 2

• InputBufferMetadata.aidl extended with the 5 nullable override fields.
• Doc-comment makes precedence explicit: bitstream-derived values win; overrides are hints.
• Doc-comment makes persistence explicit: overrides apply from this buffer onward (matching the persistence model in feat(videodecoder): add setPixelAspectRatio() and clarify hint precedence + persistence (#486) #487).
• Regenerated C++ matches.
• Build clean.

Both

• Tests / fixtures updated.
• Documentation updated in videodecoder/current/docs/video_decoder.md.
• Migration notes if the 6 typed setters are deprecated.

References

• Discussion #484 — full design discussion + coverage analysis. Closed in favour of this issue.
• feat(videodecoder): add setPixelAspectRatio() and clarify hint precedence + persistence (#486) #487 — videodecoder setPixelAspectRatio() + hint precedence. Dependency.
• To Propagate Pixel Aspect Ratio from GstCaps to HAL and expose new setPixelAspectRatio() API #441 — pixel aspect ratio precedence rules. Referenced by Surface 1's pixelAspectRatio field.
• Task: video colorimetry is not signalled as a single ‘BT709/BT2020’ #367 — colorimetry expansion (4 fields). Affects Surface 1's colorimetry field.

cc @shafi12 @hari22yuva @srinivasgtl