Skip to content

Commit 85d240d

Browse files
jyavenardjyavenard
committed
Add AudioEncoderCocoa class, an AudioToolbox implementation of WebCodec's AudioEncoder
https://bugs.webkit.org/show_bug.cgi?id=284019 rdar://140889671 Reviewed by NOBODY (OOPS!). We add AudioEncoderCocoa, AudioToolbox implementation of WebCodec's AudioEncoder. We only support encoding to Opus and AAC as the framework doesn't provide encoder for Flac, mp3 and vorbis (decoder only) Enabled WPT tests. * LayoutTests/TestExpectations: * LayoutTests/imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any-expected.txt: * LayoutTests/imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any-expected.txt: * LayoutTests/imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.worker-expected.txt: * LayoutTests/imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any-expected.txt: * LayoutTests/platform/glib/TestExpectations: Remove passing expectations, add the remaining ones that do fail. * LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any-expected.txt: Removed. * LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.worker-expected.txt: Removed. * LayoutTests/platform/ios/imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any-expected.txt: Added. The only failure remaining is related to different default for Opus encoding which yield slightly different values to what expected. The test performs very rough float comparisons. * LayoutTests/platform/ios/imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any-expected.txt: Added. * LayoutTests/platform/mac/imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any-expected.txt: Added. * LayoutTests/platform/mac/imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any-expected.txt: Added. * Source/WebCore/Modules/webcodecs/OpusEncoderConfig.h: * Source/WebCore/Modules/webcodecs/WebCodecsAudioEncoder.cpp: (WebCore::isSupportedEncoderCodec): Add additional checks to ensure that the values provided in the Opus config are sane. While this isn't probably the best place to do so, as this is common for both GStreamer and Cocoa AudioEncoder, it's the simplest. (WebCore::isValidEncoderConfig): Make sure sampleRate and numberOfChannels aren't 0. (WebCore::createAudioEncoderConfig): Increase code readability by using struct's named member initialisation. (WebCore::WebCodecsAudioEncoder::configure): (WebCore::WebCodecsAudioEncoder::encode): Both Firefox and Chromes rejects AudioData that has a different sampleRate or numberOfChannels. While we did the same, we didn't process the errors as per spec which requires that we also change the state to Closed just before queueing the error. (WebCore::WebCodecsAudioEncoder::isConfigSupported): * Source/WebCore/PlatformMac.cmake: * Source/WebCore/SourcesCocoa.txt: * Source/WebCore/WebCore.xcodeproj/project.pbxproj: * Source/WebCore/platform/AudioEncoder.cpp: (WebCore::AudioEncoder::create): Plumb AudioEncoderCocoa. * Source/WebCore/platform/AudioEncoder.h: * Source/WebCore/platform/AudioEncoderActiveConfiguration.h: Add default initialiser to comply with more recent clang version. It allows to make the struct member initialisation optional. * Source/WebCore/platform/audio/cocoa/AudioDecoderCocoa.cpp: (WebCore::InternalAudioDecoderCocoa::initialize): Change default AudioData creation to use interleaved frames instead of planar. A few tests has this expectations, even though this isn't mandated by the specs. * Source/WebCore/platform/audio/cocoa/AudioEncoderCocoa.cpp: Added. (WebCore::InternalAudioEncoderCocoa::create): (WebCore::InternalAudioEncoderCocoa::reset): (WebCore::InternalAudioEncoderCocoa::converter): (WebCore::InternalAudioEncoderCocoa::queueSingleton): (WebCore::AudioEncoderCocoa::create): (WebCore::AudioEncoderCocoa::AudioEncoderCocoa): (WebCore::AudioEncoderCocoa::~AudioEncoderCocoa): (WebCore::AudioEncoderCocoa::encode): (WebCore::AudioEncoderCocoa::flush): (WebCore::AudioEncoderCocoa::reset): (WebCore::AudioEncoderCocoa::close): (WebCore::InternalAudioEncoderCocoa::InternalAudioEncoderCocoa): (WebCore::InternalAudioEncoderCocoa::initialize): (WebCore::InternalAudioEncoderCocoa::compressedAudioOutputBufferCallback): (WebCore::InternalAudioEncoderCocoa::generateDecoderDescriptionFromSample const): (WebCore::InternalAudioEncoderCocoa::activeConfiguration const): (WebCore::InternalAudioEncoderCocoa::processEncodedOutputs): (WebCore::InternalAudioEncoderCocoa::encode): (WebCore::InternalAudioEncoderCocoa::flush): (WebCore::InternalAudioEncoderCocoa::close): * Source/WebCore/platform/audio/cocoa/AudioEncoderCocoa.h: Copied from Source/WebCore/platform/AudioEncoderActiveConfiguration.h.
1 parent fb7b82b commit 85d240d

25 files changed

+660
-217
lines changed

‎LayoutTests/TestExpectations

-4
Original file line numberDiff line numberDiff line change
@@ -6410,10 +6410,6 @@ imported/w3c/web-platform-tests/html/dom/render-blocking/element-render-blocking
64106410

64116411
webkit.org/b/278192 imported/w3c/web-platform-tests/html/dom/render-blocking/element-render-blocking-033.html [ Failure ]
64126412

6413-
# AudioEncoder implementation missing.
6414-
imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any.html [ Failure ]
6415-
imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html [ Failure ]
6416-
64176413
webkit.org/b/258192 imported/w3c/web-platform-tests/webcodecs/full-cycle-test.https.any.worker.html?h264_avc [ Pass Failure ]
64186414

64196415
# These tests are timing out since their import.

‎LayoutTests/imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any-expected.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11

22
PASS Test the Opus DTX flag works.
3-
FAIL Test the Opus bitrateMode flag works. assert_less_than: expected a number less than 40330 but got 80660
3+
PASS Test the Opus bitrateMode flag works.
44
PASS Test the AAC bitrateMode flag works.
55

‎LayoutTests/imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any-expected.txt

+13-19
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@ PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Missing
33
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Empty codec
44
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Missing sampleRate
55
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Missing numberOfChannels
6-
FAIL Test that AudioEncoder.isConfigSupported() rejects invalid config: Zero sampleRate assert_unreached: Should have rejected: undefined Reached unreachable code
7-
FAIL Test that AudioEncoder.isConfigSupported() rejects invalid config: Zero channels assert_unreached: Should have rejected: undefined Reached unreachable code
8-
FAIL Test that AudioEncoder.isConfigSupported() rejects invalid config: Bit rate too big assert_unreached: Should have rejected: undefined Reached unreachable code
6+
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Zero sampleRate
7+
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Zero channels
8+
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Bit rate too big
99
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Opus complexity too big
1010
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Opus packetlossperc too big
1111
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Opus frame duration too small
@@ -15,15 +15,9 @@ PASS Test that AudioEncoder.configure() rejects invalid config: Missing codec
1515
PASS Test that AudioEncoder.configure() rejects invalid config: Empty codec
1616
PASS Test that AudioEncoder.configure() rejects invalid config: Missing sampleRate
1717
PASS Test that AudioEncoder.configure() rejects invalid config: Missing numberOfChannels
18-
FAIL Test that AudioEncoder.configure() rejects invalid config: Zero sampleRate assert_throws_js: function "() => {
19-
codec.configure(entry.config);
20-
}" did not throw
21-
FAIL Test that AudioEncoder.configure() rejects invalid config: Zero channels assert_throws_js: function "() => {
22-
codec.configure(entry.config);
23-
}" did not throw
24-
FAIL Test that AudioEncoder.configure() rejects invalid config: Bit rate too big assert_throws_js: function "() => {
25-
codec.configure(entry.config);
26-
}" did not throw
18+
PASS Test that AudioEncoder.configure() rejects invalid config: Zero sampleRate
19+
PASS Test that AudioEncoder.configure() rejects invalid config: Zero channels
20+
PASS Test that AudioEncoder.configure() rejects invalid config: Bit rate too big
2721
PASS Test that AudioEncoder.configure() rejects invalid config: Opus complexity too big
2822
PASS Test that AudioEncoder.configure() rejects invalid config: Opus packetlossperc too big
2923
PASS Test that AudioEncoder.configure() rejects invalid config: Opus frame duration too small
@@ -43,11 +37,11 @@ PASS Test that AudioEncoder.configure() doesn't support config: Sample rate is t
4337
PASS Test that AudioEncoder.configure() doesn't support config: Way too many channels
4438
PASS Test that AudioEncoder.configure() doesn't support config: Possible future opus codec string
4539
PASS Test that AudioEncoder.configure() doesn't support config: Possible future aac codec string
46-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":8000,"numberOfChannels":1} assert_true: expected true got false
47-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2} assert_true: expected true got false
48-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"bitrate":128000,"bitrateMode":"constant","bogus":123} assert_true: expected true got false
49-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"bitrate":128000,"bitrateMode":"variable","bogus":123} assert_true: expected true got false
50-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{"complexity":5,"frameDuration":20000,"packetlossperc":10,"useinbandfec":true}} assert_true: expected true got false
51-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{"format":"opus","complexity":10,"frameDuration":60000,"packetlossperc":20,"usedtx":true,"bogus":456}} assert_true: expected true got false
52-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{}} assert_true: expected true got false
40+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":8000,"numberOfChannels":1}
41+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2}
42+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"bitrate":128000,"bitrateMode":"constant","bogus":123}
43+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"bitrate":128000,"bitrateMode":"variable","bogus":123}
44+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{"complexity":5,"frameDuration":20000,"packetlossperc":10,"useinbandfec":true}}
45+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{"format":"opus","complexity":10,"frameDuration":60000,"packetlossperc":20,"usedtx":true,"bogus":456}}
46+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{}}
5347

‎LayoutTests/imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.worker-expected.txt

+13-19
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@ PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Missing
33
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Empty codec
44
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Missing sampleRate
55
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Missing numberOfChannels
6-
FAIL Test that AudioEncoder.isConfigSupported() rejects invalid config: Zero sampleRate assert_unreached: Should have rejected: undefined Reached unreachable code
7-
FAIL Test that AudioEncoder.isConfigSupported() rejects invalid config: Zero channels assert_unreached: Should have rejected: undefined Reached unreachable code
8-
FAIL Test that AudioEncoder.isConfigSupported() rejects invalid config: Bit rate too big assert_unreached: Should have rejected: undefined Reached unreachable code
6+
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Zero sampleRate
7+
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Zero channels
8+
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Bit rate too big
99
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Opus complexity too big
1010
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Opus packetlossperc too big
1111
PASS Test that AudioEncoder.isConfigSupported() rejects invalid config: Opus frame duration too small
@@ -15,15 +15,9 @@ PASS Test that AudioEncoder.configure() rejects invalid config: Missing codec
1515
PASS Test that AudioEncoder.configure() rejects invalid config: Empty codec
1616
PASS Test that AudioEncoder.configure() rejects invalid config: Missing sampleRate
1717
PASS Test that AudioEncoder.configure() rejects invalid config: Missing numberOfChannels
18-
FAIL Test that AudioEncoder.configure() rejects invalid config: Zero sampleRate assert_throws_js: function "() => {
19-
codec.configure(entry.config);
20-
}" did not throw
21-
FAIL Test that AudioEncoder.configure() rejects invalid config: Zero channels assert_throws_js: function "() => {
22-
codec.configure(entry.config);
23-
}" did not throw
24-
FAIL Test that AudioEncoder.configure() rejects invalid config: Bit rate too big assert_throws_js: function "() => {
25-
codec.configure(entry.config);
26-
}" did not throw
18+
PASS Test that AudioEncoder.configure() rejects invalid config: Zero sampleRate
19+
PASS Test that AudioEncoder.configure() rejects invalid config: Zero channels
20+
PASS Test that AudioEncoder.configure() rejects invalid config: Bit rate too big
2721
PASS Test that AudioEncoder.configure() rejects invalid config: Opus complexity too big
2822
PASS Test that AudioEncoder.configure() rejects invalid config: Opus packetlossperc too big
2923
PASS Test that AudioEncoder.configure() rejects invalid config: Opus frame duration too small
@@ -43,11 +37,11 @@ PASS Test that AudioEncoder.configure() doesn't support config: Sample rate is t
4337
PASS Test that AudioEncoder.configure() doesn't support config: Way too many channels
4438
PASS Test that AudioEncoder.configure() doesn't support config: Possible future opus codec string
4539
PASS Test that AudioEncoder.configure() doesn't support config: Possible future aac codec string
46-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":8000,"numberOfChannels":1} assert_true: expected true got false
47-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2} assert_true: expected true got false
48-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"bitrate":128000,"bitrateMode":"constant","bogus":123} assert_true: expected true got false
49-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"bitrate":128000,"bitrateMode":"variable","bogus":123} assert_true: expected true got false
50-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{"complexity":5,"frameDuration":20000,"packetlossperc":10,"useinbandfec":true}} assert_true: expected true got false
51-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{"format":"opus","complexity":10,"frameDuration":60000,"packetlossperc":20,"usedtx":true,"bogus":456}} assert_true: expected true got false
52-
FAIL AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{}} assert_true: expected true got false
40+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":8000,"numberOfChannels":1}
41+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2}
42+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"bitrate":128000,"bitrateMode":"constant","bogus":123}
43+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"bitrate":128000,"bitrateMode":"variable","bogus":123}
44+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{"complexity":5,"frameDuration":20000,"packetlossperc":10,"useinbandfec":true}}
45+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{"format":"opus","complexity":10,"frameDuration":60000,"packetlossperc":20,"usedtx":true,"bogus":456}}
46+
PASS AudioEncoder.isConfigSupported() supports: {"codec":"opus","sampleRate":48000,"numberOfChannels":2,"opus":{}}
5347

‎LayoutTests/imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any-expected.txt

+13-13
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,19 @@
22
PASS Simple audio encoding
33
PASS Test reset during flush
44
PASS Encode audio with negative timestamp
5-
FAIL Channel number variation: 1 assert_unreached: Should have rejected: undefined Reached unreachable code
6-
FAIL Channel number variation: 2 assert_unreached: Should have rejected: undefined Reached unreachable code
7-
FAIL Sample rate variation: 3000 assert_unreached: Should have rejected: undefined Reached unreachable code
8-
FAIL Sample rate variation: 13000 assert_unreached: Should have rejected: undefined Reached unreachable code
9-
FAIL Sample rate variation: 23000 assert_unreached: Should have rejected: undefined Reached unreachable code
10-
FAIL Sample rate variation: 33000 assert_unreached: Should have rejected: undefined Reached unreachable code
11-
FAIL Sample rate variation: 43000 assert_unreached: Should have rejected: undefined Reached unreachable code
12-
FAIL Sample rate variation: 53000 assert_unreached: Should have rejected: undefined Reached unreachable code
13-
FAIL Sample rate variation: 63000 assert_unreached: Should have rejected: undefined Reached unreachable code
14-
FAIL Sample rate variation: 73000 assert_unreached: Should have rejected: undefined Reached unreachable code
15-
FAIL Sample rate variation: 83000 assert_unreached: Should have rejected: undefined Reached unreachable code
16-
FAIL Sample rate variation: 93000 assert_unreached: Should have rejected: undefined Reached unreachable code
17-
FAIL Encoding and decoding assert_true: expected true got false
5+
PASS Channel number variation: 1
6+
PASS Channel number variation: 2
7+
PASS Sample rate variation: 3000
8+
PASS Sample rate variation: 13000
9+
PASS Sample rate variation: 23000
10+
PASS Sample rate variation: 33000
11+
PASS Sample rate variation: 43000
12+
PASS Sample rate variation: 53000
13+
PASS Sample rate variation: 63000
14+
PASS Sample rate variation: 73000
15+
PASS Sample rate variation: 83000
16+
PASS Sample rate variation: 93000
17+
PASS Encoding and decoding
1818
PASS Emit decoder config and extra data.
1919
PASS encodeQueueSize test
2020
PASS Test encoding Opus with additional parameters: Empty Opus config

‎LayoutTests/platform/glib/TestExpectations

+4-8
Original file line numberDiff line numberDiff line change
@@ -1301,14 +1301,10 @@ imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.
13011301
# A bit flaky on Debug bot.
13021302
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?mp4_aac [ Pass Failure ]
13031303

1304-
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.any.html [ Pass ]
1305-
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.any.worker.html [ Pass ]
1306-
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.crossOriginIsolated.https.any.html [ Pass ]
1307-
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.crossOriginIsolated.https.any.worker.html [ Pass ]
1308-
imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any.html [ Pass ]
1309-
imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.html [ Pass ]
1310-
imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.worker.html [ Pass ]
1311-
imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html [ Pass DumpJSConsoleLogInStdErr ]
1304+
webkit.org/b/284426 imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any.html [ Failure ]
1305+
webkit.org/b/284428 imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.html [ Failure ]
1306+
webkit.org/b/284428 imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.worker.html [ Failure ]
1307+
webkit.org/b/284429 imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html [ Failure ]
13121308
http/wpt/webcodecs/encoder-task-failing.html [ Pass ]
13131309

13141310
# H.264 high-4:2:2 encoding is supported in the GStreamer ports, so this test (checking that profile

0 commit comments

Comments
 (0)