Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AVAudioEngine version AudioDeviceModule #158

Open
wants to merge 24 commits into
base: m125_release
Choose a base branch
from

Conversation

hiroshihorie
Copy link
Member

@hiroshihorie hiroshihorie commented Dec 10, 2024

  • iOS voice processing.
  • macOS voice processing.
  • Uses mute API for mic mute / unmute instead of restarting whole audio unit.
  • Muted talker detection & callback.
  • Advanced other audio ducking.
  • Custom input / output hooks for inserting EQ nodes etc.
  • Input pre-warming.
  • Advance starting and retrieving input buffers.
  • Runtime voice processing on / off (bypass).
  • Custom buffer / manual rendering.

macOS support included:

  • Ensure device switch is handled correctly on macOS. Handle AVAudioEngineConfigurationChangeNotification.
  • MacOS device selection.

Fix adm selection

Fixes

Revert adm selection in audio_device_impl

Rename IsManualRenderingMode

Simplify pcm buffer delegate

Fixes

Fixes

Ducking config

Strip manual rendering logic

Runtime-ducking config

Fix compile

Fix start recording

Connect output

Buffer logic

Enable output when input is enabled
Pre initialize mode
Pre initialize logic
Persistent
Checks
Fix buffer logic
Patch default input_mute state
Buffer checks
Start buffer on enable
Delay estimate 0
Stop engine on interrupt
Pass should_resume
Silence warning
Correct session config
Fix state
Start logic
Misc
Rem ses
Rem ses2
State helper
Minor patch
Simplify
Change stop create order
Working state
Ref
State helpers
Re-wire manual audio input
bypass & agc
Mac aec off by default
AudioOptions initially false
Engine reconfigure
AGC & AEC available
Mac device
Fix build
hiroshihorie added a commit to livekit/client-sdk-swift that referenced this pull request Jan 30, 2025
Using lib from webrtc-sdk/webrtc#158.
macOS will still use previous implementation at the moment. (Device
selection etc not implemented yet)

### New features
[Advanced other audio
ducking](https://developer.apple.com/videos/play/wwdc2023/10235/?time=199)
Lower audio from other apps dynamically when voice activity is detected.

[Muted talker
detection](https://developer.apple.com/videos/play/wwdc2023/10235/?time=475)
Detect voice activity even mic is off after calling
`AudioManager.shared.prepareRecording()`.

Provide custom AVAudioNode configuration for input and output through
`onEngineWillConnectInput` and `onEngineWillConnectOutput` handlers.

Early recording initialization with
`AudioManager.shared.prepareRecording()` without mic indicator turning
on.

Early mic input buffer capturing with
`AudioManager.shared.startLocalRecording()` without the need for Room
and connection.

### Improvements

Correct mic muting logic without the requirement of restarting whole
audio unit (previous implementation).

Correct audio session configuration timing. AudioDeviceModule requests
configuration when it's required, instead of counting tracks and
configuring it in advance (previous implementation).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant