Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more info to CPU warning logs from the Voice Focus worklet #2385

Open
hanfangAmazon opened this issue Aug 2, 2022 · 3 comments
Open
Labels
audio-ml Machine learning features on audio stream such as voice focus and echo reduction

Comments

@hanfangAmazon
Copy link

The log CPU warning: cpu doesn't provide much information under the hood, and the warning is triggered on every 10 audio chunks skipped which isn't ideal.

Consider checking percentage audio chunks skipped per second, and if it's greater than a threshold then log the message, example

Voice Focus CPU warning: skipped 5.2% audio chunks

@devalevenkatesh devalevenkatesh added the audio-ml Machine learning features on audio stream such as voice focus and echo reduction label Aug 2, 2022
@richhx
Copy link
Member

richhx commented Aug 8, 2022

Hi hanfengAmazon@, thanks for bringing this up. We can look into adding this to our backlog. We do send the number of chunks that are skipped (i.e. 10 as you mentioned) in our worker payload and we can log that. I think getting a percentage is tricky because we only process the latest buffer at a time and if CPU is falling far behind, the percentage will be based on only what we know so far. I'd prefer to not have it loop over all buffers as it'll take up more bandwidth.

Is there a particular reason you would like a percentage logged? I agree that the current log message isn't quite useful, and I can look into updating it to log the number of chunks missed. I think the most useful thing is that the worker is not able to keep up in processing, and we offer a delegate.onCPUWarning callback, which can be used to just be notified that Voice Focus is not able to keep up.

@hanfangAmazon
Copy link
Author

@richhx Thanks

Is the number of chunks VF processed per buffer always the same? Assume that we have 1,000 chunks in the latest buffer, and only 10 chunks are skipped, which isn't a big deal. In case of only 100 chunks in the buffer, 10 skipped chunks has greater impact.

That's why we want a percentage, it doesn't necessarily to be percentage skipped per second, it can also be percentage skipped in latest buffer, or any other stats data that's not based on a fixed value.

@richhx
Copy link
Member

richhx commented Aug 9, 2022

The number of chunks processed per buffer is usually finite for a given render quanta, but it can accumulate if the worker is falling behind, so makes sense to potentially add some way to measure it better. I've added it as an action item in our backlog to look into further.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
audio-ml Machine learning features on audio stream such as voice focus and echo reduction
Projects
None yet
Development

No branches or pull requests

3 participants