Record microphone when sound amplitude crosses threshold

[RomanU-mouse]

Hello,
I would like to record from the "AudioCapture" source the signal from the microphone each time it reaches a threshold. So if the audio signal reaches a threshold I want to record the next 5 seconds and save the chunk on disk. For that I first smooth the audio signal as shown bellow:

Then I want to compare the smoothed signal to a threshold:

However Bonsai doesn't let me do the comparison above (the issue is at the GreaterThan node). It is probably due to the fact that the output of AudioCapture is not numeric but has a special format.
Does anyone have a hint how I can output the numeric values out of AudioCapture (similarly to MouseMove for which we can extract the X component)?

Best regards,
Roman

[banchan86]

Hi @RomanU-mouse What are the characteristics of the sound that you are trying to detect or condition on?

So in your screenshot, the Add operator isn't what you would normally think of as smoothing, its adding the audio buffers together and in essence making the output buffer "louder". That may give the appearance of smoothing out the background noise in the MatVisualizer. You are probably looking for something like an Average, but I know the Average reactive and dsp operators don't work in that regard. I think there is a way to make it work, but actually depending on your signal you might not even need a "smoothed" signal.

If you are just trying to detect a high amplitude peak (like a finger snap), you can use something like this instead. We first publish the AudioSamples using a PublishSubject and detect high amplitude peaks using the Extrema operator which can be found in the Bonsai.Dsp package. MaxValue is one of the outputs from that node. You probably want to add a DistinctUntilChanged to only emit a single trigger when the threshold is crossed in either direction, a Condition to select only when the threshold is crossed in the positive direction, and a GateInterval with a 5 second interval to filter out additional triggers which will result in overlapping audio clips being saved. The trigger will activate the workflow within the SelectMany.

edit: Forgot the GateInterval in the original screenshot and added it back in, but also wanted to add an important note about the GateInterval operator, it operates on a master timer that is distinct from when the peak is detected, meaning the gate opens every 5 seconds regardless of when the last element was sent out. This can help to reduce the number of overlapping audio clips, but to achieve a true "refractory period" will be harder.

Within the SelectMany you can use a SubscribeSubject to the AudioSample subjects and use a TakeUntil with a 5 second timer to record each clip. You probably also want to turn on the Timestamp suffix and make sure that the Overwrite property is turned off in AudioWriter to record multiple clips.

If the sound you are trying to condition on is different (for instance, you actually want to condition on the overall background amplitude) then the workflow needs to be modified, but let me know if this works for you.

Workflow attached:

#2424 TriggerAudio.zip

[RomanU-mouse]

Thank you banchan86 very much! It solved my problem.