Mixing two wav files from flash works in v0.9.3 - crashs in v0.9.4 or higher

[veitk]

Hello Phil,

i just took my esp32 out of the drawer and update a project from last year where i mix two wav files from flash. the esp32 freezes after finishing playing a sound.

i tried older versions of your library and it works perfectly with 0.9.3.
this is not a high prio problem because i can of course use 0.9.3 but it would be nice to fix this :)

so in short this is what i use:

uint8_t channels = 2;
I2SStream i2sStream; 
MemoryStream hitSound(shot22_lowfiltered_wav, shot22_lowfiltered_wav_len);
MemoryStream shotSound(hit, hit_len);

InputMixer<int16_t> mixer;
StreamCopy mixCopier(i2sStream, mixer);

...

auto config = i2sStream.defaultConfig(TX_MODE);

config.pin_ws = 27;
config.pin_bck = 26;
config.pin_data = 25;

config.sample_rate = sample_rate;
config.channels = channels;
config.bits_per_sample = 16;

mixer.add(shotSound);
mixer.add(hitSound);

i2sStream.begin(config);

loop:

at a button press i call hitSound.begin()

  if (!mixCopier.copy()) {
      Serial.println("finished playing sound");
      playSound = false;
    }

i set the log level to debug and it seems that it just stops working right at the end of the sound after there is less than 1024 bytes:

.....
15:51:47.479 > [D] AudioCopy.h : 90 - size_t audio_tools::StreamCopyT::copy() [with T = unsigned char; size_t = unsigned int]
15:51:47.487 > [D] AudioStreams.h : 1128 - readBytes: 1024
15:51:47.493 > [D] AudioStreams.h : 1132 - sample_count: 512
15:51:47.496 > [D] AudioStreams.h : 1136 - size_value: 2
15:51:47.501 > [D] I2SStream.h : 86 - virtual size_t audio_tools::I2SStream::write(const uint8_t*, size_t)
15:51:47.510 > [D] I2SESP32.h : 82 - size_t audio_tools::I2SDriverESP32::writeBytes(const void*, size_t)
15:51:47.515 > [D] I2SESP32.h : 89 - i2s_write 1024 -> 1024 bytes
15:51:47.521 > [I] AudioCopy.h : 134 - StreamCopy::copy 1024 -> 1024 -> 1024 bytes - in 1 hops
15:51:47.529 > [D] AudioCopy.h : 90 - size_t audio_tools::StreamCopyT::copy() [with T = unsigned char; size_t = unsigned int]
15:51:47.537 > [D] AudioStreams.h : 1128 - readBytes: 582
15:51:47.543 > [D] AudioStreams.h : 1132 - sample_count: 291
15:51:47.546 > [D] AudioStreams.h : 1136 - size_value: 2
15:51:47.548 > [D] I2SStream.h : 86 - virtual size_t audio_tools::I2SStream::write(const uint8_t*, size_t)
15:51:47.557 > [D] I2SESP32.h : 82 - size_t audio_tools::I2SDriverESP32::writeBytes(const void*, size_t)
15:51:47.565 > [D] I2SESP32.h : 89 - i2s_write 582 -> 582 bytes
15:51:47.571 > [I] AudioCopy.h : 134 - StreamCopy::copy 582 -> 582 -> 582 bytes - in 1 hops
15:51:47.576 > [D] AudioCopy.h : 90 - size_t audio_tools::StreamCopyT::copy() [with T = unsigned char; size_t = unsigned int]
15:51:47.587 > [D] AudioStreams.h : 1128 - readBytes: 1024
15:51:47.590 > [D] AudioStreams.h : 1132 - sample_count: 512
15:51:47.596 > [D] AudioStreams.h : 1136 - size_value: 2

here the esp32 freezes.

[vkjambit]

Hi, i know i said this is not high prio ... but maybe i should open a bug ticket for you? Or do you need more information?

[pschatzmann]

I am not sure if this is a bug, after all you tell the system to process 2 streams.
Therefore it waits for the second stream to provide more data.

I suggest that you handle this in your code: e.g. I think you can just mute the stream with no data

[vkjambit]

i'm sorry - i don't understand. my goal is to mix two audio files independently and it works perfect in 0.93 ....

[pschatzmann]

What exactly is it that you don't understand ?

To be able to mix you need to have data: If you mix data and no data you get no data.
So when the one stream has no more data, you need to make a decision:

• stop the processing (copy)
• mute the channel that has no more data (though I have never checked if this is working in this case) in the mixer
• stop mixing and just copy the stream that has data
• wait until data actually gets available

Since all of these are valid options, I can't make this decision for you!

[vkjambit]

ok i will look into that. i guess there is a way available to check if any data is left, didn't look for stuff like that yet.

[pschatzmann]

The copy() is returning the processed bytes
You can check with available() on the stream.

[pschatzmann]

I will also look into it: Maybe I will change the default setting back to what it was and use a flag to use the actual behaviour...

[itman496]

I've been having some trouble with this too. I was hoping the
player.setSilenceOnInactive(true);

would allow me to keep feeding it into the buffer to stop the mixer from freaking out from lack of data, but it appears to have no effect. What does this function actually do?

[pschatzmann]

Are you sure that you use the actual version of the library ?

This call just makes sure that the player sends out 0 values instead of no output.

[itman496]

I'm pretty sure I am.. My issue is whenever one player runs out of the data the mixer goes berserk and sounds choppy and horrible. I'm guessing because I'm trying to mix no data and data. I was hoping the set silence on inactive function would mean that even though I'm running player.copy on something that stopped playing, it would keep pumping out zeros to keep the mixer happy.

For now my workaround is to generate a silent track and switch to copying that to the mixer in place of the player when it runs out of data, then switch back. Not ideal, but it works!

[pschatzmann]

I was testing with this sketch by commenting out the setLoop() and did not notice any issues.

Maybe it will help if you decrease the retry count in the mixer ?

[itman496]

Oh interesting, you are using input mixer. I am using output mixer. I guess that matters? Can I send the output of two wav audioplayers into an input mixer? I see you're doing things differently and not using an audio player either but directly streaming things some other way. I guess two totally different ways of doing the same goal? Is one better than the other?

[pschatzmann]

Working with Stream classes gives usually more flexibility and control.

I am not sure what could go wrong when you try to mix the output from an AudioPlayer.
You would need to debug this...

In any case I would try decrease the timeout in the AudioSource and maybe deacivate the auto fade in the player.

[pschatzmann]

Not sure if this helps but I committed a correction to the AudioPlayer to generate silence not only when it is inactive

[itman496]

I tried with your updated code but it still does the same abrupt cutoff of data for me. I must be doing something wrong with the code on my end.