simultaneous, advanced use case

[weow147]

Mr Schatzmann-

Once again i'll start by expressing my appreciation for your work. I'm glad to see this project gathering attention. Big thanks from myself and others - buy this man a coffee!

Now, my question. Thankyou in advance for your opinion, thoughts and insight.

BLACK block diagram works flawlessly, and as I'm waiting for my custom hardware to arrive from easyeda/JLCPCB, my thoughts are turning to version 2.0 of this project.

I'm wondering if it's possible to implement the RED pieces...

I'm sure i can go from sd card to i2s (playback) looks like it needs an encodedAudioStream. Plenty of examples showing that. trivial.

But i have some unknowns...

• I'm NOT sure i can go from (eq to SD, record) and (eq to i2s_out) simultaneously. This is critical because you will want to hear yourself sing through the speakers as you record. Can i add another 'multi' to eq? i guess not, it would need a second copier? I might be able to add another multi-output and 'record' to SD, but it would not have the effects i want.

• NOT sure i can go from (SD Card to i2s_out, playback) and (eq to i2s_out) simultaneously. I think i would need to use an OutputMixer, which is PCM only. i can force mp3/wav into pcm, but not sure what format is 'record'. simultaneous record and playback is important, and must be cumulative.

• NOT sure i can PLAYBACK two files from SD simultaneously. The design goal is something like a "looper" where you can build up a mutli-part harmony (with pitch correction and other effects) singing solo, adding one part at a time, always recording the latest input. AND, it would be slick if you could build up that harmony on top of an mp3 backing trackor WAV drum sample.

Thoughts from Mr Schatzmann and the community?

[pschatzmann]

• Your you can add another MultiOutput which sends the output to a file and I2S at the same time and it must not involve any copier! The EncodedAudioStream is not really needed and you could write directly the binary data into a file. Wav has the only advantage that it has a header that contains the sample rate, number of channels and bits_per_sample).

                       -> I2S (or OutputMixer from step below)
Effects -> MultiOutput  
                       -> EncodedAudioStream ->  File

• I confirm, if you want to output another prerecorded file at the same time, this will involve an Output Mixer. I don't think mp3 will be realistic, but WAV should not be any issue and you can add multiple files:

 ... -> Effects or (MultiOutput from above) -> 
           File -> EncodedAudioStream --copy-> 
                                                OutputMixer  -> I2S
           File -> EncodedAudioStream --copy-> 

If file size should be an issue you can try to use ADPCM which does not add too much overhead. Maybe something for a future release, after you have it working with WAV(PCM).

I have never built something so complex, so I don't know where the limit is. If things get too slow you might need to go down with the sample rate...

[weow147]

Like this?

[pschatzmann]

Yes, that looks right to me assuming that vocals is a prerecorded file

[weow147]

yup, awesome. lemme try it. oh! do you see any opportunity to improve the concept by pinning tasks to separate cores? Maybe put the copiers on separate threads?

[pschatzmann]

That would be something for the implementation of mp3 support: The decoder would try to fill a thread-save queue (in a separate task) which will serve as the input for the OutputMixer.

[weow147]

i'll get wav working first... prost

[weow147]

EncodedAudioStream decoder(&i2s, new WAVDecoder());
StreamCopy decoderCopier(i2s, decoder);
File audioFile;

...
Setup
 audioFile = SD.open("/StarWars60.wav");
decoder.setNotifyAudioChange(i2s);
decoder.begin();
decoderCopier.begin();

...
Loop
decoderCopier.copy();


output

[E] CodecWAV.h : 460 - WAV format not supported: 0

? is there somethign special about wav files i'm not aware of?

[weow147]

Welp, the solution was simple and right under my nose....

The problem was with the second line above... should copy from audiofile to decoder, not decoder to i2s....

if only i was smarter i could save myself so much time...

[pschatzmann]

WAV format 0 does not exist: 1 is PCM! So it seems that this is not a valid header in your WAV file...
Is this from my examples ?

[weow147]

nah i found it online, i'll use your samples... as soon as i find them

[pschatzmann]

You can create/convert one with Audacity

[weow147]

ok i'll try that

[weow147]

still...

[E] CodecWAV.h : 460 - WAV format not supported: 0

tried both signed 16bit pcm and MS adpcm...

i must have something else wrong...

[pschatzmann]

adpcm is not supported by this decoder, just pcm.
It is critical that you call begin before you send any data, so that the decoder knows that the stream starts with a header

[weow147]

hm, still stuck. strange. i'll work on it later...

[weow147]

ok, something else is wrong - it gives me the same error without even opening a file.
Can you point me at an example to follow?

[pschatzmann]

Maybe there is an issue with your SD. Try to read and print the first 44 bytes of a wav file. It should start with RIFF

[weow147]

great hint. the first 44 bytes are trash. i see from google that the first 4 bytes should be ascii riff, size, ascii wave etc...
i think you've helped me discover a problem with my asycwebserver garbling uploads... looks like i've got some debugging to do :)

should be more like this i see

[weow147]

82737070
R I F F

Thought about this some more, i'm getting the ASCII values for RIFF...

[weow147]

solution above... stupid.

[Jvsjvs]

Hi Weow147,

Are you willing to share some of your code as I am looking to setup something like your "black graph" in the initial picture, but then I want to route it to ESPNow and render it on a 2nd device to I2S.

If you can share your setup, then I have a huge headstart.

Thanks,

John.

[weow147]

23sep24.zip

John- here's my esp32 code. Dont judge too harshly, I'm an EE not a programmer :) There's going to be a lot of stuff in here you don't need, sort thru it as best you can and i'd be HAPPY to answer questions.

Working from this block diagram: #1376 (reply in thread)

• i got the BLACK part working decently well
• everything else was a no-go because the esp32 can only handle ONE copier

Ultimately, i gave up on the project because i already have a vocal effects pedal (tc helicon mic mechanic) and the custom JCLPCB hardware i designed ended up not working... So i quit (in shame) and spent my time rehearsing with my band instead. I still have the mockup in my well-stocked failure showcase.

If i ever re-engage on this project, i'll try a raspberry pi solution and force myself to like python.

Let me know if i can help!

[Jvsjvs]

Thanks, I will look at it for inspiration and a starting point. I will not judge at all, that does not help anybody :-)

Have fun with the band!!

[pschatzmann]

Just as a sinde note: the esp32 supports multiple tasks and you can connect them via queues. A RPI has definitly more processing power...

[weow147]

could you put one copier on each core?

[pschatzmann]

I think that would be advisable to process one full input or output chain