High quality 8 channel FLAC decoding using one or more ESP32s

[geoffw64]

The following is cross-posted from my post on philschazmann.ch...
I’ve been looking through your site. You’ve done a lot of great work – well done.
I’m planning to build a multi-channel music player. Minimum requirements are 8 channels, 24-bit samples at 96 kHz. Aspiration is 8 channels, 32-bit samples at 192 kHz. The music is encoded as FLAC and stored in files on my network. I plan to use a separate hardware DAC, so I only need to decode FLAC to something the DAC can use, probably PCM. I may be able to feed the PCM directly to my Oppo Bluray player but this would need to use HDMI.
I can’t imagine a single ESP32 has enough CPU power but perhaps several could. I can see challenges synchronising the output data. What do you think might be a sensible approach? Appreciate your thoughts.

Response from Phil noted that the ESP32 has insufficient I2S outputs for 8 channels.
My thought is that 4 ESP32s would be needed, each dealing with 2 channels. Alternatively, a another means of transmitting the data (other than I2S) would be needed.
Would appreciate inputs and ideas from the community.
Best, Geoff.

[pschatzmann]

I2S is usually the preferred protocol for audio: Most cheap external DACs use I2S and there you are limited to 2 channels and using an ESP32 which has 2 ports, lets you output 4 channels in total: So one thing to investigate first would be, if you can receive, decode and output 4 channels, 24 bit at 96khz on an ESP32 reliably. I am usually working with an AudioKit/LyraT board and there you can only go up to 48000 samples per second and more is not supported.

Some ESP32 like the ESP32-S3 support TDM which can provide more than 2 channels and this would be another solution to evaluate: You would need to find a DAC that supports TDM and should maybe do some more investigations into this direction.

Another solution approach that you could try to investigate would be to use snapcast to distribute the audio and on the client side just output selected channels, e.g. on one ESP32 channels 0,1,2,3 and a second channel 4,5,6,7 or you could dedicate one ESP32 to output one channel only.

The thing I don't quite understand is where your high end audio requirements of minimum 8 channels, 24-bit samples at 96 kHz come from and why you would like to solve this with a cheap but limited ESP32. I would not be able to distinguish the audio of 16 bits from 24 bits and 44100 from 48000.

[geoffw64]

Hi Phil,
Thanks for your thoughts - appreciated. I will have a look at the ESP32-S3 TDM features and also check out snapcast.
Regarding the high end requirements... I have high quality AV components and speakers. I'm after the best reproduction possible.
The reason for considering ESP32 and Arduino is that I have software development skills in that environment - no other particular reason.
Cheers, Geoff.

[pschatzmann]

Just a word of warning: The processing speed, floating point operations and network performance is very limited compared on what you get on the desktop: so the challenge is always to operate at the limit of the capabilities of these small devices and you might need to compromise on your settings to get things working reliably.

Another cheap option that you could look into is to use a Rasperry PI via HDMI

[geoffw64]

I agree Phil - this application would be pushing (or exceeding) the limits and compromises are probably necessary. The projects I've done on ESP32/Arduino have not been so intensive. Quick back of a beer coaster calc... 8 channels at 96k samples per channel per second is a total of 768k samples per second. With a 240MHz clock, we have 312 clock cycles to produce each sample (240M / 768k). I don't know how many instructions you can run for this many cycles but it wouldn't seem to be sufficient to read, decode and send the data. I've ignored the fact that the processor is dual core but this may help with network I/O. It seems likely that multiple ESP32's would be needed and that makes it more complicated.

The Raspberry Pi looks like a good cheap option and there is a community around it too. I'm checking out the Pi 5 which has a 2.4GHz quad core and is 64-bit. Obviously there is operating system overhead but still a lot more horsepower. HDMI output makes it simpler too. I've never developed software on the platform but I see JRiver runs on it. No doubt there are many options here.

[TangerineDreamer]

Hi,@geoffw64
pshatzmann is right about esp32,too poor to deal with multichannels audio,i'm using a cs42448 board with
TEENSY 4.0 or 4.1 and can handle 6 inputs -> 8 outputs,including effects...
Some are also using Audacity in multichannels recording or playing up to 16 channels in association with these boards
using the TDM driver of teensy,it's really another world!

https://forum.pjrc.com/index.php?threads/cs42448-board-6-audio-inputs-8-outputs.72479/#post-338869

[geoffw64]

Hi TangerineDreamer,
That really is another world - thanks for sharing. Will look more closely.
Cheers, Geoff.

[pschatzmann]

Thanks for your input: I ordered a CS42448 and AD1938 based board in order to investigate how far we can get with an ESP32 or RP2040. Since this is just a hobby, I tried so far to keep it on the cheap side and these were by far the most expensive boards I have ever bought.

It might also be interesting to have this running with a cheap stm32h7 microcontroller.

[geoffw64]

I am very interested to see how you go with this!

[TangerineDreamer]

i've seen STM32H7can handle 10-12 audio channels, for 12€ the module looks very interesting,
my cs42448 cost about 30 € and a teensy 4X about 40€,but you can run in parrallel two of cs42448 with
one teensy 4X -> 12 inputs -> 16 outputs !
With all those channels to handle,maybe you're bound to create an API like teensy audio editor :)

[geoffw64]

Hi,
It looks like the Teensy Audio library is limited to 16-bit samples at 44.1kHz. I understand why Paul at Teensy made this design choice. Any idea if the extra horsepower and TDM capability would be sufficient for 8 channels at 24/96?

[TangerineDreamer]

teensy 4.1 is a 600 MHZ racer but also can be extra powered by psram up to 16Mo & spiflash working yet, up to 64Mo ,
i think there is many possibilities for creativity.
96khz and 8 channels also seems to work:
https://forum.pjrc.com/index.php?threads/usb-interface-for-multi-channel-outputs-not-just-stereo.70176/page-2#post-340691

[geoffw64]

Thx for sharing. They're doing great work.
Looking at this recent post:
"... we should be able to get 8 channels at 96kHz working properly? That needs 1536 bytes/ms, so a bInterval of 0.5ms or 4 microframes."
This indicates 16-bit samples.

[TangerineDreamer]

The CS42448 is today at 12,45€ big deal :)

https://fr.aliexpress.com/item/1005006569746141.html?spm=a2g0o.order_list.order_list_main.4.21ef5e5b81d3fM&gatewayAdapt=glo2fra