spdiv and fft simultaniously

[amicado]

First THX for the big library, It covers a lot of use cases though the documentation is hard to plow through as there are examples linked in the wiki which seems outdated. I guess somewhere in the process of switching more to a stream API approach things got mixed up. anyway..
Id like to use A2DP as source and FFT and spdiv as "Sink". I dont get why A2DP is initialised as a2dp.sink even its a source. However..
I got this weired behavior if i set:
a2dp_sink.set_stream_reader(writeDataStream, true); i get spdiv out on pin 23 which is the same as usualy used for i2s out. If i set,
a2dp_sink.set_stream_reader(writeDataStream, false); i dont get any spdiv output at all, which let me guess is because it sets i2s off.

it might be even that the FFT gets wrong values fed in the writeDataStream() function. Could it be that it uses the spdiv stream as input idk because i get weired noise behaviour in the upper bands if i sum up bins to bands logarythmicaly and using an Sinewave generator over bluetooth. then i get logarhytmical increasing noise values in the order of the magnitudes themself. might be that the overhead of the spdiv stream gets interpreted in the fft too.

here is my complete code so far maybe someone can give me a hint if i initialize the instances correctly :)
btw the (10*log10f(fft.magnitudeFast(ibin)/1024)); scaling was just for testing purposes. it doesnt give as good results as plain magnitude(ibin) values for my display even it should theoretically work.

#include "AudioTools.h"
#include "AudioLibs/AudioRealFFT.h"
#include "BluetoothA2DPSink.h"
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>
#include <SPI.h>
#include <Wire.h>

//DEFINITIONS
//DISPLAY
#define SCREEN_WIDTH 128 // OLED display width, in pixels
#define SCREEN_HEIGHT 32 // OLED display height, in pixels
#define OLED_RESET -1
#define SCREEN_ADDRESS 0x3C
Adafruit_SSD1306 display(SCREEN_WIDTH, SCREEN_HEIGHT, &Wire, OLED_RESET);

//RGB
#define PIN_RED 33
#define PIN_GREEN 32
#define PIN_BLUE 25

//FFT-Related
double bandValue[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
int bandValueDisp64[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
int bandValueDisp128[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
double oldBarHeights[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };

int peakbar = 0;
int peakbarold = 0;

//Timer_Values
unsigned long rgb_fade_timer = 0; //RGB-Led Timer
unsigned long rgb_fade_update_interval = 25; //Led update timer 40 colorshifts per second

unsigned long disp_fps_timer = 0; //frame limiter timer
unsigned long disp_fps_timer_update_interval = 16; // 62,5fps

unsigned long animation_timer = 0; //times the animationwindows
unsigned long animation_timer_update_interval = 10000;

bool animationflag = false; //toggle animationscreen
bool fft_result_flag = false; //toggle when fft results in bins for preocessing

//RGB-Values
int red_value = 254;
int blue_value = 1;
int green_value = 127;

//*RGB-Fade_direction_registers
int rDir = -2;
int gDir = 2;
int bDir = -2;

AudioInfo info(44100, 2, 16);
SPDIFOutput spdif;
AudioRealFFT fft;
BluetoothA2DPSink a2dp_sink(spdif);
//Hann hann;
//BufferedWindow buffered(&hann);

// Provide data to FFT
void writeDataStream(const uint8_t *data, uint32_t length) {
fft.write(data, length);
}

void setup() {
Serial.begin(115200);
AudioLogger::instance().begin(Serial, AudioLogger::Warning);

pinMode(PIN_RED, OUTPUT);
pinMode(PIN_GREEN, OUTPUT);
pinMode(PIN_BLUE, OUTPUT);

//Display-Initializing
if (!display.begin(SSD1306_SWITCHCAPVCC, SCREEN_ADDRESS)) { //Generate display voltage from 3.3V internally
Serial.println(F("SSD1306 allocation failed"));
for (;;)
; // Don't proceed, loop forever
}
//Initial display buffer contents on the screen, initialize with an Adafruit splash screen
display.display();
delay(1000); // Pause for 3 seconds
// Clear the buffer and show
display.clearDisplay();
display.display();

// setup output
auto cfg = spdif.defaultConfig();
cfg.copyFrom(info);
cfg.buffer_size = 768;
cfg.buffer_count = 8;
cfg.pin_data = 23;
spdif.begin(cfg);

// Setup FFT
auto tcfg = fft.defaultConfig();
tcfg.length = 1024;
tcfg.sample_rate = 32000;
tcfg.channels = 2;
tcfg.channel_used = 0;
// tcfg.window_function = &buffered;
tcfg.bits_per_sample = 16;
tcfg.callback = &fftResult;
fft.begin(tcfg);

// Start Bluetooth Audio Receiver
a2dp_sink.set_auto_reconnect(true, 1000);
a2dp_sink.set_stream_reader(writeDataStream, true);
a2dp_sink.start("Toslink");
}

void loop() {
// RGB Timer Logic
if ((millis() - rgb_fade_timer) > rgb_fade_update_interval) {
fadeRGB();
rgb_fade_timer = millis();
}

//FFT Processing
if (fft_result_flag == true) {
peakbar = 0;
for (byte band = 0; band < 16; band++) {
bandValue[band] = (oldBarHeights[band] * 3 + bandValue[band]) / 4;
oldBarHeights[band] = bandValue[band]; //save old BarHeights for averaging
if (bandValue[band] > peakbar) peakbar = bandValue[band]; //detect peak_value out of 16 bands save in peakbar
}
for (byte band = 0; band < 16; band++) {
bandValueDisp64[band] = map((bandValue[band]), 0, /(((peakbarold * 2) + peakbar) / 3)/ 1000, 0, 32); //scale to dispsize with dynamic scaling integrating peakbar over peakbarold
bandValueDisp128[band] = map((bandValue[band]), 0, /(((peakbarold * 2) + peakbar) / 3)/ 1000, 0, 128);
}
peakbarold = peakbar; //save old_peakvalue for later integration
fft_result_flag = false; //reset and wait for new fft array data
}

//window flow Disp related
if ((millis() - animation_timer) > animation_timer_update_interval) {
animationflag = !animationflag;
animation_timer = millis();
}

if ((millis() - disp_fps_timer) > disp_fps_timer_update_interval) {
if (!animationflag) {
//display bands horizontaly
display.clearDisplay();
for (byte band = 0; band < 16; band++) {
int barHeightDisp = bandValueDisp64[band];
display.fillRect(band * 8, (32 - barHeightDisp), 6, barHeightDisp, WHITE); //load current bar in display buffer
}
display.display(); // show all bars
} else if (animationflag) {
//display bands verticaly
display.clearDisplay();
for (byte band = 0; band < 16; band++) {
int barHeightDisp = bandValueDisp128[band];
display.fillRect(64 - (barHeightDisp / 2), 32 - (band * 2) - 2, barHeightDisp, 2, WHITE);
}
display.display();
}
disp_fps_timer = millis();
}
}

int bands[16][2] = {
{ 2, 2 }, // Band 1: Bin 2 to 2 (eine einzige Bin)
{ 3, 4 }, // Band 2: Bin 3 to 4
{ 5, 7 }, // Band 3: Bin 5 to 7
{ 8, 11 }, // Band 4: Bin 8 to 11
{ 12, 16 }, // Band 5: Bin 12 to 16
{ 17, 22 }, // Band 6: Bin 17 to 22
{ 23, 30 }, // Band 7: Bin 23 to 30
{ 31, 42 }, // Band 8: Bin 31 to 42
{ 43, 58 }, // Band 9: Bin 43 to 58
{ 59, 79 }, // Band 10: Bin 59 to 79
{ 80, 107 }, // Band 11: Bin 80 to 107
{ 108, 144 }, // Band 12: Bin 108 to 144
{ 145, 192 }, // Band 13: Bin 145 to 192
{ 193, 256 }, // Band 14: Bin 193 to 256
{ 257, 341 }, // Band 15: Bin 257 to 341
{ 342, 511 } // Band 16: Bin 342 to 511
};

// load fft result
void fftResult(AudioFFTBase &fft) {
memset(bandValue, 0, sizeof(bandValue));
for (int ibin = 3; ibin < 512; ibin++) {
//16 bands, 30-15kHz
if (fft.magnitudeFast(ibin) > 2) {
if (ibin <= 3) bandValue[0] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 3 && ibin <= 4) bandValue[1] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 4 && ibin <= 5) bandValue[2] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 5 && ibin <= 6) bandValue[3] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 6 && ibin <= 8) bandValue[4] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 8 && ibin <= 11) bandValue[5] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 11 && ibin <= 15) bandValue[6] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 15 && ibin <= 21) bandValue[7] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 21 && ibin <= 31) bandValue[8] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 31 && ibin <= 45) bandValue[9] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 45 && ibin <= 67) bandValue[10] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 67 && ibin <= 101) bandValue[11] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 101 && ibin <= 151) bandValue[12] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 151 && ibin <= 227) bandValue[13] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 227 && ibin <= 341) bandValue[14] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
if (ibin > 341) bandValue[15] += (10 * log10f(fft.magnitudeFast(ibin) / 1024));
}
}

for (int i = 0; i < 16; i++) {
Serial.println(bandValue[i]);
}
Serial.println(" ");
fft_result_flag = true;
}

//RGB-FADE-LOOP
void fadeRGB() {
analogWrite(PIN_GREEN, green_value);
analogWrite(PIN_RED, red_value);
analogWrite(PIN_BLUE, blue_value);

red_value = red_value + rDir;
green_value = green_value + gDir;
blue_value = blue_value + bDir;

if (red_value >= 255 || red_value <= 0) {
rDir = rDir * -1;
}

if (green_value >= 255 || green_value <= 0) {
gDir = gDir * -1;
}

if (blue_value >= 255 || blue_value <= 0) {
bDir = bDir * -1;
}
}

[pschatzmann]

I am sorry, I don't quite understand your problem:

• An A2DP Source is sending Bluetooth to a Bluetooth Speaker and I don't think that this is what you want.
• If you want to define your custom pins for spdif, you need to set up your specific configuration which is missing in your sketch so it is using the default settings
• If you switch off the output when you define the A2DP callback (with false) you need add the corresponding write to spdif in the callback as well
• The sample rate in the fft configuration is wrong: A2DP is providing 44100 and not 32000

Did you test that your log calculation logic is correct and fast enough ? I would not recommend to do this in the audio task and move it out and call it from the loop.

All a2dp examples have been moved to the communications folder: Please let me know which links are broken, so that I can correct them!

Finally I highly recommend to consider the advice given here

[amicado]

ok sry for the confusion, no i just wanted to know how i can route the stream coming from the a2db (from a blt device to the esp ) to my two two sinks effectively.
in some examples i saw the declaration BluetoothA2DPSink a2dp_sink(spdif); with the spdif parameter while others just define BluetoothA2DPSink a2dp_sink; i guessed that its some switch to tell the class interenally where to route, which isnt intuitively obvious.
turning a2dp_sink.set_stream_reader(writeDataStream, true); to ,false stops the spdif output which is working on pin 23 no need to remap, but it suggests that the spdif function reads from the i2s buffer? and turning i2s off stops spdiv output?
i saw examples where A2DP_sink set with RX or TX for receiving or sending, though the name A2DP_sink stays the same which would be more recogniceable as A2DP_source. Apart from that there is a StreamReader class, a stream copier with copy.copy in loop and one multioutput declaration, is that what i want for 2 sinks. I honestly cant say which one to use and when. neither where i find all parameters for the default configs of the different modules.
the sample rate mismatch is a start, ill test that,. but am i bound now to use the a2dp frequency for my fft? how can i change the sample rate if i say just want frequ. till 16khz in the 512 bins?

If you switch off the output when you define the A2DP callback (with false) you need add the corresponding write to spdif in the callback as well`
AAh i see a2dp_sink.set_stream_reader(writeDataStream, true); turning false when declaring the callback stops the interal stream, even if the instance is declared like this: BluetoothA2DPSink a2dp_sink(spdif); or would i have to use BluetoothA2DPSink a2dp_sink; instead as declaration because double declaration like:

BluetoothA2DPSink a2dp_sink(spdif);
BluetoothA2DPSink a2dp_sink(fft);
is forbidden in the class

i guess
so i woud then do this:?

// Provide data to FFT & SPDIV
void writeDataStream(const uint8_t *data, uint32_t length) {
fft.write(data, length);
spdiv.write(data. length);
}
but whats cheaper?
stream from a2dp to spdif and grabbing the stream from a2dp with void writeDataStream(const uint8_t *data, uint32_t length) {
fft.write(data, length); from the buffer somewhere? as i did
or double write it in the callback, or even use this multiooutput?

i cant see the forest in all this trees xD spent 1 week to figure out how the audiodisp class wraps around audiorealfft class which wraps around the audiofft class and so on haha, iam blind.

wegen der Wikilinks schau ich noch aber kann sein dass das tote Verweise von Google auf die seite waren wenn ich es finde geb ich Dir bescheid ;) LG aus Österreich

Did you test that your log calculation logic is correct and fast enough ? I would not recommend to do this in the audio task and move it out and call it from the loop.
yh yh it was just as test to see the values it creates because magnitudes even the sqrt are huge. and first 3 bins are lost to banding from dc bin 1. i couldnt cope with that and tried all to correct somewhere without success. But i tested my autoscaling for the display which when the fft sigal is clear works surprisingly well.

and yeah strictly speaking an a2dp_source would be a sender but as all other devices are sources in terms of input like an frequencygenerator for the class so its just a mismnomer and i have to wrap my head around it imho^^

[pschatzmann]

There is no difference performance wise! So Just do what makes most sense to you.

It is quite recent that you can define the output class for a BluetoothA2DPSink, so before that you were forced to do all output via the callback.

However in all cases you need to be careful and make sure that your fft processing is fast enough.

ps. BluetoothA2DPSink a2dp_sink. w/o output class is using the legacy output logic

[amicado]

Top that helps a lot :) ill do this with two writes for readability so i think,

BluetoothA2DPSink a2dp_sink;
// Start Bluetooth Audio Receiver
a2dp_sink.set_auto_reconnect(true, 1000);
a2dp_sink.set_stream_reader(writeDataStream, true);
a2dp_sink.start("Toslink");
}
// Provide data to FFT & SPDIV
void writeDataStream(const uint8_t *data, uint32_t length) {
fft.write(data, length);
spdiv.write(data. length);
}

will do then..
iam totally new to audioprocessing on esp32 and glad that u made it even that simple to be done hehe.

yeah there allready sometimes breakups especially when changing volume, i guess i have to play a bit with the buffers in Audioconfig.h

btw. is it ok to have a different buffers sizes like here:

// setup output
auto cfg = spdif.defaultConfig();
cfg.copyFrom(info);
cfg.buffer_size = 768;
cfg.buffer_count = 8;
cfg.pin_data = 23;
spdif.begin(cfg);

or should i stick to #define DEFAULT_BUFFER_SIZE 1024 on all to prevent hickups.

and as a2dp is 44100khz i cant use a differnt one on the fft without complication right? XD

[amicado]

and the last question before i steal too much of your time is, why is there so much DC content on the first bin if the PCM is centered around 0? Isnt it to be expected to be very low then even if it is to be the sum of all bin dc magnitudes combined?