Fir filter using 2 ports

[ballistRC]

I have been trying to make a FIR filter work. So far I have tried a 1 port solution and I believe that is just bypassing the filter and copying in to out as the results should be a 90degree phase difference between the 2 output channels. Today I followed the 2 port example, code below. I see wierd timings on the pins and neither "in" or "out" corrolate with the what is reported by Audiologger. I can see it trying to work in so far as there are bursts of output data. Any idea what is going on here?

10][D][esp32-hal-cpu.c:244] setCpuFrequencyMhz(): PLL: 480 / 2 = 240 Mhz, APB: 80000000 Hz

[ 27][V][esp32-hal-uart.c:330] uartBegin(): UART0 baud(115200) Mode(800001c) rxPin(3) txPin(1)
[ 37][V][esp32-hal-uart.c:416] uartBegin(): UART0 not installed. Starting installation
[ 48][V][esp32-hal-uart.c:463] uartBegin(): UART0 initialization done.
starting I2S...
[I] AudioTypes.h : 127 - in: sample_rate: 44100 / channels: 2 / bits_per_sample: 24
[I] AudioTypes.h : 127 - out: sample_rate: 44100 / channels: 2 / bits_per_sample: 24
[I] AudioTypes.h : 127 - sample_rate: 44100 / channels: 2 / bits_per_sample: 24
[I] I2SConfigESP32.h : 80 - rx/tx mode: RX_MODE
[I] I2SConfigESP32.h : 81 - port_no: 0
[I] I2SConfigESP32.h : 82 - is_master: Master
[I] I2SConfigESP32.h : 83 - sample rate: 44100
[I] I2SConfigESP32.h : 84 - bits per sample: 24
[I] I2SConfigESP32.h : 85 - number of channels: 2
[I] I2SConfigESP32.h : 86 - signal_type: Digital
[I] I2SConfigESP32.h : 88 - i2s_format: I2S_STD_FORMAT
[I] I2SConfigESP32.h : 90 - auto_clear: false
[I] I2SConfigESP32.h : 92 - use_apll: true
[I] I2SConfigESP32.h : 95 - fixed_mclk: 16934400 " I measure this at 12.5Mhz "
[I] I2SConfigESP32.h : 97 - buffer_count:6
[I] I2SConfigESP32.h : 98 - buffer_size:512
[I] I2SConfigESP32.h : 101 - pin_mck: 3
[I] I2SConfigESP32.h : 103 - pin_bck: 21
[I] I2SConfigESP32.h : 105 - pin_ws: 22
[I] I2SConfigESP32.h : 107 - pin_data: 19
[I] AudioTypes.h : 127 - in: sample_rate: 44100 / channels: 2 / bits_per_sample: 24 "Wclk 29Khz "
[I] AudioTypes.h : 127 - out: sample_rate: 44100 / channels: 2 / bits_per_sample: 24 "Wclk 47Khz "
[I] AudioTypes.h : 127 - sample_rate: 44100 / channels: 2 / bits_per_sample: 24 "Bclk out 2.27Mhz bClk in 1.4Mhz"
[I] I2SConfigESP32.h : 80 - rx/tx mode: TX_MODE
[I] I2SConfigESP32.h : 81 - port_no: 1
[I] I2SConfigESP32.h : 82 - is_master: Master
[I] I2SConfigESP32.h : 83 - sample rate: 44100
[I] I2SConfigESP32.h : 84 - bits per sample: 24
[I] I2SConfigESP32.h : 85 - number of channels: 2
[I] I2SConfigESP32.h : 86 - signal_type: Digital
[I] I2SConfigESP32.h : 88 - i2s_format: I2S_STD_FORMAT
[I] I2SConfigESP32.h : 90 - auto_clear: true
[I] I2SConfigESP32.h : 92 - use_apll: true
[I] I2SConfigESP32.h : 97 - buffer_count:6
[I] I2SConfigESP32.h : 98 - buffer_size:512
[I] I2SConfigESP32.h : 103 - pin_bck: 4
[I] I2SConfigESP32.h : 105 - pin_ws: 5
[I] I2SConfigESP32.h : 107 - pin_data: 18
I2S started...
[I] StreamCopy.h : 147 - StreamCopy::copy 1024 -> 1024 -> 1024 bytes - in 1 hops
[I] StreamCopy.h : 147 - StreamCopy::copy 1024 -> 1024 -> 1024 bytes - in 1 hops

Copy of code

#include "AudioTools.h"
#include "fir_coeffs_61Taps_44100_200_19000.h"
#include "fir_coeffs_101Taps_44100_200_19000.h"
#include "fir_coeffs_161Taps_44100_200_19000.h"
#include "fir_coeffs_251Taps_44100_350_6000.h"

uint16_t sample_rate = 44100;
uint16_t channels = 2;
uint16_t bits_per_sample = 24;

AudioInfo info(sample_rate, 2, 24);
I2SStream in;
I2SStream out;

// copy filtered values
FilteredStream<int16_t, float> filtered(in, info.channels); // Defiles the filter as BaseConverter
StreamCopy copier(out, filtered); // copies sound into i2s

// define FIR filter parameters
//float coef[] = { 0.021, 0.096, 0.146, 0.096, 0.021};

// Arduino Setup
void setup(void) {
// Open Serial
Serial.begin(115200);
// change to Warning to improve the quality
AudioLogger::instance().begin(Serial, AudioLogger::Info);

// setup filters for all available channels
filtered.setFilter(0, new FIR( coeffs_hilbert_61Taps_44100_200_19000));
filtered.setFilter(1, new FIR( coeffs_delay_61));

// start I2S in
Serial.println("starting I2S...");
auto config_in = in.defaultConfig(RX_MODE);
config_in.copyFrom(info);
config_in.sample_rate = sample_rate;
config_in.bits_per_sample = bits_per_sample;
config_in.channels = channels;
config_in.i2s_format = I2S_STD_FORMAT;
config_in.is_master = true;
config_in.port_no = 0;
config_in.pin_ws = 22;
config_in.pin_bck = 21;
config_in.pin_data = 19;
config_in.fixed_mclk = sample_rate * 384;
config_in.pin_mck = 3;

in.begin(config_in);

// start I2S out
auto config_out = out.defaultConfig(TX_MODE);
config_out.copyFrom(info);
config_out.sample_rate = sample_rate;
config_out.bits_per_sample = bits_per_sample;
config_out.channels = channels;
config_out.i2s_format = I2S_STD_FORMAT;
config_out.is_master = true;
config_out.port_no = 1;
config_out.pin_ws = 5;
config_out.pin_bck = 4;
config_out.pin_data = 18;
out.begin(config_out);

Serial.println("I2S started...");
}

// Arduino loop - copy sound to out
void loop() {
copier.copy();
}

[pschatzmann]

You can't process an input of 24 bits with a filter for 16 bits (FilteredStream<int16_t, float> )!
I suggest to stick with 16 bits....

[ballistRC]

ok, makes sence but things won't work correcly as the ADC requires either 64 bits/frame or 48bits/frame. In the above example I have it configured for 48 and only other option is 64. I suppose I can shift out 16 bits after it is aquired before the filters. Would that work?

[ballistRC]

So I tired to get this working
AudioInfo from(44100, 2, 32);
AudioInfo to(44100, 2, 16);
I2SStream in;
I2SStream out;
GeneratedSoundStream<int32_t> in_stream(in); // Stream generated from sine wave
FormatConverterStream conv(in_stream);
FilteredStream<int16_t, float> filtered(conv, to.channels);
StreamCopy copier(filtered, conv); // copies sound into i2s
StreamCopy copier2(out, filtered); // copies sound into i2s

but I get a compile error
ompiling .pio/build/esp32dev/src/main.cpp.o
src/main.cpp:23:43: error: no matching function for call to 'audio_tools::GeneratedSoundStream::GeneratedSoundStream(audio_tools::I2SStream&)'
GeneratedSoundStream<int32_t> in_stream(in);

.pio/libdeps/esp32dev/audio-tools/src/AudioTools/AudioStreams.h:674:3: note: candidate expects 0 arguments, 1 provided
src/main.cpp: In function 'void setup()':
src/main.cpp:71:15: error: no matching function for call to 'audio_tools::I2SStream::begin(audio_tools::AudioInfo&)'
out.begin(to);

Any pointers?

[ballistRC]

_ also tried this AudioInfo from(44100, 2, 32);
AudioInfo to(44100, 2, 16);
I2SStream in;
I2SStream out;
GeneratedSoundStream<int32_t> in_stream(in);
but I can't get the compile through GeneratedSoundStream

[pschatzmann]

You are very confusing: 48bits/frame ? The bits are specified by sample: as the field name (bits_per_sample) implies !
I don't understand your logic why you do config_in.copyFrom(info); and just in the following lines overwrite the very same values.
I also don't understand why you start to use a GeneratedSoundStream which has nothing to do with your original sketch/problem

I think the error message is speaking enough to figure out what you are doing wrong...

[ballistRC]

ok, it was late 48clks/frame ie 24bit words or 64clks/frame 32bit words. This is a hardware limitation for the ADC. 32clks/frame is a no go. Thats the reason I moved to 32bit words. Bear with me as I am new to this library and a HW guy.
I simplified things by taking out the filter for now to see that I could convert a 32bit word to 16bits.

//AudioInfo info(sample_rate, 2, bits_per_sample);
AudioInfo from(44100, 2, 32);
AudioInfo to(44100, 2, 16);
I2SStream in;
I2SStream out;
// convert 32worsd -> 16 bit words
FormatConverterStream conv(in);
// copies 16bit words to output
StreamCopy copier(out, conv);
void setup(void) {
// Open Serial
Serial.begin(115200);
// change to Warning to improve the quality
AudioLogger::instance().begin(Serial, AudioLogger::Info);

// start I2S in
Serial.println("starting I2S...");
auto config_in = in.defaultConfig(RX_MODE);
config_in.copyFrom(from);
config_in.sample_rate = sample_rate;
config_in.bits_per_sample = 32;
config_in.channels = channels;
config_in.i2s_format = I2S_STD_FORMAT;
config_in.is_master = true;
config_in.port_no = 0;
config_in.pin_ws = 22;
config_in.pin_bck = 21;
config_in.pin_data = 19;
config_in.use_apll = true;
// config_in.fixed_mclk = sample_rate * 256;
config_in.pin_mck = 3;

in.begin(config_in);

conv.begin(from, to);
// start I2S out
auto config_out = out.defaultConfig(TX_MODE);
config_out.copyFrom(to);
config_out.sample_rate = sample_rate;
config_out.bits_per_sample = 16;
config_out.channels = channels;
config_out.i2s_format = I2S_STD_FORMAT;
config_out.is_master = true;
config_out.port_no = 1;
config_out.pin_ws = 5;
config_out.pin_bck = 4;
config_out.pin_data = 18;
config_out.use_apll = true;
out.begin(config_out);

Serial.println("I2S started...");
}

It seems the input 32bit config gets overwritten by the 16bit TX config. I see this with Audiologger tool and confirm it by measure Fo of bclk. it's 1.4 1mhz for both were the ADC should be 2.82mhz. Its all a bit strange as earlier today, I thought it was working correctly.

[ballistRC]

Ok, I reverted to v0.9.8 and that has fixed this problem with correct bclk on both ADC and DAC, After unmuting the DAC I see correct output :-)

[pschatzmann]

If you think that there is a problem with the latest version, please provide the full code so that I can reproduce your issue.
In the code above you still did not fix the overwriting of the values of 'from' and 'to' !

[pschatzmann]

I can reproduce your problem and I am currently not sure how to deal with this the best way. In any case, I have updated the documentation

Here are a couple of solution approaches that also work with the latest code:
a) Do the conversion on the output side

FormatConverterStream conv(out);
StreamCopy copier(conv, in);

b) Mark the input as input so that it is not notified about the change

FormatConverterStream conv((Stream&)in);

[ballistRC]

Hi, ok I have tried some of your suggestions using version against v0.9.8 which now includes the FIR filters. I'll put the new code below.
This code seems to be working but the CPU is running out of steam using 61taps fir plus a 61 tap delay, but in the bursts. I do see corrrect function for a Hilbert filter.
I tried this change to see if it would improve CPU load, as changing buffer sizes in the copyier didn't seem to have any effect.
FormatConverterStream conv((Stream&)in); // Works correctly
//NumberFormatConverterStream conv ((Stream&) in); // Doesn't work, all clocks are incorrect. v0.9.8. I will test against "master"
Because I have now introduced the FIR filters, it makes not sence to convert on the output.
I then got rid of the converter completely and changed the FIR to be 32bits. This works much better without buffers running dry. There does seem to be some inblance in gain between the filter and delay, I didn't see this with the converting to 16 bit first time but I do see it now..

uint16_t sample_rate = 44100;
uint16_t channels = 2;
uint16_t bits_per_sample_in = 32;

#define FILTER_16BIT true

#ifdef FILTER_16BIT
uint16_t bits_per_sample_out = 16;
#else
uint16_t bits_per_sample_out = 32;
#endif

//AudioInfo info(sample_rate, 2, bits_per_sample);
AudioInfo from(sample_rate, channels, bits_per_sample_in);
AudioInfo to (sample_rate, channels, bits_per_sample_out);
I2SStream in;
I2SStream out;
#ifdef FILTER_16BIT
// convert 32word -> 16 bit words
FormatConverterStream conv((Stream&)in);
//NumberFormatConverterStream conv ((Stream&) in);// Doesn't work
// FIR copy into FIR from conv 16bit.
FilteredStream<int16_t, float> filtered(conv, channels); //6db difference between filter and delay. Phases correct.
StreamCopy copier(filtered, conv, 2048);
#else // 32 bit FIR.
FilteredStream<int32_t, float> filtered(in, channels); //6db difference between filter and delay. Phases correct.
#endif

// Copy from FIR to out.
StreamCopy copier2(out, filtered, 2048);

// Arduino Setup
void setup(void) {
// Open Serial
Serial.begin(115200);
// change to Warning to improve the quality
AudioLogger::instance().begin(Serial, AudioLogger::Error);
filtered.setFilter(0, new FIR(coeffs_hilbert_61Taps_44100_200_19000));
filtered.setFilter(1, new FIR( coeffs_delay_61));
// start I2S in
Serial.println("starting I2S...");
auto config_in = in.defaultConfig(RX_MODE);
config_in.copyFrom(from);
config_in.sample_rate = sample_rate;
config_in.bits_per_sample = bits_per_sample_in;
config_in.channels = channels;
config_in.i2s_format = I2S_STD_FORMAT;
config_in.is_master = true;
config_in.port_no = 0;
config_in.pin_ws = 22;
config_in.pin_bck = 21;
config_in.pin_data = 19;
config_in.use_apll = true;
// config_in.fixed_mclk = sample_rate * 256;
config_in.pin_mck = 3;
in.begin(config_in);
// start I2S out
auto config_out = out.defaultConfig(TX_MODE);
config_out.copyFrom(to);
config_out.sample_rate = sample_rate;
config_out.bits_per_sample = bits_per_sample_out;
config_out.channels = channels;
config_out.i2s_format = I2S_STD_FORMAT;
config_out.is_master = true;
config_out.port_no = 1;
config_out.pin_ws = 5;
config_out.pin_bck = 4;
config_out.pin_data = 18;
config_out.use_apll = true;
out.begin(config_out);

#ifdef FILTER_16BIT
// Start conversion
conv.begin(from, to);
#endif

Serial.println("I2S started...");
}

// Arduino loop - copy sound to out
void loop() {
#ifdef FILTER_16BIT
copier.copy();
#endif
copier2.copy();
}

[pschatzmann]

Floating point operations on microcontrollers are very slow so you need to keep the number of taps as small as possible.

Did you try to reduce the sample rate to double check if you passed your limit ?

ps.

• I will not be able to reproduce this, if you don't provide me the coeffs_hilbert_61Taps_44100_200_19000 and coeffs_delay_61 values.
• If you want to use the more efficient NumberFormatConverterStream you will need to adjust the begin call to conv.begin(from, 16); conv.begin(from, to); will only work if you apply the latest corrections.
• I expect the filter on the output side to be more efficient so it should support some more tabs
• The issue with the notifications reaching the input has been corrected with the last commit

FilteredStream<int16_t, float> filtered(out, channels); 
NumberFormatConverterStream conv(filtered);
StreamCopy copier(conv, in);

[ballistRC]

I tried your suggestions above, however, NumberFormatConverter still doesn't seem to work. I don't see any difference in operation for 16 bit mode, it is still flatlining the CPU just as much.
#ifdef FILTER_16BIT
// FIR copy into FIR from conv 16bit.
FilteredStream<int16_t, float> filtered(out, channels); //6db difference between filter and delay. Phases correct.
// convert 32word -> 16 bit words
FormatConverterStream conv(filtered);
//NumberFormatConverterStream conv (filtered);// Still Doesn't work

//StreamCopy copier(filtered, conv, 2048);
StreamCopy copier(conv, in);

[ballistRC]

coeff_61.zip
coeff_101.zip
coeff_161.zip
Coefficient files.

[ballistRC]

ok in 32bit mode in/out 161 taps work without issue,either with phase or amplitude.
//#define FILTER_16BIT true
//#define FILTER_61_TAPS
//#define FILTER_101_TAPS
#ifdef FILTER_61_TAPS
filtered.setFilter(0, new FIR(coeffs_hilbert_61Taps_44100_200_19000));
filtered.setFilter(1, new FIR( coeffs_delay_61));
#elifdef FILTER_101_TAPS
filtered.setFilter(0, new FIR(coeffs_hilbert_101Taps_44100_200_19000));
filtered.setFilter(1, new FIR( coeffs_delay_101));
#else
filtered.setFilter(0, new FIR(coeffs_hilbert_161Taps_44100_200_19000));
filtered.setFilter(1, new FIR( coeffs_delay_161));
#endif
16 bit FIT 161 is probably flatlining the CPU more as expected.

This has all been against v0.9.8, should I try this all against "master" now or wait a bit?

[pschatzmann]

It is my expectation that maser should work as well

[ballistRC]

ok, I'll give it a try

[ballistRC]

I see there are some updates in main compared to last night.
NumberFormatConversion still doesn't work.
32bit in/out still working file at 161 taps

[pschatzmann]

Before running my own tests: did you correct the bug in your sketch that you call both copier.copy() and copier2.copy() with 16 bits ?

[ballistRC]

I have them like this ATM.
//StreamCopy copier(filtered, conv, 2048);
// 32 bit in.
StreamCopy copier(conv, in);
// Copy from FIR to out. filtered should be 16 bits
StreamCopy copier2(out, filtered, 2048);
// Copy from FIR to out.
StreamCopy copier2(out, filtered, 2048);
// Copy from FIR to out.
StreamCopy copier2(out, filtered, 2048);

I'm just adding a fix point FIR option so the code not quite done or tested. I'll upload that soon after lunch

[ballistRC]

I suppose this means it will only take coeffs as floats?
FilteredStream< T, TF > Class Template Reference

[ballistRC]

new code dump. I had to put in my own git repo as I almost lost track :-)

/**

• @file streams-i2s-filter-i2s.ino
• @brief Copy audio from I2S to I2S using an FIR filter
• @author Phil Schatzmann
• @copyright GPLv3
  */

#include "AudioTools.h"
#include "fir_coeffs_61Taps_44100_200_19000.h"
#include "fir_coeffs_101Taps_44100_200_19000.h"
#include "fir_coeffs_161Taps_44100_200_19000.h"
#include "fir_coeffs_251Taps_44100_350_6000.h"
#include "fir_coeffs_189Taps_8000_300_3000.h" // Fixed point coeffs.

//#define FILTER_16BIT true
//#define FILTER_61_TAPS
//#define FILTER_101_TAPS
#define FILTER_161_TAPS
//#define FILTER_189_FIXED_POINT
#define FILTER_SR_44100
#define FILTER_FLOAT_32BIT

#ifdef FILTER_SR_44100
uint16_t sample_rate = 44100;
#else
uint16_t sample_rate = 8000;
#endif

uint16_t channels = 2;
uint16_t bits_per_sample_in = 32;

#ifdef FILTER_16BIT
uint16_t bits_per_sample_out = 16;
#else
uint16_t bits_per_sample_out = 32;
#endif

//AudioInfo info(sample_rate, 2, bits_per_sample);
AudioInfo from(sample_rate, channels, bits_per_sample_in);
AudioInfo to (sample_rate, channels, bits_per_sample_out);
I2SStream in;
I2SStream out;
#ifdef FILTER_FLOAT_16BIT
// FIR copy into FIR from conv 16bit.
FilteredStream<int16_t, float> filtered(out, channels); //6db difference between filter and delay. Phases correct.
// convert 32word -> 16 bit words
//FormatConverterStream conv(filtered);
NumberFormatConverterStream conv (filtered);// Doesn't work

//StreamCopy copier(filtered, conv, 2048);
StreamCopy copier(conv, in);
#else// FILTER_FLOAT_32BIT // 32 bit FIR.
FilteredStream<int32_t, float> filtered(in, channels); //6db difference between filter and delay 61 taps. Phases correct.
//#else
//FilteredStream<int32_t, int16_t> filtered(in, channels);
#endif

// Copy from FIR to out.
StreamCopy copier2(out, filtered, 2048);

// Arduino Setup
void setup(void) {
// Open Serial
Serial.begin(115200);
// change to Warning to improve the quality
AudioLogger::instance().begin(Serial, AudioLogger::Error);
#ifdef FILTER_61_TAPS
filtered.setFilter(0, new FIR(coeffs_hilbert_61Taps_44100_200_19000));
filtered.setFilter(1, new FIR( coeffs_delay_61));
#elifdef FILTER_101_TAPS
filtered.setFilter(0, new FIR(coeffs_hilbert_101Taps_44100_200_19000));
filtered.setFilter(1, new FIR( coeffs_delay_101));
#else// FILTER_161_TAPS
filtered.setFilter(0, new FIR(coeffs_hilbert_161Taps_44100_200_19000));
filtered.setFilter(1, new FIR( coeffs_delay_161));
//else
// filtered.setFilter(0, new FIR<int16_t>( coeffs_hilbert_189Taps_8000_300_3000 ));
// filtered.setFilter(1, new FIR<int16_t>( coeffs_delay_189 ));
#endif

// start I2S in
Serial.println("starting I2S...");
auto config_in = in.defaultConfig(RX_MODE);
config_in.copyFrom(from);
config_in.sample_rate = sample_rate;
config_in.bits_per_sample = bits_per_sample_in;
config_in.channels = channels;
config_in.i2s_format = I2S_STD_FORMAT;
config_in.is_master = true;
config_in.port_no = 0;
config_in.pin_ws = 22;
config_in.pin_bck = 21;
config_in.pin_data = 19;
config_in.use_apll = true;
// config_in.fixed_mclk = sample_rate * 256;
config_in.pin_mck = 3;
in.begin(config_in);
// start I2S out
auto config_out = out.defaultConfig(TX_MODE);
config_out.copyFrom(to);
config_out.sample_rate = sample_rate;
config_out.bits_per_sample = bits_per_sample_out;
config_out.channels = channels;
config_out.i2s_format = I2S_STD_FORMAT;
config_out.is_master = true;
config_out.port_no = 1;
config_out.pin_ws = 5;
config_out.pin_bck = 4;
config_out.pin_data = 18;
config_out.use_apll = true;
out.begin(config_out);

#ifdef FILTER_16BIT
// Start conversion
conv.begin(from, to);
#endif

Serial.println("I2S started...");
}

// Arduino loop - copy sound to out
void loop() {
#ifdef FILTER_16BIT
copier.copy();
#endif
copier2.copy();
}

[pschatzmann]

I tried to reproduce your issue with the 16 bit conversion with a test sketch, but I don't see any issue here

[ballistRC]

ok, I worked out what was happening. The code is identical to what I have in the FILTER_FLOAT_16BIT, the difference was the version of libs. I had been testing against v0.9.8 and there it won't work. I switched git to main and it works as above. I switch back to tagged version and it broke, so confirmed it works with lastest main.

[ballistRC]

ok, the last part will be to add the outputs from both FIRs. I'm a bit confused with the last part. I think I need to do something like this?
MultiOutput addFIRs;
// Copy from FIRs and add them.
StreamCopy copier2(addFIRs, filtered );
// Output the added stream
StreamCopy copier3(out, addFIRs);
But from your example this doesn't look correct.

[ballistRC]

So I guess this is incorrect as the FIR is 1 stream containing 2 channels and what I want to do is add the 2 channels together. ChannelSplitOutput L_out(filtered, 0);
ChannelSplitOutput R_out(filtered, 1);
MultiOutput addFIRs;
StreamCopy copier(addFIRs, L_out);
I2Stream out;
StreamCopy copier(out, addFIRs);
setup() {
addFIRs.add(L_out);
addFirs.add(R_out);

I'l give this a try.

[ballistRC]

Compiling .pio/build/esp32dev/src/main.cpp.o
src/main.cpp:64:41: error: no matching function for call to 'audio_tools::StreamCopy::StreamCopy(audio_tools::MultiOutput&, audio_tools::ChannelSplitOutput&, int)'

[pschatzmann]

I think you are looking for a Mixer: https://github.com/pschatzmann/arduino-audio-tools/wiki/Splitting-and-Merging-Audio

After a second thought: if I get you right you want to create a mono signal out of the stereo. I don't think there is currently a class which would do this, so you would need to write some logic yourself e.g. using a CallbackStream

[pschatzmann]

After some more thinking: A channel converter might do the job! Just convert from 2 to 1 channel

[ballistRC]

I want to add the 2 channels together and get a signle output. The magic happens when you "add or subtract" the 2 channels leaving a upper or lower sideband of audio. I thought MultiOutput "add" function would do that. Maybe a callback would be better as I would like to control the Add/Subtract function.

[pschatzmann]

MultiOutput just distributes the same signal to multiple output devices

[ballistRC]

I was thinking I could try and add the following but no idea if it would work.
GeneratorFromArray< int32_t> audio_out;
void loop() {

if (UpperSideband )
adder = (int32_t)(L_out + R_out);
else // Lower sideband
adder = (int32_t)(L_out - R_out);
audio_out.readSample(adder);

Thats what I want to happen, so I need to get "adder" back into a stream to output. Could I use GeneratorFromArray()

[pschatzmann]

Just had a look at your TAB include files. I suggest to define the array as const to make sure that it is ending up in PROGMEM and not in RAM

[ballistRC]

Done! seems to reduce RAM a little as it went from 30904 to 29624 saves 1280 bytes using the 161 tap filter.

[ballistRC]

Just for interest 16bit FIR 29976 so actually a big larger RAM image, verses 32bit FIR 29624

[pschatzmann]

I would do something like the follows: This provides the sum on channel 0 and dff on channel 1:

size_t calcSumAndDiff(uint8_t *data, size_t len){
   int32_t *data32 = (int32_t*) data;
   int samples = len / sizeof(int32_t);
   for (int j=0;j<samples;j+=2){
       int32_t add = (0.5f * data32[j]) + (0.5f * data32[j+1]);
       int32_t diff = data32[j] - data32[j+1];
      data32[j] = add
      data32[j+1] = diff;
   }
   return len;
}
CallbackStrem cb(out, calcSumAndDiff);

ps. I did not test this, so there might still be some errors

[ballistRC]

I guess I am missing somethijng about how to implement this. Anyway I try and use clacSumAndDiff doesn't seem to want to compile.
Compiling .pio/build/esp32dev/src/main.cpp.o
src/main.cpp:73:38: error: no matching function for call to 'audio_tools::StreamCopy::StreamCopy(audio_tools::I2SStream&, size_t (&)(uint8_t*, size_t))'
StreamCopy copier(out, calcSumAndDiff);

[pschatzmann]

I don't know how your current chain looks like, but you will need to add this to the output chain!

CallbackStrem cb(out, calcSumAndDiff);
StreamCopy copier(cb, filtered); // copies sound into i2s

or
CallbackStrem cb(out, calcSumAndDiff);
FilteredStream<int16_t, float> filtered(cb, channels);
StreamCopy copier(filtered, in); // copies sound into i2s

Maybe with so many objects it would be a good idea to start to use a Pipeline

[ballistRC]

Something is not quite right. Seems to me the filter input is getting passed to CalcSumAndDiff rather than the phase shifted 90 degree data. I see the 2 signals go in and out of phase, rather than 1 signal getting cancelled depending on the phase of the input. I gather also the byte stream is interleaved ie. ch0,ch1,ch0,ch1 give 2 16 bit words. I changed it a little for readability
#ifdef FILTER_FLOAT_16BIT
// FIR copy into FIR from conv 16bit.
size_t calcSumAndDiff(uint8_t *data, size_t len ){
int32_t data32 = (int32_t) data;
uint8_t samples = len / sizeof(int32_t);
for (int j=0;j<samples;j+=2){
int32_t I = data32[j];
int32_t Q = data32[j+1];
data[j] = I + Q;
data[j+1] = Q + I;
}
return len;
}
CallbackStream cb(out, calcSumAndDiff);
FilteredStream<int16_t, float> filtered(out, channels);
// convert 32word -> 16 bit words
NumberFormatConverterStream conv (filtered);
StreamCopy copier(conv, in);

[pschatzmann]

That is because you send the result of the filtered to out instead of to cb

[ballistRC]

I took a picture of it and will upload a jpeg.

[ballistRC]

The red/green are inputs to ADC from IQ mixer. Blue and yellow are seen on DAC output

[ballistRC]

with a sample rate of 44100 it is around 2ms between bursts. I made a new set of coeffs sampling at 16000 and I now see around 8ms between repeating bursts.

[ballistRC]

reducing the length of the filter didn't seem to make a difference

[pschatzmann]

There is another but in your sketch: if you keep the conversion to 16 bits, the CallbackStream needs to work with 16 bits and not with 32 bits

[ballistRC]

So I put the callback in the 32bit filter path. I assume it should be working correctly as I guess len is passed to it somehow. This looks like it is outputting the filter input again,but the bursting has gone away.
#ifdef FILTER_FLOAT_32BIT // 32 bit FIR.
size_t calcSumAndDiff(uint8_t *data, size_t len ){
int32_t data32 = (int32_t) data;
uint8_t samples = len / sizeof(int32_t);
for (int j=0;j<samples;j+=2){
int32_t I = data32[j];
int32_t Q = data32[j+1];
data[j] = I + Q;
data[j+1] = Q + I;
}
return len;
}

CallbackStream cb(out, calcSumAndDiff);
FilteredStream<int32_t, float> filtered(cb, channels);
StreamCopy copier(filtered, in);
#endif

[ballistRC]

Back again. Took a break and implemented same functionality using Arduino_pico libs and the built in I2S functions as a sanity check. I don't know why I overlooked it, but the sums need to be done on "ints" not on bytes which is what I think is presented to the callback. What I think I need to do is, reconstruct the "Ints" from the byte stream, do the sums then back to bytes. Is there something I can call or do I need to write it in the callback from scratch, which is starting to sound complicated. I noticed "NA5Y Ian" has written his own wrapper ofr the FIR and implemented the sums there. https://github.com/thaaraak/firduino. It is built against a much older version of "audio_tools_lib" and also depends on a old version for ESP_IDF. Tried to get that working but ran into a number of compile issues and didn't think it worthwhile to proceed further.

[pschatzmann]

Isn't issue that you write the result to data[j] instead of data32[j]

[ballistRC]

Data32 is an temp internal variable, what is getting streamed "int32"? The example pointer in the example code makes me think they are bytes espically the "for loop". Adding and subtracting bytes is not the same as int16 or int32 as there is no borrow or carry.

[pschatzmann]

I don't understand your statement: where do you see any adding or subtracting of bytes ?

In the example above you are adding or subtracting int32_t values and the only issue I see is that you don't prevent any overflows when you add the values

[ballistRC]

Just to address the possible overflow concerns first, the int32 is only carrying 24 bits of data as thats all the adc will output, this gets padded to 32bits.
So I believe something is actually wrong with the CB mechanism. I am forcing a zero on 1 output channel inside the CB but I still see both channels unchanged with the data streasm being 180 degrees out of phase! I also have a printf inside the CB and thats also not working. When I make the output = input I see the signals unchanged at 180 degrees. I see both channels even when I force 1 channel to zero. I don't believe the callback is getting called as even a print statement inside doesn't work.

[ballistRC]

This code doesn't do anything. size_t calcSumAndDiff(uint8_t *data, size_t len ){
int32_t data32 = (int32_t) data;
uint8_t samples = len / sizeof(int32_t);
// uint8_t samples = 8;
for (int j=0;j<samples;j+=2){
int32_t I = data32[j];
int32_t Q = data32[j+1];
// data32[j] = (int32_t) (I + Q);
data32[j] = 0;
// data32[j+1] = (int32_t) (I - Q);
data32[j+1] = Q;
Serial.printf("I == %x Q == %x", I, Q);
}
return len;
}

[ballistRC]

int32_t data32 = (int32_t) data;

[pschatzmann]

this should be int32_t* data32 = (int32_t*) data;

[ballistRC]

wierd also, when I copy the code from emacs to here, the pointer "*" gets deleted but it is correct inside the real code.

[ballistRC]

Still not working for me. I was missing the cb.begin(to), which I added plus the assert(samples % 2 == 0); I've attached code plus the filter coeffs.
lastest_code.tar.gz

[pschatzmann]

This is very hard to read with all the different ifdefs.
The target variables in the callback are still wrong in the 16 bit case.
The asserts are not really necessary: it is just to give an error if the number of passed samples if not even.
Add some asserts to make sure that you really don't have any overflow in your addition

Did you write a test case to demonstrate your problem ?
What exactly is the problem ?

[ballistRC]

Hi, only the 32bit code is defined. I use emacs and it greys out all the undefined code so easier to read :-) If you check the 32bit CB I have both add and minus commented out and the output on both channels are hardcoded to zero, but I still have output which remains unchanged no matter what I do in the CB plus the print statement doesn't get excuted. I wasted a day before trying to get my Jtag debugger talking to the chip and failed. I'll have a go again tomorrow, if that fails I will try again with an esp32-s3. I have 2 systems built up, an RP2040 plus the esp32 on breadbord. I have enough bits to make a 3rd I guess with the S3. I've not tried this code on the RP2040 as there is too much floating point, the RP2040 has a fixed point implementation which is working but stardard Arduino-pico libs. I gave you the whole project so it should be easy to get going with platformio. You would need to point at your own copy of "audio_tools_lib", reset should work out of the box after.

The attached project is really the test case, it is the signal processing for a SSB demodulator.
The problem is "the CB never seems to be called" I always see the same data out, which is 2 sinewaves 180degrees out of phase, instead 1 sinewave and 1 very supressed > 20db sinewave down in the noise on the other channel. This is to be run on a dedicated CPU with the other handling control and general IO, at least thats what I have working on the RP2040.

[pschatzmann]

This is way too complex for a simple test case!
I am also not sure if there are any differences between the latest audio-tools and what you are using...

[pschatzmann]

You completely messed up with the data type for the sample count: It should be size_t or int - uint8_t does not make any sense and this is creating your problem !

[ballistRC]

I have no idea why, but changing the from int8 to size_t fixed the thing.
I see you have a resampling block, will this do decimination as the way I read it, it doesn't loose samples which you need to do in decimination? I would like to deciminate the input from 16KHz down to 8kHz to shorten the FIRs and then interpolate "stuffing zero" back to 16Khz at the DAC.

[pschatzmann]

The Wiki contains chapters for Resampling and Format Changes!

[ballistRC]

I'm not sure we are talking about the same thing. Decminate by 2 would be drop every other sample then apply a small FIR to clean it up at half the sample rate.

[pschatzmann]

My resampler only does the decimation and if you want to apply a filter you would need to do this yourself.

[ballistRC]

Hi, I added another CB on the input so I can extract Magnitude before the Hilbert. I've decided not to do resampling at this stage. I'll save it in case I run out of CPU at 16Khz sampling.
I was looking into the mixer classes looking for a source selector I could control. ie select where in the stream I want to monitor. The only way I could see to do it was with a CB "inputProcessing" I added. Maybe there is another way to do processing at various points in the Stream? Also is it possible to reconfigure FIRs on the fly? I would like to change the LPfilter depending on user mode selected. ie USB/LSB, CW, AMWide, AMnarrow, FMwide, FMnarrow. The Hilbert is only used with SSB and CW, other modes need to be in the InputCB.
I was thinking to port this to a RP2040 but I need fixed point FIRs which don't seem to be supported?

size_t inputProcessing(uint8_t *data, size_t len ){
int32_t I, Q;
int32_t * data32 = (int32_t * ) data;
size_t samples = len / sizeof(int32_t);

assert(samples % 2 == 0);
for (int j=0;j<samples;j+=2){
I = data32[j];
Q = data32[j+1];
inputMag = sqrt((pow(I,2) + pow(Q,2)));
data32[j] = I * gainAGC;
data32[j+1] = Q * gainAGC;
}
return len;
}

[pschatzmann]

You can usually dynamically reconfigure the process chain at runtime by calling setOutput() or setStream().
To reconfigure a filter just call open with the new parameters again.

I am not sure if it is even possible to have an int16_t FIR implementation because I would expect you will run into number overflows and when I did some tests on the ESP32 a 32 operations were slower then using floats.

Maybe you can use the CMSIS-DSP functionality on the RP2040 to speed things up.

[ballistRC]

I'll have a go reconfiguring my output LP filter. Int16_t filers are fine. The size of the accumulator is the only issue, you could make it an int64_t with the ability to shift the result. I have int16 based filters in my rp2040 project currently, working fine. This is a system design issue, so the responsibility should be with the designer IMHO. Your not normally looking for a lot of gain in an FIR, 1 normally, maybe 2 max, so the return "type" is usually the same as the input. I am running the same functionality on the RP2040 currently, but different code.