Storage Read Issue

[gregenoccaz]

Hi All,

Firstly, awesome project and im almost there.
Using an ESP32 S3 dev board n16r8
ili9341 display
breadboard wiring/breakout board

have built custom firmware using the devkit c target

all flashed, launcher loading, but have the following issues that i just cant get to the bottom of, hope someone can help!

SD Card Mounting
[error] rg_storage_read_file: Fopen failed (22): '/sd/retro-go/config/global.json'
E (1738) diskio_sdmmc: Check status failed (0x107)

have tried multiple cards, capacitors, pull ups. I do get some mounting at launch, as it creates the basic system files and folders on a freshly formatted card.

So i tried to run it in internal flash mode to see if i can at least get all my hardware, buttons e.t.c confirmed as working. But same issue
[error] rg_storage_write_file: Fopen failed (22): '/storage/retro-go/config/global.json'
[error] rg_storage_read_file: Fopen failed (2): '/storage/retro-go/cache/crc32.bin'
I was able to successfully

flash a bin file to my storage partition
Wrote 5242880 bytes (19424 compressed) at 0x00900000 in 32.2 seconds (effective 1300.8 kbit/s)...

my partition table for reference:

ESP-IDF Partition Table

Name, Type, SubType, Offset, Size, Flags

nvs,data,nvs,0x9000,20K,
otadata,data,ota,0xe000,8K,
phy_init,data,phy,0x10000,4K,
launcher,app,ota_0,0x20000,1792K,
retro-core,app,ota_1,0x1e0000,3M,
prboom-go,app,ota_2,0x4e0000,1152K,
gwenesis,app,ota_3,0x600000,2M,
fmsx,app,ota_4,0x800000,1M,
storage,data,spiffs,0x900000,5M,

Config.h file for refrence
// Target definition
#define RG_TARGET_NAME "ESP32S3-DEVKIT-C"

// Storage
#define RG_STORAGE_ROOT "/storage"
//#define RG_STORAGE_SDSPI_HOST SPI3_HOST
//#define RG_STORAGE_SDSPI_SPEED 1000
// #define RG_STORAGE_SDMMC_HOST SDMMC_HOST_SLOT_1
// #define RG_STORAGE_SDMMC_SPEED SDMMC_FREQ_DEFAULT
#define RG_STORAGE_FLASH_PARTITION "storage"

// Audio
#define RG_AUDIO_USE_INT_DAC 0 // 0 = Disable, 1 = GPIO25, 2 = GPIO26, 3 = Both
#define RG_AUDIO_USE_EXT_DAC 1 // 0 = Disable, 1 = Enable

// Video
#define RG_SCREEN_DRIVER 0 // 0 = ILI9341/ST7789
#define RG_SCREEN_HOST SPI2_HOST
#define RG_SCREEN_SPEED SPI_MASTER_FREQ_40M // SPI_MASTER_FREQ_80M
#define RG_SCREEN_BACKLIGHT 1
#define RG_SCREEN_WIDTH 320
#define RG_SCREEN_HEIGHT 240
#define RG_SCREEN_ROTATE 0
#define RG_SCREEN_VISIBLE_AREA {0, 0, 0, 0}
#define RG_SCREEN_SAFE_AREA {0, 0, 0, 0}
#define RG_SCREEN_INIT()
ILI9341_CMD(0xCF, 0x00, 0xc3, 0x30);
ILI9341_CMD(0xED, 0x64, 0x03, 0x12, 0x81);
ILI9341_CMD(0xE8, 0x85, 0x00, 0x78);
ILI9341_CMD(0xCB, 0x39, 0x2c, 0x00, 0x34, 0x02);
ILI9341_CMD(0xF7, 0x20);
ILI9341_CMD(0xEA, 0x00, 0x00);
ILI9341_CMD(0xC0, 0x1B); /* Power control //VRH[5:0] /
ILI9341_CMD(0xC1, 0x12); / Power control //SAP[2:0];BT[3:0] /
ILI9341_CMD(0xC5, 0x32, 0x3C); / VCM control /
ILI9341_CMD(0xC7, 0x91); / VCM control2 /
ILI9341_CMD(0x36, 0x68); / Memory Access Control (MX|MV|BGR) /
ILI9341_CMD(0xB1, 0x00, 0x10); / Frame Rate Control (1B=70, 1F=61, 10=119) /
ILI9341_CMD(0xB6, 0x0A, 0xA2); / Display Function Control /
ILI9341_CMD(0xF6, 0x01, 0x30);
ILI9341_CMD(0xF2, 0x00); / 3Gamma Function Disable /
ILI9341_CMD(0x26, 0x01); / Gamma curve selected */
ILI9341_CMD(0xE0, 0x0F, 0x31, 0x2B, 0x0C, 0x0E, 0x08, 0x4E, 0xF1, 0x37, 0x07, 0x10, 0x03, 0x0E, 0x09, 0x00);
ILI9341_CMD(0xE1, 0x00, 0x0E, 0x14, 0x03, 0x11, 0x07, 0x31, 0xC1, 0x48, 0x08, 0x0F, 0x0C, 0x31, 0x36, 0x0F);

// Refer to rg_input.h to see all available RG_KEY_* and RG_GAMEPAD__MAP types
#define RG_GAMEPAD_ADC_MAP {
/ ADC Joystick not used /
}
#define RG_GAMEPAD_GPIO_MAP {
/ D-Pad /
{RG_KEY_LEFT, .num = GPIO_NUM_4, .pullup = 1, .level = 0},
{RG_KEY_RIGHT, .num = GPIO_NUM_5, .pullup = 1, .level = 0},
/ Action Buttons /
{RG_KEY_A, .num = GPIO_NUM_6, .pullup = 1, .level = 0},
{RG_KEY_B, .num = GPIO_NUM_7, .pullup = 1, .level = 0},
{RG_KEY_X, .num = GPIO_NUM_8, .pullup = 1, .level = 0},
{RG_KEY_Y, .num = GPIO_NUM_19, .pullup = 1, .level = 0},
/ System Buttons (Commented out) /
/ {RG_KEY_START, .num = GPIO_NUM_20, .pullup = 1, .level = 0}, /
/ {RG_KEY_SELECT, .num = GPIO_NUM_26, .pullup = 1, .level = 0}, /
/ {RG_KEY_MENU, .num = GPIO_NUM_31, .pullup = 1, .level = 0}, /
/ {RG_KEY_OPTION, .num = GPIO_NUM_32, .pullup = 1, .level = 0}, */
}

// Battery
//efine RG_BATTERY_DRIVER 1
//efine RG_BATTERY_ADC_UNIT ADC_UNIT_1
//efine RG_BATTERY_ADC_CHANNEL ADC_CHANNEL_3
//efine RG_BATTERY_CALC_PERCENT(raw) (((raw) * 2.f - 3500.f) / (4200.f - 3500.f) * 100.f)
//efine RG_BATTERY_CALC_VOLTAGE(raw) ((raw) * 2.f * 0.001f)

// Status LED
#define RG_GPIO_LED GPIO_NUM_38

// SPI Display (back up working)
#define RG_GPIO_LCD_MISO GPIO_NUM_NC
#define RG_GPIO_LCD_MOSI GPIO_NUM_11
#define RG_GPIO_LCD_CLK GPIO_NUM_12
#define RG_GPIO_LCD_CS -1
#define RG_GPIO_LCD_DC GPIO_NUM_47
#define RG_GPIO_LCD_BCKL GPIO_NUM_40
#define RG_GPIO_LCD_RST GPIO_NUM_39

//#define RG_GPIO_SDSPI_MISO GPIO_NUM_14
//#define RG_GPIO_SDSPI_MOSI GPIO_NUM_15
//#define RG_GPIO_SDSPI_CLK GPIO_NUM_16
//#define RG_GPIO_SDSPI_CS GPIO_NUM_10

// External I2S DAC
#define RG_GPIO_SND_I2S_BCK 41
#define RG_GPIO_SND_I2S_WS 42
#define RG_GPIO_SND_I2S_DATA 20
// #define RG_GPIO_SND_AMP_ENABLE 18

[ducalex]

[error] rg_storage_write_file: Fopen failed (22): '/storage/retro-go/config/global.json'
[error] rg_storage_read_file: Fopen failed (2): '/storage/retro-go/cache/crc32.bin'

There errors are normal if the file doesn't exist yet, you can ignore them. Hard to say what the problem with your SD Card is because you didn't share enough details. Have you tried both SPI and SDMMC mode? Maybe one is less picky than the other.

If you get the SD Card working on a basic esp-idf project (not retro-go) and you can share that working code + the non working retro-go config, I could try to find the mismatch.

flash a bin file to my storage partition

Note that the bin must be FAT, not SPIFFS. But it might be possible to add spiffs support to retro-go. After a cursory look it seems mainly replacing esp_vfs_fat_spiflash_mount with esp_vfs_spiffs_register. I have no objection to adding support for both if it's desirable.

By the way I have removed esp32-s3-devkit-c target from retro-go (in the dev branch) because it lacked documentation and photos. If you finish your project and feel like contributing it would be nice to have a documented breadboard esp32-s3 target :)

[gregenoccaz]

Thanks so much for your reply. I have tried fat and spiffs for the internal flash, I have also tried basic test program to see if LCD and SD work, which they do, see attached, have tried to use your screen initialisation for the LCD.

main.c

I have also uploaded my project file to the following so you can see all my files and any tweaks i have made (and or mistakes!)
https://github.com/gregenoccaz/Retro-Go-ESP32-S3/tree/master

The SD card is class 10 U1, 32 gb, using in built reader from ili9341 display (generic red pcb aliexpress type), but as noted above, it works with both arduino and esp idf test program. Let me know what other info might be helpful for you?

And yes, would love to upload a working target, have been on it all week, using Opus to help somewhat. Currently got everything to flash, launcher and buttons work, just stuck at the SD mount issue

[gregenoccaz]

fyi, after some further trouble shooting, i had updated the rg_storage.c to match my test code, see attached, however still no luck. Same error log

rg_storage (updated).c

Error log, same as before

[gregenoccaz]

Sorry this was the code I updated, to match the test code
#if defined(RG_STORAGE_SDSPI_HOST)

RG_LOGI("Looking for SD Card using SDSPI...");

spi_bus_config_t bus_cfg = {
    .mosi_io_num = RG_GPIO_SDSPI_MOSI,
    .miso_io_num = RG_GPIO_SDSPI_MISO,
    .sclk_io_num = RG_GPIO_SDSPI_CLK,
    .quadwp_io_num = -1,
    .quadhd_io_num = -1,
    .max_transfer_sz = 4096,
};

esp_err_t err = spi_bus_initialize(RG_STORAGE_SDSPI_HOST, &bus_cfg, SPI_DMA_CH_AUTO);
if (err != ESP_OK) // check but do not abort, let esp_vfs_fat_sdspi_mount decide
    RG_LOGW("SPI bus init failed (0x%x)", err);

sdmmc_host_t host_config = SDSPI_HOST_DEFAULT();
host_config.slot = RG_STORAGE_SDSPI_HOST;
host_config.max_freq_khz = RG_STORAGE_SDSPI_SPEED;
host_config.do_transaction = &sdcard_do_transaction;

sdspi_device_config_t slot_config = SDSPI_DEVICE_CONFIG_DEFAULT();
slot_config.host_id = RG_STORAGE_SDSPI_HOST;
slot_config.gpio_cs = RG_GPIO_SDSPI_CS;

// If we're using esp-idf >= 5.0 and the SPI bus is not shared...
// (some ifdef code here)

vTaskDelay(pdMS_TO_TICKS(100));

esp_vfs_fat_mount_config_t mount_config = {
    .format_if_mount_failed = false,
    .max_files = 5,
    .allocation_unit_size = 16 * 1024,
};

err = esp_vfs_fat_sdspi_mount(RG_STORAGE_ROOT, &host_config, &slot_config, &mount_config, &card_handle);

Also lowered speed
#define RG_STORAGE_SDSPI_SPEED 4000

[ducalex]

Thanks for the details. As you said, there doesn't seem to be any meaningful difference between the test program and what retro-go+your config does. The only difference is that retro-go uses the do_transaction hook. You could try commenting that line host_config.do_transaction = &sdcard_do_transaction;, in retro-go it's only used to flash the LED to sd activity anyway.

One possible thing to test is to use the sdkconfig from your working sample project. Copy it to your target's folder, then rg_tool.py clean and rebuild. Because maybe there's a difference in configuration regarding the SPI or FAT driver that could have an impact?

[gregenoccaz]

Thanks again for your help. I am close and believe a lot of the issues are with my sdkconfig files.
For some reason I have ended up with sdkconfig files in the launcher and retro-core folders, yet when reviewing your base code, it is only present in the target build folders.
So my question, does this sdkconfig in the esp32 s3 target folder act as a master file for all the apps? Or do I need to copy it and have it for each app?

And if yes, this is the central sdkconfig file, then I assume the partition table would live in the target folder also?

With my debugging and working on just the launcher I think I messed things up and ended up the sdkconfig and partition tables in each app I was working on, which ended up different and created a bit of a mess….

[fcipaq]

Hi, I had exactly the same problem with my ESP32-P4 port. What I did to solve it was to copy the ODROID's sdkconfig file, and then run menuconfig afterwards (instead of importing my own sdkconfig).

[ducalex]

So my question, does this sdkconfig in the esp32 s3 target folder act as a master file for all the apps? Or do I need to copy it and have it for each app?

Correct, the sdkconfig in the target folder is the master file. The sdkconfig you see in apps folder is generated when you call rg_tool.py build for the first time and it is deleted by rg_tool.py clean.

As long as you don't run rg_tool.py clean, manual changes made to an app's sdkconfig will remain (but will only apply to that one app). Conversely, changes made to the target's sdkconfig will usually be ignored until you run rg_tool.py clean.

My usual workflow for tweaking the sdkconfig is:

1. rg_tool.py build launcher (To generate a default sdkconfig)
2. cd launcher
3. idf menuconfig
4. cd ..
5. rg_tool.py run launcher
6. does it work?
   • yes: cp launcher/sdkconfig components/retro-go/targets/<my-target>/sdkconfig (I also run the extra idf command to only preserve modified values, but it's not necessary)
   • no: go back to step 2 and tweak more

And if yes, this is the central sdkconfig file, then I assume the partition table would live in the target folder also?

No the partition table is generated dynamically by rg_tool.py as needed, usually based on which apps you're building. Any changes you make to the partitions.csv created in the root will likely be ignored for this reason. There is currently no official way to have a custom partition table for a target but I've been considering adding this capability.