Writing files with SPIFFS no longer working for ESP-01 #4061

funtastic99 · 2018-01-01T13:33:10Z

Basic Infos

SPIFFS no longer works on a ESP-01.

Hardware

Hardware: ESP-01
Core Version: 2.3.0-rc2
also tried with stable

I reproduced this problem on several different ESP-01. So seems not to be hardware related

Description

Problem description
SPIFFS worked for a long time but all of a sudden it stoped working on a ESP-01.
Whenever i write a file its not existing or empty after next boot.
Using the same sketch on an Wemos D1 mini its works without problems. Actually im only facing this problem on a ESP-01.

Settings in IDE

Module: Generic ESP8266 Module
Flash Size: 1MB (64k SPIFFS), also tried with 512k (64k SPIFFS)
CPU Frequency: 80Mhz
Flash Mode: DIO
Flash Frequency: 40Mhz
Upload Using: SERIAL
Reset Method: ck

Sketch

#include <FS.h>

void setup()
{
Serial.begin(115200);

if (!SPIFFS.begin()) Serial.println("error while mounting filesystem!");

readFile();
writeFile();
readFile();

Serial.println("done");
}

void readFile()
{
Serial.println("reading");
File f = SPIFFS.open("/myFile.txt", "r");
if (!f) Serial.println("file not available");
else if (f.available()<=0) Serial.println("file exists but available <0");
else
{
String ssidString = f.readStringUntil('#');
Serial.print("read from file: ");
Serial.println(ssidString);
}
f.close();
}

void writeFile()
{
Serial.println("writing");
File f = SPIFFS.open("/myFile.txt", "w");
if (!f) Serial.println("File creation failed");
else
{
f.print("networkConfig");
f.print("#");
f.flush();
f.close();
}
}

void loop()
{
}

Debug Messages

messages here

Output of first run:
reading
file not available
writing
reading
read from file: <--- here we see the problem already
done
Output of second run:
eading
file not available
writing
reading
read from file:
done
Output of third run:
reading
read from file:
writing
File creation failed <---- check this!
reading
read from file:
done

Any ideas?
thank you!

funtastic99 · 2018-01-03T22:29:04Z

I found out some additional information that might be helpful.
The working ESP-01 report as flash_id:
Manufacturer: e0
Device: 4014

and the new ones that are not working:
Manufacturer: 85
Device: 6014

May the new flash chip is not supported by the arduino esp plugin yet ?

ondabeach · 2018-01-05T02:22:59Z

I'm having the same problem.

Out of the first batch of 20 where the problem first showed up only 7 of them worked properly. All functions except for SPIFFS seem fine.

Out of the next batch of 20, 9 worked properly.

I just received another 2 batches of 20. One batch from a different seller on eBay and the other batch from a seller on AliExpress.

Out of the new batch from eBay only 2 work properly.

All 20 of the new batch from Aliexpress are faulty.

I checked the Chip ID's and I can confirm that Flash Chips with a manufacturer ID of 85 and Device ID of 6014 do not work. I can also confirm that the 2 that did work have the manufacturer ID of E0 and Chip ID of 4014.

I had a look at the flash chips with a magnifying glass and found the faulty ones were all branded FUYA and the 2 that have working SPIFFS are branded BergMicro.

I sure hope someone has a solution for this as it's a major problem for anyone who needs to store stuff like settings. EEPROM is an alternative but it's so clunky compared to SPIFFS and I really don't want to have to recode my sketches.

funtastic99 · 2018-01-05T11:45:48Z

Its definitely a problem with writing SPIFFS. Reading works because i uploaded a flash image of a functional device with SPIFFS data present. But for reading the flash as anyway only mapped to memory while for writing a correct initializiation and timing is needed. I wonder why the booter itself has no problems writing the flash but SPIFFS has. I would have expected that finally the same functions are used to write the flash but it seems that the spiffs implementation is doing something strang and chip dependend.

Pablo2048 · 2018-01-05T11:50:12Z

So it seems like problem in SPIFFS and/or write implementation. Right?

funtastic99 · 2018-01-05T11:58:50Z

yes seems like

ondabeach · 2018-01-05T12:02:50Z

I found that the 'Upload Sketch Data' function of the IDE works. A sketch can then read the data but a sketch can't write to SPIFFS.

This will impact a lot of people, I've bought esp-01 modules witj these PUYA branded flash chips from 3 different suppliers now. Out of the 80 modules only 18 have worked as they had BergMicro flash chips onboard.

weswitt · 2018-01-06T01:26:18Z

I'm seeing a similar, but different problem. My sketch does not find any files in the fs after uploading the spiffs image. I're tried uploading with esptool and the Arduino IDE, but I get the same result. The processor is an ESP01 on a Sonoff device. I've tried both the 512k64k and 1m64k configurations and neither work.

Is the issue I'm seeing related to this report? Am I possibly doing something wrong? What is the best way to diagnose this? I've added some code to the setup function that lists all files and it finds none.

ondabeach · 2018-01-06T02:14:09Z

@weswitt is this a new board purchased recently?
Can you check the writing on the Flash chip, is it PUYA brand? Look for a number like P25Q80?.
Or you can load and run the following code to get the chip data. You're looking for the manufacturer ID of 85. If you have that flash chip then yes, you have the same problem as us.

//*************************************************
#include "FS.h"

void setup() {
Serial.begin(9600);

SPIFFS.begin();
FSInfo fs_info;
SPIFFS.info(fs_info);

float fileTotalKB = (float)fs_info.totalBytes / 1024.0;
float fileUsedKB = (float)fs_info.usedBytes / 1024.0;

float flashChipSize = (float)ESP.getFlashChipSize() / 1024.0 / 1024.0;
float realFlashChipSize = (float)ESP.getFlashChipRealSize() / 1024.0 / 1024.0;
float flashFreq = (float)ESP.getFlashChipSpeed() / 1000.0 / 1000.0;
FlashMode_t ideMode = ESP.getFlashChipMode();

Serial.printf("\n#####################\n");

Serial.printf("__________________________\n\n");
Serial.println("Firmware: ");
Serial.printf(" Chip Id: %08X\n", ESP.getChipId());
Serial.print(" Core version: "); Serial.println(ESP.getCoreVersion());
Serial.print(" SDK version: "); Serial.println(ESP.getSdkVersion());
Serial.print(" Boot version: "); Serial.println(ESP.getBootVersion());
Serial.print(" Boot mode: "); Serial.println(ESP.getBootMode());

Serial.printf("__________________________\n\n");

Serial.println("Flash chip information: ");
Serial.printf(" Flash chip Id: %08X (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))\n", ESP.getFlashChipId());
Serial.printf(" Sketch thinks Flash RAM is size: "); Serial.print(flashChipSize); Serial.println(" MB");
Serial.print(" Actual size based on chip Id: "); Serial.print(realFlashChipSize); Serial.println(" MB ... given by (2^( "Device" - 1) / 8 / 1024");
Serial.print(" Flash frequency: "); Serial.print(flashFreq); Serial.println(" MHz");
Serial.printf(" Flash write mode: %s\n", (ideMode == FM_QIO ? "QIO" : ideMode == FM_QOUT ? "QOUT" : ideMode == FM_DIO ? "DIO" : ideMode == FM_DOUT ? "DOUT" : "UNKNOWN"));

Serial.printf("__________________________\n\n");

Serial.println("File system (SPIFFS): ");
Serial.print(" Total KB: "); Serial.print(fileTotalKB); Serial.println(" KB");
Serial.print(" Used KB: "); Serial.print(fileUsedKB); Serial.println(" KB");
Serial.printf(" Block size: %lu\n", fs_info.blockSize);
Serial.printf(" Page size: %lu\n", fs_info.pageSize);
Serial.printf(" Maximum open files: %lu\n", fs_info.maxOpenFiles);
Serial.printf(" Maximum path length: %lu\n\n", fs_info.maxPathLength);

Dir dir = SPIFFS.openDir("/");
Serial.println("SPIFFS directory {/} :");
while (dir.next()) {
Serial.print(" "); Serial.println(dir.fileName());
Serial.println(" "); Serial.println(dir.fileSize());
}

Serial.printf("__________________________\n\n");

Serial.printf("CPU frequency: %u MHz\n\n", ESP.getCpuFreqMHz());
Serial.print("#####################");

// open file for reading
File f = SPIFFS.open("/login.txt", "r");
if (!f) {
Serial.println("file open failed");
} Serial.println("====== Reading from SPIFFS file =======");
// write 10 strings to file
for (int i=1; i<=10; i++){
String s=f.readStringUntil('\n');
Serial.print(i);
Serial.print(":");
Serial.println(s);
}
}

void loop() {
delay(2);
}
//********************************************

devyte · 2018-01-06T05:36:16Z

Same issue as #3616 , tracking there.
Closing this one.

weswitt · 2018-01-06T14:40:24Z

@ondabeach thanks for the assistance. below is the diagnostic output from the sketch you provided:

#####################

Firmware:
Chip Id: 0042A910
Core version: 2_4_0
SDK version: 2.1.0(deb1901)
Boot version: 31
Boot mode: 1

Flash chip information:
Flash chip Id: 001440E0 (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))
Sketch thinks Flash RAM is size: 0.50 MB
Actual size based on chip Id: 1.00 MB ... given by (2^( "Device" - 1) / 8 / 1024
Flash frequency: 40.00 MHz
Flash write mode: DIO

File system (SPIFFS):
Total KB: 51.72 KB
Used KB: 0.00 KB
Block size: 4096
Page size: 256
Maximum open files: 5
Maximum path length: 32

SPIFFS directory {/} :

CPU frequency: 80 MHz

#####################file open failed
====== Reading from SPIFFS file =======
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:

ondabeach · 2018-01-06T22:57:25Z

@weswitt The last sketch was just to get the flash chip data, the printing of 1~10 at the bottom was left in by mistake.

The chip ID retrieved shows that you have a flash chip that should work, it does for the OP and myself.
FYI, you have flash size set to 512k 64k but your board actually has 1MB. That isn't an issue as the actual size is bigger than your setting.

Try the following code, it should work on your board.

//**********************************************************
#include "FS.h"

String realSize = String(ESP.getFlashChipRealSize());
String ideSize = String(ESP.getFlashChipSize());
bool flashCorrectlyConfigured = realSize.equals(ideSize);

void setup() {
Serial.begin(9600);

if(flashCorrectlyConfigured){
SPIFFS.begin();
Serial.println("flash correctly configured, SPIFFS started, IDE size: " + ideSize + ", real size: " + realSize);
}
else Serial.println("flash incorrectly configured, SPIFFS cannot start, IDE size: " + ideSize + ", real size: " + realSize);

Serial.println("\nVery basic Spiffs example, writing 10 lines to SPIFFS filesystem, and then read them back");
SPIFFS.begin();

if (!SPIFFS.exists("/formatComplete.txt")) {
Serial.println("Please wait 30 secs for SPIFFS to be formatted");
SPIFFS.format();
//delay(30000);
Serial.println("Spiffs formatted");

File f = SPIFFS.open("/formatComplete.txt", "w");
if (!f) {
    Serial.println("file open failed");
} else {
    f.println("Format Complete");
}

} else {
Serial.println("SPIFFS is formatted. Moving along...");
}
}

void loop() {
// open file for writing
File f = SPIFFS.open("/f.txt", "w");
if (!f) {
Serial.println("file open failed");
}
Serial.println("====== Writing to SPIFFS file =========");
// write 10 strings to file
for (int i=1; i<=10; i++){
f.print("Millis() : ");
f.println(millis());
Serial.println(millis());
}

f.close();

// open file for reading
f = SPIFFS.open("/f.txt", "r");
if (!f) {
Serial.println("file open failed");
} Serial.println("====== Reading from SPIFFS file =======");
// write 10 strings to file
for (int i=1; i<=10; i++){
String s=f.readStringUntil('\n');
Serial.print(i);
Serial.print(":");
Serial.println(s);
}
// wait a few seconds before doing it all over again
delay(3000);

}
//*************************************************************************

weswitt · 2018-01-07T01:17:52Z

Thanks @ondabeach. Running that code does confirm that writing to and reading from SPIFFS work fine on my device.

But this brings me back to my problem. What I'm seeing is that when I flash a SPIFFS to the device and then run a sketch the sketch sees zero files.

This happens if I use the Arduino IDE or esptool. This is the esptool command line that I'm using:

esptool.exe -vv -cd ck -cb 115200 -cp com3 -ca 0xEB000 -cf test-spiffs.bin

Does the tool used to flash the SPIFFS need to be up todate with the SPIFFS in the core? Maybe the tool is too old and is not writing a format that the core runtime understands?

ondabeach · 2018-01-07T04:37:54Z

No problem @weswitt.

I'm not sure if esptool needs to.be updated. Much quicker and simpler to use the IDE as you're already in there rather than using command line methods.

We know your hardware is ok and that sketch can write and read SPIFFS so to look at your uploading files to SPIFFS issue:

No idea of your experience level, I'm just a hacker myself so we'll start with the basics.

Use "Show Sketch Folder" under "Sketch" menu.
Create a folder there called "data".
Create/copy your files there. (For testing just create a text file there called "Test.txt" with contents as follows:
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10

Make sure you carriage return after the "10" or that line won't be read.
Select "Sketch Data Upload" from Tools menu. (Make sure serial monitor is not open and board is in upload mode)

Here's a tidied up version of the Get Flash Data/SPIFFS Read sketch.

//*************************************************************
#include "FS.h"

void setup() {
Serial.begin(9600);

SPIFFS.begin();
FSInfo fs_info;
SPIFFS.info(fs_info);

float fileTotalKB = (float)fs_info.totalBytes / 1024.0;
float fileUsedKB = (float)fs_info.usedBytes / 1024.0;

float flashChipSize = (float)ESP.getFlashChipSize() / 1024.0 / 1024.0;
float realFlashChipSize = (float)ESP.getFlashChipRealSize() / 1024.0 / 1024.0;
float flashFreq = (float)ESP.getFlashChipSpeed() / 1000.0 / 1000.0;
FlashMode_t ideMode = ESP.getFlashChipMode();

Serial.println("==========================================================");
Serial.println("Firmware: ");
Serial.printf(" Chip Id: %08X\n", ESP.getChipId());
Serial.print(" Core version: "); Serial.println(ESP.getCoreVersion());
Serial.print(" SDK version: "); Serial.println(ESP.getSdkVersion());
Serial.print(" Boot version: "); Serial.println(ESP.getBootVersion());
Serial.print(" Boot mode: "); Serial.println(ESP.getBootMode());

Serial.printf("__________________________\n\n");

Serial.println("Flash chip information: ");
Serial.printf(" Flash chip Id: %08X (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))\n", ESP.getFlashChipId());
Serial.printf(" Sketch thinks Flash RAM is size: "); Serial.print(flashChipSize); Serial.println(" MB");
Serial.print(" Actual size based on chip Id: "); Serial.print(realFlashChipSize); Serial.println(" MB ... given by (2^( "Device" - 1) / 8 / 1024");
Serial.print(" Flash frequency: "); Serial.print(flashFreq); Serial.println(" MHz");
Serial.printf(" Flash write mode: %s\n", (ideMode == FM_QIO ? "QIO" : ideMode == FM_QOUT ? "QOUT" : ideMode == FM_DIO ? "DIO" : ideMode == FM_DOUT ? "DOUT" : "UNKNOWN"));
Serial.printf(" CPU frequency: %u MHz\n\n", ESP.getCpuFreqMHz());
Serial.printf("__________________________\n\n");

Serial.println("File system (SPIFFS): ");
Serial.print(" Total KB: "); Serial.print(fileTotalKB); Serial.println(" KB");
Serial.print(" Used KB: "); Serial.print(fileUsedKB); Serial.println(" KB");
Serial.printf(" Block size: %lu\n", fs_info.blockSize);
Serial.printf(" Page size: %lu\n", fs_info.pageSize);
Serial.printf(" Maximum open files: %lu\n", fs_info.maxOpenFiles);
Serial.printf(" Maximum path length: %lu\n\n", fs_info.maxPathLength);
Serial.printf("__________________________\n\n");

Dir dir = SPIFFS.openDir("/");
Serial.println("SPIFFS directory {/} :");
while (dir.next()) {
Serial.print("File Name "); Serial.println(dir.fileName());
Serial.print("File Length "); Serial.println(dir.fileSize());
Serial.print("====== First 10 lines of file ("); Serial.print(dir.fileName()); Serial.println(") =======");
// open file for reading
File f = SPIFFS.open("/Test.txt", "r");
if (!f) {
Serial.println("file open failed");
}
else{
// read 10 strings from file
for (int i=1; i<=10; i++){
String s=f.readStringUntil('\n');
Serial.print(i);
Serial.print(" : ");
Serial.println(s);
}
}
Serial.println("==========================================================");
}

}

void loop() {
delay(3000);
}
//****************************************************************

ondabeach · 2018-01-07T09:44:10Z

Hi @igrr, here's the output with the PUYA chip:

phys_addr=962560
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 1048576
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Mounting FS...
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
33: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=2

ondabeach · 2018-01-07T09:52:45Z

@igrr For comparison, this is the output from a BergMicro chip on one of my last 2 modules that can use SPIFFS (hint hint ;)

phys_addr=962560
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 001440E0
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 1048576
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Mounting FS...
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
33: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
File contents:
foobar

SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=14
File contents:
foobar
foobar

SPIFFS_close: fd=1

igrr · 2018-01-07T11:43:34Z

@ondabeach thanks for these logs! What happens if you run the test on the PUYA module, setting flash configuration in IDE to, say, 512K/64K?

ondabeach · 2018-01-07T11:55:42Z

@igrr pretty much the same result Ivan. Thanks for your help with this issue. Whatever you need, just let me know.

phys_addr=438272
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 524288
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Mounting FS...
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
33: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=2

igrr · 2018-01-07T13:40:57Z

Hmm, that doesn't give any clue, to be honest. Okay, let's try another kind of a test:

https://gist.github.com/igrr/964b0d139ce63fcee07c8c1f1facb24d

Same as before, please have debug level set to CORE. No need to run on the "working" board, just on the PUYA one.

funtastic99 · 2018-01-07T13:48:50Z

Hi igrr,
i am also still facing this problem but as a workaround now switched to eeprom.
I also loaded your test sketch. Here is the output:

scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 12
cnt

connected with CAROLAN, channel 6
dhcp client start...
phys_addr=962560
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 1048576
Testing write...
Test data partition @ 0xeb000
dst=0 src=0 len=0
dst=0 src=0 len=4
dst=0 src=0 len=16
dst=0 src=0 len=64
dst=0 src=0 len=1
dst=0 src=1 len=1
dst=1 src=0 len=1
dst=1 src=1 len=1
dst=1 src=1 len=2
dst=1 src=1 len=3
dst=1 src=1 len=4
dst=1 src=1 len=5
dst=3 src=2 len=5
dst=4 src=4 len=60
dst=59 src=0 len=5
dst=60 src=0 len=4
dst=60 src=0 len=3
dst=60 src=0 len=2
dst=63 src=0 len=1
dst=64 src=0 len=0
dst=59 src=59 len=5
dst=60 src=60 len=4
dst=60 src=60 len=3
dst=60 src=60 len=2
dst=63 src=63 len=1
dst=64 src=64 len=0
Testing write/read...
Seed=0 Start=235 Count=16
235: erase write ip:192.168.178.107,mask:255.255.255.0,gw:192.168.178.1
read done
236: erase write read done
237: erase write read done
238: erase write read done
pm open,type:2 0
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write bcn_timout,ap_probe_send_start
read done
Seed=1 Start=235 Count=16
235: erase write ap_probe_send over, rest wifi status to disassoc
state: 5 -> 0 (1)
rm 0
pm close 7
read done
236: erase write scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 12
cnt

connected with CAROLAN, channel 6
dhcp client start...
read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write pm open,type:2 0
read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Seed=42 Start=235 Count=16
235: erase write read done
236: erase write read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Mounting FS...
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
36: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
File contents:
foobar

SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=14
File contents:
foobar
foobar

SPIFFS_close: fd=1

igrr · 2018-01-07T13:51:40Z

@funtastic99 Your output is what i would describe as "normal". Perhaps the issues you and @ondabeach are seeing are not the same after all...

funtastic99 · 2018-01-07T13:58:34Z

@igrr The strange thing is that ondebeach an me both have spiffs problems with flash id Manufacturer: 85, Device: 6014. Maybe its causing different effects.
But still the initial testprogramm at the beginning of this ticket is not showing the expected output after writing. Any idea why ?
Thank!

ondabeach · 2018-01-07T14:01:38Z

@igrr Ok, exact same settings as last test, same board, nothing moved or changed.

phys_addr=962560
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 1048576
Testing write...
Test data partition @ 0xeb000
dst=0 src=0 len=0
dst=0 src=0 len=4
dst=0 src=0 len=16
dst=0 src=0 len=64
dst=0 src=0 len=1
dst=0 src=1 len=1
dst=1 src=0 len=1
dst=1 src=1 len=1
dst=1 src=1 len=2
dst=1 src=1 len=3
dst=1 src=1 len=4
dst=1 src=1 len=5
dst=3 src=2 len=5
dst=4 src=4 len=60
dst=59 src=0 len=5
dst=60 src=0 len=4
dst=60 src=0 len=3
dst=60 src=0 len=2
dst=63 src=0 len=1
dst=64 src=0 len=0
dst=59 src=59 len=5
dst=60 src=60 len=4
dst=60 src=60 len=3
dst=60 src=60 len=2
dst=63 src=63 len=1
dst=64 src=64 len=0
Testing write/read...
Seed=0 Start=235 Count=16
235: erase write read done
236: erase write read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Seed=1 Start=235 Count=16
235: erase write read done
236: erase write read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Seed=42 Start=235 Count=16
235: erase write read done
236: erase write read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Mounting FS...
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
36: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=2

igrr · 2018-01-07T15:22:21Z

Sorry, not getting any closer...

Please try replacing Esp.cpp in your esp8266 core directory with the one attached here, run same test.

Esp.zip

ondabeach · 2018-01-07T15:46:29Z

@igrr esp.cpp replaced, IDE restarted and all settings the same.

phys_addr=438272
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 524288
Testing write...
Test data partition @ 0x6b000
dst=0 src=0 len=0
dst=0 src=0 len=4
dst=0 src=0 len=16
dst=0 src=0 len=64
dst=0 src=0 len=1
dst=0 src=1 len=1
dst=1 src=0 len=1
dst=1 src=1 len=1
dst=1 src=1 len=2
dst=1 src=1 len=3
dst=1 src=1 len=4
dst=1 src=1 len=5
dst=3 src=2 len=5
dst=4 src=4 len=60
dst=59 src=0 len=5
dst=60 src=0 len=4
dst=60 src=0 len=3
dst=60 src=0 len=2
dst=63 src=0 len=1
dst=64 src=0 len=0
dst=59 src=59 len=5
dst=60 src=60 len=4
dst=60 src=60 len=3
dst=60 src=60 len=2
dst=63 src=63 len=1
dst=64 src=64 len=0
Testing write/read...
Seed=0 Start=107 Count=16
107: erase write Flash verify error @0x6b000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
108: erase write Flash verify error @0x6c000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
109: erase write Flash verify error @0x6d000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
110: erase write Flash verify error @0x6e000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
111: erase write Flash verify error @0x6f000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
112: erase write Flash verify error @0x70000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
113: erase write Flash verify error @0x71000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
114: erase write Flash verify error @0x72000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
115: erase write Flash verify error @0x73000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
116: erase write Flash verify error @0x74000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
117: erase write Flash verify error @0x75000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
118: erase write Flash verify error @0x76000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
119: erase write Flash verify error @0x77000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
120: erase write Flash verify error @0x78000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
121: erase write Flash verify error @0x79000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
122: erase write Flash verify error @0x7a000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
Seed=1 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Seed=42 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Flash verify error @0x6b0fc, orig=0xffff, data=0xffff0429, read=0xfff70429
Flash verify error @0x6c0fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x6d0fc, orig=0xffff, data=0xffff0429, read=0xffdf0429
Flash verify error @0x6e0fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x6f0fc, orig=0xffff, data=0xffff0429, read=0xefff0429
Flash verify error @0x700fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x710fc, orig=0xffff, data=0xffff0429, read=0xfbff0429
Flash verify error @0x720fc, orig=0xffff, data=0xffff0429, read=0x7fff0429
Flash verify error @0x730fc, orig=0xffff, data=0xffff0429, read=0x7fff0429
Flash verify error @0x740fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x750fc, orig=0xffff, data=0xffff0429, read=0xefff0429
Flash verify error @0x760fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x770fc, orig=0xffff, data=0xffff0429, read=0xfff70429
Flash verify error @0x780fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x790fc, orig=0xffff, data=0xffff0429, read=0xfff70429
Flash verify error @0x7a0fc, orig=0xffff, data=0xffff0429, read=0xbfff0429
Mounting FS...
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
36: Failed to open file
appendFile
Flash verify error @0x6b105, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b106, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b107, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b108, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b109, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b10a, orig=0xffffffff, data=0x00000000, read=0x00000000
File size=0
f.write returned 7
Flash verify error @0x6b000, orig=0xffff8001, data=0x0001ffff, read=0x0001f77f
Flash verify error @0x6b204, orig=0xfffffffc, data=0x6f6f66ff, read=0x6f6f66ff
Flash verify error @0x6b105, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b106, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b107, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b108, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b109, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b10a, orig=0x00, data=0x00000000, read=0x00000000
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=2

ondabeach · 2018-01-07T17:07:27Z

@igrr I have a second machine as a backup. I made everything the same as far as I can see, including the new Esp.cpp. The only difference I'm aware of is that on the original machine that gave the Flash Verify errors in the output The esp8266com/esp8266 folder was updated to v2.4.0 using GitGui whereas on this machine it was done through the Boards Manager and the URL in IDE preferences.

I can't write to SPIFFS via a sketch on either machine but the esp8266 Sketch Data Upload plugin can.

phys_addr=438272
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 524288
Testing write...
Test data partition @ 0x6b000
dst=0 src=0 len=0
dst=0 src=0 len=4
dst=0 src=0 len=16
dst=0 src=0 len=64
dst=0 src=0 len=1
dst=0 src=1 len=1
dst=1 src=0 len=1
dst=1 src=1 len=1
dst=1 src=1 len=2
dst=1 src=1 len=3
dst=1 src=1 len=4
dst=1 src=1 len=5
dst=3 src=2 len=5
dst=4 src=4 len=60
dst=59 src=0 len=5
dst=60 src=0 len=4
dst=60 src=0 len=3
dst=60 src=0 len=2
dst=63 src=0 len=1
dst=64 src=0 len=0
dst=59 src=59 len=5
dst=60 src=60 len=4
dst=60 src=60 len=3
dst=60 src=60 len=2
dst=63 src=63 len=1
dst=64 src=64 len=0
Testing write/read...
Seed=0 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Seed=1 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Seed=42 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Mounting FS...
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
36: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=2

igrr · 2018-01-07T17:30:21Z

Thanks @ondabeach , these logs with "flash verify error" mean that we are indeed seeing some hardware dependent issue.
I can't explain why you are not seeing these with the other machine, my only guess is that you are copying Esp.cpp file into one core installation, but the IDE is using some other one.

I have updated the test: https://gist.github.com/igrr/964b0d139ce63fcee07c8c1f1facb24d and Esp.zip, please give it another try. If the log ends up being long, could you please paste it into gist.github.com and post a link here?
Suggest doing this on the machine where you got Flash verify errors in the previous test.

ondabeach · 2018-01-08T01:00:25Z

@igrr No longer than the others I think.

phys_addr=438272
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 524288
Testing flash write bits...
Mask=0xffffffff
Mask=0xfffffffe
Mask=0xfffffffd
Mask=0xfffffffb
Mask=0xfffffff7
Mask=0xffffffef
Mask=0xffffffdf
Mask=0xffffffbf
Mask=0xffffff7f
Mask=0xfffffeff
Mask=0xfffffdff
Mask=0xfffffbff
Mask=0xfffff7ff
Mask=0xffffefff
Mask=0xffffdfff
Mask=0xffffbfff
Mask=0xffff7fff
Mask=0xfffeffff
Mask=0xfffdffff
Mask=0xfffbffff
Mask=0xfff7ffff
Mask=0xffefffff
Mask=0xffdfffff
Mask=0xffbfffff
Mask=0xff7fffff
Mask=0xfeffffff
Mask=0xfdffffff
Mask=0xfbffffff
Mask=0xf7ffffff
Mask=0xefffffff
Mask=0xdfffffff
Mask=0xbfffffff
Mask=0x7fffffff
Testing flash random write...
Seed=0 Start=107 Count=16
Seed=1 Start=107 Count=16
Seed=2 Start=107 Count=16
Seed=3 Start=107 Count=16
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Flash write verify error addr=0x6b0fc, orig=0x0000ffff, data=0xffff0429, read=0xfff70429
Flash write verify error addr=0x6c0fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x6d0fc, orig=0x0000ffff, data=0xffff0429, read=0xffdf0429
Flash write verify error addr=0x6e0fc, orig=0x0000ffff, data=0xffff0429, read=0xffef0429
Flash write verify error addr=0x6f0fc, orig=0x0000ffff, data=0xffff0429, read=0xefff0429
Flash write verify error addr=0x700fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x710fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x720fc, orig=0x0000ffff, data=0xffff0429, read=0x7fff0429
Flash write verify error addr=0x730fc, orig=0x0000ffff, data=0xffff0429, read=0x7fff0429
Flash write verify error addr=0x740fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x750fc, orig=0x0000ffff, data=0xffff0429, read=0xefdf0429
Flash write verify error addr=0x760fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x770fc, orig=0x0000ffff, data=0xffff0429, read=0xff770429
Flash write verify error addr=0x780fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x790fc, orig=0x0000ffff, data=0xffff0429, read=0xffd70429
Flash write verify error addr=0x7a0fc, orig=0x0000ffff, data=0xffff0429, read=0xbfff0429
Mounting FS...
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
Flash read verify (#0) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#1) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#2) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#0) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#1) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#2) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#0) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#1) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#2) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#0) error addr=0x790fc, data=0xffd70429, check=0xffdf0429
Flash read verify (#2) error addr=0x790fc, data=0xffd70429, check=0xffdf0429
Flash read verify (#0) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#1) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#2) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
37: Failed to open file
appendFile
File size=0
f.write returned 7
Flash write verify error addr=0x6b000, orig=0xffff8001, data=0x0001ffff, read=0x0001f57f
Flash write verify error addr=0x6b204, orig=0xfffffffc, data=0x6f6f66ff, read=0x6f6f66ff
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:

SPIFFS_close: fd=2

DarrenSchwartz · 2018-09-19T15:33:30Z

Take this file Esp.zip from @mrlightsman's post above. Replace it with the esp.cpp file in your arduino IDE program folder. I am using the portable version so it looks like this: C:\arduino-1.8.6\portable\packages\esp8266\hardware\esp8266\2.4.2\cores\esp8266\Esp.cpp. Just look for the 'packages' folder in your install directory and follow it down. Once the esp.cpp file is replaced, restart the IDE and flash your sketch using the "Tools -> Erase Flash -> All Flash Contents" option. After it's done flashing, go to "Tools -> ESP8266 Sketch Data Upload". This will upload all the files in the "data" subfolder of your sketch folder.

mcpicoli · 2018-10-05T20:47:04Z

I can confirm that @uzi18 v2 patch works nicely (thanks to everyone involved in creating it!). Using Arduino IDE, on some "Sonoff Basic" modules with PUYA chips.

However, I have some tips to give:

Make SURE you're patching the correct Esp.h/Esp.cpp files. Lost some time patching the wrong ones.
Make SURE you've cleaned any precompiled/cached versions. They're usually stored on (Windows) "<system_drive>/Users/<name_of_user>/AppData/Local/Temp/arduino_*". Kill them all and then restart the IDE. Also lost some significant amount of time before fixing this.
Make sure that you select (at least for the first upload) "Erase ALL Flash" under "Tools" -> "Erase Flash".
Clean the SPIFFS area using your favorite ESP8266 Sketch Data Uploader (answer "yes" to the question "do you want to upload a clean SPIFFS image?"). Your SPIFFS image MAY contain files already, but it must be pristine, that is, NOTHING may have been written to it after is was uploaded.
Having your sketch (after patch) write anything to an "unclean" SPIFFS (one that was "touched" by any kind of write from a pre-patch sketch) doesn't make it "clean".
Any kind of write from a pre-patch sketch to any SPIFFS image will make it unclean. Patching and reuploading your sketch will NOT work. You've got to reupload your SPIFFS image and then upload your sketch after patching before the old one still in flash writes anything to SPIFFS. I solved this by adding a large temporary delay to my sketch, giving me time to put it again in flash mode after SPIFFS image upload and before actual old sketch execution.

PaulBortnikov · 2018-10-08T23:08:41Z

Edit file Esp.cpp:

 bool EspClass::flashWrite(uint32_t offset, uint32_t *data, size_t size) {
    static uint32_t flash_chip_id = 0;
        uint32_t *read_buf=NULL;
    if (flash_chip_id == 0)
        flash_chip_id = getFlashChipId();
    ets_isr_mask(FLASH_INT_MASK);
    int rc;
    uint32_t* ptr = data;
    if ((flash_chip_id & 0x000000ff) == 0x85) { // 0x146085 PUYA
        read_buf=new uint32_t [SPI_FLASH_SEC_SIZE / 4];
        if(!read_buf) return false;
        rc = spi_flash_read(offset, read_buf, size);
        if (rc != 0) {
            delete read_buf;
            ets_isr_unmask(FLASH_INT_MASK);
            return false;
        }
        for (size_t i = 0; i < size / 4; ++i) {
            read_buf[i] &= data[i];
        }
        ptr = read_buf;
    }
    rc = spi_flash_write(offset, ptr, size);
        if(read_buf!=NULL) delete read_buf;
    ets_isr_unmask(FLASH_INT_MASK);
    return rc == 0;
}

@uzi18

Applied this patch made by @uzi18 esp8266/Arduino#4061 (comment) It is now included in all builds

@uzi18

Applied this patch made by @uzi18 esp8266/Arduino#4061 (comment) It is now included in all builds

uzi18 · 2018-11-07T11:24:18Z

@ALL updated puya patch v3 used now in ESPEasy project:
https://github.com/letscontrolit/ESPEasy/blob/mega/patches/puya_v3.patch#L8

maxim-kukushkin · 2018-12-01T14:01:09Z

The described fix (flashWrite function) worked for me as well with Sonoff S26, which also uses PUYA flash. Any plans to perform the change in mainline?

mrWheel · 2018-12-13T10:38:23Z

Any plans to perform the change in mainline?
Is this patch part of 2.5.0-Beta1?

devyte · 2018-12-13T12:50:07Z

It is not in -beta1. A variation of the patch used in ESPEasy will be used in the core, but it's not targeted to a release yet.

TD-er · 2018-12-13T12:53:00Z

@devyte Just curious, what will be different?

mrWheel · 2018-12-13T12:56:17Z

Ok, so all the people that use the ESP-01 with (almost all nowadays) a PUYA chip cannot upgrade to a new version in the foreseeable future ?!
If I may ask: What's holding them up?

maxim-kukushkin · 2018-12-13T13:19:34Z

Not only ESP-01 but many other new devices. Sonoff S26 is one of the examples

devyte · 2018-12-13T18:46:15Z

@devyte Just curious, what will be different?

The patch:

uses a static 4KB buffer, which means everyone using the patch has 4KB less heap, including those users who do not have a puya chip. Instead, a dynamically allocated buffer should be used.
evaluates the first if condition every single call of the function, although only the first one after boot actually does anything. Instead of a private static, a global could be used that is set elsewhere only once at bootup, and then if condition is no longer needed. Alternatively, The ESPClass could get a new data member (i.e.: instead of the global flag) which is set on construction, and a method to return its value.
calculates the flash_chip_id condition in the second if every single call of the function, which makes no sense, because it can't ever change at runtime. Instead, the condition should be fully calculated once and the result stored in a flag, which should then be checked.
detects the condition based on flash_chip_id, which isn't very robust, as has been shown with the flash chip size. Instead, a general purpose detection approach should be implemented that transparently covers other potential brands and manufacturers that have the same quirk, should they appear.
Example: on first bootup after flashing, choose a sector, e.g.: within the update area of the flash layout., preferably one that is unlikely to be used. Do a write test and read back sequence known to behave differently with puya chips. If puya is detected, set a quirk flag somewhere. On all successive bootups, detect that the quirk flag is set, so don't do the test again. Use the quirk flag in the write function to decide whether the additional logic is done or not.

Given the above, the patch won't be accepted as-is into our core. There have been several internal discussions on exactly what to do, but given that the patch is available, there have been, and still are, higher priorities, e.g.: figuring out how to ease IRAM usage.

There have been a lot of comments from many users in this discussion, and I see the patch is even being applied elsewhere, but I still don't see a PR here. Instead of wondering what will happen, I invite those involved to make a PR with the proposed patch, and implement at least points 1-3 above on top of it. If you come up with a good solution for point 4, even better.

maxim-kukushkin · 2018-12-13T19:32:31Z

While do understand that the expected behavior in open source is to some up with a proper patch for an issue, I tend to disagree with this point:
"given that the patch is available, there have been, and still are, higher priorities"

No. You've just indicated that the patch is a really basic workaround, apparently not suitable for any production use. People only use it as PoC because it makes their devices work at least somehow, comparing to not working at all

So, if there's any prioritization done, pease consider this issue as not having any workaround at all, as apparently there's no candidate for a merge

Current firmware literally doesn't work on recent chips. Also, comparing to other issues, at least some investigation has been done by the community, identifying the root cause and a PoC fix

mrWheel · 2018-12-13T19:45:48Z

Totally agree with maxim-kukushkin

earlephilhower · 2018-12-13T19:49:47Z

@maxim-kukushkin and @mrWheel everyone else on the thread: you obviously care very much about the Puya patch.

Please, one of you take what @devyte has suggested and wrap it into a PR so we can advance.

mrWheel · 2018-12-13T20:27:42Z

@maxim-kukushkin & @devyte
I would if only I new how ...
But yes, your right: I try to make software for devices with ESP-01’s and thats not without problems!

TD-er · 2018-12-13T20:42:06Z

@devyte Sounds like very legitimate points.
I didn't have a look at it myself, but wasting 4k of heap is reason enough to not apply it like it is now.
When is the deadline for 2.5.0? ;)

devyte · 2018-12-14T03:01:10Z

Given the length of this thread, I'm closing in favor of #5493 . Resolution will be handled there.

see esp8266/Arduino#4061 (comment)

devyte closed this as completed Jan 6, 2018

weswitt mentioned this issue Jan 6, 2018

FLASH_SIZE problem > update error #3616

Closed

igrr reopened this Jan 7, 2018

javiercuellar73 mentioned this issue Oct 24, 2018

Crash on SPIFFS.format() or SPIFFS.begin() only with ESP-12s. #5275

Closed

6 tasks

robynjayqueerie mentioned this issue Oct 25, 2018

v2.4.0 and Soft WDT reset while WiFi.forceSleepBegin() execution #4082

Closed

uzi18 mentioned this issue Nov 5, 2018

SONOFF POW PUYA 4MB letscontrolit/ESPEasy#2003

Closed

TD-er added a commit to TD-er/ESPEasy that referenced this issue Nov 6, 2018

[PUYA] Apply generic patch on all builds checking for PUYA flash

b1374dc

Applied this patch made by @uzi18 esp8266/Arduino#4061 (comment) It is now included in all builds

TD-er added a commit to TD-er/ESPEasy that referenced this issue Nov 6, 2018

[PUYA] Apply generic patch on all builds checking for PUYA flash

0a72b71

Applied this patch made by @uzi18 esp8266/Arduino#4061 (comment) It is now included in all builds

TD-er mentioned this issue Nov 6, 2018

[PUYA] Apply generic patch on all builds checking for PUYA flash letscontrolit/ESPEasy#2005

Merged

renat2985 mentioned this issue Nov 13, 2018

PUYA flash chip does not have SPIFFS available. SpacehuhnTech/esp8266_deauther#980

Closed

zombodotcom mentioned this issue Dec 4, 2018

esp8266-fastled-audio jasoncoon/esp8266-fastled-audio#7

Open

renat2985 mentioned this issue Dec 11, 2018

esp-01 на p25q80h. ошибки в работе tretyakovsa/Sonoff_WiFi_switch#115

Open

TD-er mentioned this issue Dec 14, 2018

Rewrite PUYA patch to be more universal and mem friendly. #5493

Closed

devyte closed this as completed Dec 14, 2018

radimkeseg added a commit to radimkeseg/esp8266-DS18B20-DeepSleep that referenced this issue Mar 20, 2019

link to PUYA SPIFF issue added

156c80f

see esp8266/Arduino#4061 (comment)

drzony mentioned this issue Aug 11, 2020

New flash writing method with offset/memory/size alignment handling #7514

Merged

Writing files with SPIFFS no longer working for ESP-01 #4061

Writing files with SPIFFS no longer working for ESP-01 #4061

Comments

funtastic99 commented Jan 1, 2018 • edited Loading

Basic Infos

Hardware

Description

Settings in IDE

Sketch

Debug Messages

funtastic99 commented Jan 3, 2018

ondabeach commented Jan 5, 2018

funtastic99 commented Jan 5, 2018

Pablo2048 commented Jan 5, 2018

funtastic99 commented Jan 5, 2018

ondabeach commented Jan 5, 2018

weswitt commented Jan 6, 2018

ondabeach commented Jan 6, 2018

devyte commented Jan 6, 2018

weswitt commented Jan 6, 2018

ondabeach commented Jan 6, 2018

weswitt commented Jan 7, 2018

ondabeach commented Jan 7, 2018

ondabeach commented Jan 7, 2018

ondabeach commented Jan 7, 2018

igrr commented Jan 7, 2018

ondabeach commented Jan 7, 2018

igrr commented Jan 7, 2018

funtastic99 commented Jan 7, 2018

igrr commented Jan 7, 2018

funtastic99 commented Jan 7, 2018

ondabeach commented Jan 7, 2018

igrr commented Jan 7, 2018

ondabeach commented Jan 7, 2018

ondabeach commented Jan 7, 2018

igrr commented Jan 7, 2018

ondabeach commented Jan 8, 2018

DarrenSchwartz commented Sep 19, 2018

mcpicoli commented Oct 5, 2018

PaulBortnikov commented Oct 8, 2018 • edited Loading

uzi18 commented Nov 7, 2018 • edited Loading

maxim-kukushkin commented Dec 1, 2018

mrWheel commented Dec 13, 2018

devyte commented Dec 13, 2018

TD-er commented Dec 13, 2018

mrWheel commented Dec 13, 2018

maxim-kukushkin commented Dec 13, 2018

devyte commented Dec 13, 2018

maxim-kukushkin commented Dec 13, 2018

mrWheel commented Dec 13, 2018

earlephilhower commented Dec 13, 2018

mrWheel commented Dec 13, 2018

TD-er commented Dec 13, 2018

devyte commented Dec 14, 2018

funtastic99 commented Jan 1, 2018 •

edited

Loading

PaulBortnikov commented Oct 8, 2018 •

edited

Loading

uzi18 commented Nov 7, 2018 •

edited

Loading