I2C Fail 3.2.0 #11228
Comments
|
I also see strange behavior on the I2c bus. Serial debug is disabled, but I get the message - i2c.master: i2c transaction timeout detected. Full log: E (1124) i2c.master: I2C transaction timeout detected |
Pleas provide full information and minimal compilable code example |
|
I am also experiencing this issue. It occurred after Updating Arduino IDE to 2.3.5 from 2.3.4. The ESP IDF also updated at this time. I had never seen this error previously. |
|
@lucasssvaz I noticed some changes attributed to you in the most recent update, any chances that is what’s causing the issue? test(wokwi): Add I2C Master test and enable GPIO and PSRAM tests by @lucasssvaz in #10848 test(i2c): Add test to scan bus by @lucasssvaz in #11022 test(i2c): Do not use delta as Wokwi timing can be inconsistent by @lucasssvaz in #11080 |
|
@MarksWizard These PRs are related only to our I2C test in the CI. It does not affect how I2C works in any way. |
Jason2866
added
Area: Peripherals API
|
After a bunch of testing, and even more hair pulling... I think it ended up being a wiring issue for me. I finally got code to run without the error, but it would still be nice if that message didn’t spam the serial like it does. |
|
I see the same issue, but I can confirm that it does not seem to be a wiring issue. |
|
Please list devices and libraries that you use with them, so that we can try to replicate any of the confirmed non-wiring issues. |
|
If everything is wired correctly, the sketch runs fine. If you disconnect a single wire from the OLED and reboot it will trigger the issue. It’s not specifically related to the OLED though, the issue seems to occur when you initialize an I2C device that isn’t connected. Unfortunately the solution isn’t simply “check your wiring”, because this completely eliminates the ability to test code “headless” without a sensor or screen attached. Often times I test code on just an unwired board just to make sure everything is working correctly before I upload to the actual device. Previous to the ESP update, typical initialization code would just tell you the sensor wasn’t present and would maybe report something like -127 or whatever, which is perfectly fine when I’m testing something that doesn’t actually need the sensor to be present. Now it spams the serial with the error messages and the official ESP IDE I2C_oled example I tested caused the device into a boot/crash loop. I can get it to trigger using the code below, it should trigger before printing “Wire Started” if you don’t have an I2C device connected and should run fine if you do. #include <Arduino.h> //Library for using Arduinio IDE specific commands
#include <Adafruit_GFX.h> //https://github.com/adafruit/Adafruit-GFX-Library
#include <Adafruit_SSD1306.h> //https://github.com/adafruit/Adafruit_SSD1306
#define SCREEN_WIDTH 128 // OLED display width, in pixels
#define SCREEN_HEIGHT 64 // OLED display height, in pixels
Adafruit_SSD1306 display(SCREEN_WIDTH, SCREEN_HEIGHT, &Wire, -1);
int timer = 0;
#define SDA 12 //variables for OLED display
#define SCL 13
void setup() {
delay(50);
Serial.begin(115200); //Begin serial connection
Serial.println("Initializing display”);
delay(1000);
Wire.begin(SDA, SCL);
Serial.println("Wire started”);
delay(1000);
if(!display.begin(SSD1306_SWITCHCAPVCC, 0x3C)) { // Address 0x3D for 128x64
Serial.println(F("SSD1306 allocation failed”));
}
display.clearDisplay();
Serial.println("Display Ready”);
}
void loop() {
while (timer < 10){
display.clearDisplay();
display.setTextColor(WHITE);
display.setTextSize(3);
display.setCursor(0,0);
display.print(timer);
display.display();
timer = timer + 1;
delay(1000);
}
} |
|
@MarksWizard so this is rather a logging issue. Your terminal gets flooded when there is no device. This I would say is oK. You should check if the device is there and not issue any more I2C transactions. That would stop the logs. I understand this is change in behavior, but that change comes from ESP-IDF's driver and not from Arduino, so we can not control it (other than blank stop it and not allow any logs, which is also not a good idea). I am looking for a case where everything was working before and is no longer working at all with the new driver and proper wiring. |
|
@me-no-dev |
|
@HanksSaloon sure! Please also give some basic info of the setup. What devices you have connected, how do you manage communication with them and pointers to what should we look at in the traces |
|
@me-no-dev first thanks for all of your help with this, and everything you do on the other projects.. I would not be able to play with these things without all of the hard work you have contributed. I understand that it’s not an issue with Arduino, since the ESP IDF example code causes the issue. With the new update they changed some I2C stuff and added these error messages I guess. However it’s bad code behavior as there is no rate limit, flag or anything to limit the error messages. In the ESP IDF example it causes the device to boot/crash loop, which I can’t imagine is the preferred behavior if a simple I2C device becomes disconnected. Your suggestion to not send any I2C if not connected seems absurd? So before every screen refresh or sensor read, I need to check that the actual device is present? That would add 100’s of lines of code to my project. This issue breaks 1000’s of example code found online that doesn’t involve those checks. EDIT: actually this is not possible, since initializing the sensor is what causes the issue.. and we don’t know if it’s present before we try to initialize. I have not been able to progress my own code because I have been trying to deal with this issue. Unfortunately I feel like in trying initially determine and explain the issue I made things confusing. Now that I know exactly where the issue lies, can you please help me properly explain this to the ESP devs? |
|
Ok, here goes: Hardware I/O DEVICE ; Type ; address; I2C-ch; Library All libraries have been slightly modified, but only to the extent of adding or removing some interface routines. Physically, all circuits are on the same 100x100mm PCB, except for the ACS37800 which is located about 8 meters away. Standard #includes are To explain the analyzer outputs: The RTC-module sends a 1Hz interrupt signal to pin D25. The interrupt handling routine couldn't be simpler, it just increases the value of a semaphore. Takes minimum of time. The semaphore is then processed elsewhere in the S process. Both processes poll their I2C channel regularly, process S once every second, process R according to it's own internal logic. Finally, the samples: At first, the two outputs seem very similar and there is no clear explanation to what the problem is in 3.2.0. In 3.2.0 SCL is released much later. Compare these examples: I haven't had time to check the standard if there's a time limit when the SCL must be released after the data is sent, but as a delayed SCL release also delays the ACK/NACK response from the slave it seems that the ESP (ver 3.2.0) doesn't understand where the response comes from and therefore logs an error condition. |
|
@HanksSaloon awesome work, thanks for helping to get this resolved. Can you post all of the above info in the issue opened on the ESP GitHub? Mythbuster5 has been asking for the logic analyzer stuff which I can’t provide. |
|
@HanksSaloon How to open the DMS file in your zip. Or it's not dma, because I download from mac. So the question is how to open that file? Or what's the correct extention name? |
|
Or just provide key picture is ok I think. Do you mean this part is suspicious？
|
|
The .Zip contains two files, <A_ESP32-3_1_3.sr> and <A_ESP32-3_2_0.sr>. They can be opened in PulseView / Sigrok.
|
|
Here are two more screenshots, of I2C channel 2. Highlight the difference.
|
|
Got it. |
|
0001-fix-i2c-optimization-performance-on-esp32.txt @HanksSaloon I have generated patch for shorten the time between last byte and stop. If your every transaction data is smaller than 63 byte (include address byte), you can try this. However, data in one transaction is larger than 63 bytes still have some problem, I still need time for debugging. Thank you. @me-no-dev PTAL, and please help to share a method how to patch idf code in ardiuno environment if needed. Thanks. |
|
@mythbuster5 @me-no-dev |
|
@mythbuster5 here is how the issue is manifesting for me in Arduino IDE IMG_6624.mov |
|
@enjoyneering @HanksSaloon patched libs can be downloaded from here: https://github.com/espressif/esp32-arduino-lib-builder/actions/runs/14531871574/artifacts/2968983657 You need to replace the ones in |
|
@me-no-dev Sorry, but you need to explain a little more where the patches should go. |
|
@HanksSaloon please see this: espressif/esp-idf#15734 (comment) |
|
@HanksSaloon also good idea to uninstall from BoardManager, so that you do not mistake which core you are compiling with |
|
I am sorry.. but I feel like I am screaming into the void at this point. To me, it seems like both of you are failing to acknowledge that the error message spamming the serial on every I2C read/write is bad behavior and not at all how things were previously. Checking the I2C before every action would add 100’s of useless lines, that aren’t included in any example online. Numerous users have reported even with serial debugging turned off, these messages still come though. Currently there is no way to turn them off. Again that’s not proper behavior, and if you’re going to force it on us at least rate limit it. In my “production” code I have 2 I2C sensors and a screen. In ESP 5.4, when I upload this code to a bare dev board, the serial is completely unreadable because of the wall of error messages. Again this should not be the case, I have done everything I can possibly think of to explain and illustrate the issue and how I am affected by it. In the videos below, you can see that under ESP 5.3 the code runs fine, even though sensors are not present. The software actually picks up that the sensors are not present and report this in the output. On ESP 5.4 it causes a boot/crash loop for me. This will be my final comment on the issue, I’ll stay on ESP 5.3 and hope that it eventually gets resolved. Arduino 2.3.4 w/ [v3.1.3 based on ESP-IDF v5.3] IMG_6630.movArduino 2.3.5 w/ [v3.2.0 based on ESP-IDF v5.4] IMG_6633.mov |
|
@MarksWizard here is what is happening: You are annoyed that you are doing something wrong and that the logs are telling you that. How is that a bad behavior of the driver? Wouldn't it be better to write proper code and not waste resources of the chip talking to device that you have yourself disconnected? If you are doing what you are supposed to: have the device connected and/or check if it's connected before trying to access it, then you will not be getting errors, unless something else goes wrong, at which point you will like having the logs tell you that and not just have a broken device. |
|
None of the examples online have this error check before every I2C read/write. Please post a very basic example of what that would look like. I use the below code to initializes the SHT31 temp/humidity sensor. I do keep track of whether the sensor is connected or not, but this is only checked at initialization. This does not catch cases where the sensor becomes disconnected or misreads somehow. I would have to add a check to the SHT31_FLAG before every read from the sensor? |
|
@MarksWizard that should be done in the library that you are using to talk to the device, not your user code. But here is a sample anyway: #include <Arduino.h> //Library for using Arduinio IDE specific commands
#include <Adafruit_GFX.h> //https://github.com/adafruit/Adafruit-GFX-Library
#include <Adafruit_SSD1306.h> //https://github.com/adafruit/Adafruit_SSD1306
#define SCREEN_WIDTH 128 // OLED display width, in pixels
#define SCREEN_HEIGHT 64 // OLED display height, in pixels
Adafruit_SSD1306 display(SCREEN_WIDTH, SCREEN_HEIGHT, &Wire, -1);
int timer = 0;
#define SDA 12 //variables for OLED display
#define SCL 13
bool displayConnected = false;
bool isDeviceConnected(uint8_t address) {
Wire.beginTransmission(address);
return Wire.endTransmission() == 0;
}
void setup() {
delay(50);
Serial.begin(115200); //Begin serial connection
Serial.println("Initializing display”);
delay(1000);
Wire.begin(SDA, SCL);
Serial.println("Wire started”);
delay(1000);
displayConnected = isDeviceConnected(0x3c);
if(!displayConnected){
Serial.println(F("SSD1306 not connected”));
return;
}
if(!display.begin(SSD1306_SWITCHCAPVCC, 0x3C)) { // Address 0x3D for 128x64
Serial.println(F("SSD1306 allocation failed”));
}
display.clearDisplay();
Serial.println("Display Ready”);
}
void loop() {
while (displayConnected && timer < 10){
display.clearDisplay();
display.setTextColor(WHITE);
display.setTextSize(3);
display.setCursor(0,0);
display.print(timer);
display.display();
timer = timer + 1;
delay(1000);
}
} |
|
@me-no-dev yeah in your example you are doing a check before printing to the screen. This means anytime I output anything to the screen, or read a sensor I need to do the check, which equals 100’s of lines for me. Your suggestion means that every library for the I2C devices is currently “broke”. |
|
I'm not sure where you see that, but OK. Also getting a log that something is wrong is not equal to "broken". I offered you two approaches: check if the device is connected or just make sure that you have it connected in reality. You refuse both... you want to not connect the screen and to be able to write to it and not get an error message. This is not OK. |
|
@enjoyneering @HanksSaloon looking forward to your results |
|
@me-no-dev Does espressif support require that the IDE is natively installed, i.e not running from an image? |
|
I think I have the same problem. I am using a feather ESP32S3 and a BNO085 via i2c. On esp32 core 3.1.3 everything worked fine. I updated to 3.2.0 and the BNO085 begin method returned i2c not found. I then ran a simple i2c scanner (WireScan in the examples) and it returned an error 5 (timeout) on the address of the BNO085. I then installed 3.1.3 and everything worked fine. So it looks like 3.2.0 has broken the i2c for BNO085, other i2c devices seem to work fine Here is the code from WireScan void setup() { loop() { delay(5000); Serial.println("Scanning for I2C devices ..."); |
|
@me-no-dev I apologize for being reactionary, I recognize how much you do and you don’t deserve that. In your example, you use displayConnected = isDeviceConnected(0x3c); as the flag to catch if the display is connected. That means before every display.display() or sensor.read() in my code I have to add if displayConnected{}, which happens 100’s of times. I agree having this handled in the library is the easiest, and I can do that for the libraries that I am using locally. But I am using all publicly available libraries, which I do not maintain and therefore doesn’t help the next person. I know that the error messages are useful, but again this was not the behavior before the update, and breaks all current examples / libraries. So the real solution is that something in the way that error is handled/output should be changed. I know this issue doesn’t fall on Arduino, or is even possible to “patch” on your side. However, multiple users are reporting this. My frustration is that I feel like I am the only one actually trying to resolve it. Running the OLED screen example I posted above on an Arduino Mega does not cause any such error messages either, which as a user is the currently expected “normal”. |
|
@HanksSaloon I suspect you did not install in the correct location, forgot to run get.exe/get.py and then overwrite the libs that were installed with the ones in the link above. Custom cores are installed in your Arduino sketches folder: |
|
@davidcliddell you can try to install the custom core with the patched libs and see if that will help. |
|
@MarksWizard if your display is properly connected, it will work and errors will not be displayed. That means that there is no bug in either the library or the I2C driver. Most libraries out there assume that you have a working screen connected properly to the board and that is working just fine. Yes, AVR core does not print logs or anythng, mostly because it can not afford the memory penalty to do so, but ESP32 chips have plenty of that and can actually tell you when something is wrong. For very custom cases like yours, we offer Arduino as IDF component (where you can turn off error logs in IDF) and also a way to compile your custom IDF libs with logs off. We can not blankly turn all errors off, because then we would not know when something goes wrong. It is obvious that writing to device that is not connected at all is not a valid case. Why write code for it if you are not going to connect to it anyway? |
|
@mythbuster5 will your patch fix this issue: #11228 (comment) |
|
@me-no-dev thank you for taking the time to respond again. The issue occurs for me because my the device I’m writing code to is to simulate tidal changes in a grow bin for a research experiment. The device has 2 motors, 2 flow sensors, air temp/humidity, water temp, time of flight sensor and a screen. I also use your web server code to change parameters and such. These are all connected and working great, so I do deeply appreciate what you do because none of it would be possible without it. However, if I want to test some small piece of code on a “headless” board at home or my desk, I currently can’t under ESP-IDF 5.4. If one of the I2C devices becomes disconnected, the board will freak out. I don’t always care if I can’t read temp or see the screen, previously my code handled these scenarios just fine. This is why I am being stubborn, but I will keep my word to roll back to 5.3 and hope for the best. Thanks again, have a great weekend. |
|
@MarksWizard here is my last option for you. Same number of lines, but you address the #include "Wire.h"
#include <Adafruit_GFX.h> //https://github.com/adafruit/Adafruit-GFX-Library
#include <Adafruit_SSD1306.h> //https://github.com/adafruit/Adafruit_SSD1306
#define SCREEN_WIDTH 128 // OLED display width, in pixels
#define SCREEN_HEIGHT 64 // OLED display height, in pixels
Adafruit_SSD1306 display(SCREEN_WIDTH, SCREEN_HEIGHT, &Wire, -1);
int timer = 0;
#define SDA 12 //variables for OLED display
#define SCL 13
bool displayConnected = false;
bool isDeviceConnected(uint8_t address) {
Wire.beginTransmission(address);
return Wire.endTransmission() == 0;
}
#define oled if(displayConnected) display
void setup(){
Serial.begin(115200);
displayConnected = isDeviceConnected(0x3c);
if(!displayConnected){
Serial.println(F("SSD1306 not connected"));
return;
}
if(!display.begin(SSD1306_SWITCHCAPVCC, 0x3C)) { // Address 0x3D for 128x64
Serial.println(F("SSD1306 allocation failed"));
}
oled.clearDisplay();
Serial.println("Display Ready");
}
void loop(){
while (timer < 10){
oled.clearDisplay();
oled.setTextColor(WHITE);
oled.setTextSize(3);
oled.setCursor(0,0);
oled.print(timer);
oled.display();
timer = timer + 1;
delay(1000);
}
} |
|
@me-no-dev thank you. Is there a way for me to tip you or buy you lunch? |
|
@me-no-dev I uploaded your example code, and the issue persists. The only thing that you were missing in your code is Wire.begin(SDA,SCL);, which I included after Serial.begin(115200); Edit: Nevermind, I realize that everything works properly on reboot, since It’s the initialization where we set the flag. This solution works for me, and I will work on making the appropriate changes in my code. It will be better for it in the end, so thank you again for all of your help. Please let me know if I can send a tip. IMG_6643.mov |
|
@me-no-dev I ran the installation (a lot of stuff installed), all the way including the last "python3 get.py. (That script installed even more). Anyway, I've tried both options. In both cases Arduino IDE starts with the following two messages. First one is an error msg, second is a suggestion to install the required board drivers:
|
|
@me-no-dev |
|
@me-no-dev @mythbuster5
I had to clean out (i.e renamed) all previous Arduino IDE settings from the hidden files and folders, by renaming them in order to hide them from Arduino IDE: So now, a clean start with the patch. ~/Arduino/hardware/espressif/esp32/cores/esp32/esp32-hal-uart.c: In function '_uartDetachPins': Compilation error: exit status 1 |
|
@HanksSaloon when you cloned the Arduino repository in the hardware folder, did you make sure that you are using the |
|
I cannot answer that directly, as I did exactly according your instructions, except for two steps. I am on OpenSUSE, so I did according to the instructions you pointed me to: However, at that point Arduino IDE 2.3.6 was already installed (AppImage) and I (user) already member of the dialout group. So I went directly to: sudo zypper install git python3-pip python3-pyserial. Continued with these commands: Finally: The last step, and this is where I did differently from your instructions: I replaced the files and folders in ~/Arduino/hardware/espressif/esp32/tools/esp32-arduino-libs with the ones that I had downloaded earlier from https://github.com/espressif/esp32-arduino-lib-builder/actions/runs/14531871574/artifacts/2968983657 And, as I explained earlier today, I decided to try one more thing and that was a fresh install of Arduino IDE, (not the Appimage). Looking at the scripts the stuff has been cloned from the branch in https://github.com/espressif/arduino-esp32. |
|
You cloned the master branch. Proper command should have been |
|
@HanksSaloon there is another change that we need to make for UART to compile with current IDF master. Will make the change and let you know that is clear to update the repository and try to compile. The error we are currently getting is https://github.com/espressif/arduino-esp32/actions/runs/14535332246/job/40782515042#step:6:123 |
|
@me-no-dev Allright! I'll wait for the update. And thanks! I really appreciate your effort. |
Board
ESP32 - WROOM
Device Description
Custom ESE32 Wroom .
Hardware Configuration
GPIO 33,32, Wire library.
Version
v3.2.0
IDE Name
ARD
Operating System
Windows 11
Flash frequency
40
PSRAM enabled
yes
Upload speed
921600
Description
Compare to 3.1.3
3.2.0 Reported with 2 issues :
1-Executing i2c call time increase.
2-Deadlock [I2C hardware NACK detected] .
Sketch
Debug Message
Other Steps to Reproduce
No response
I have checked existing issues, online documentation and the Troubleshooting Guide
The text was updated successfully, but these errors were encountered: