Skip to content

Support Arduino ESP32 v3.0 based on ESP-IDF v5.1 #1225

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ivankravets opened this issue Nov 4, 2023 · 321 comments
Open

Support Arduino ESP32 v3.0 based on ESP-IDF v5.1 #1225

ivankravets opened this issue Nov 4, 2023 · 321 comments
Labels

Comments

@ivankravets
Copy link
Member

ivankravets commented Nov 4, 2023

The ESP32 Core for Arduino 2.x is the most recent major version currently recommended for use with PlatformIO. The decision to discontinue support was made by the Espressif company, as indicated in their official statement: espressif/arduino-esp32#8606

If you're an Espressif user who relies on PlatformIO for your projects, kindly share your thoughts in the comments below and cast your vote on this matter. Once we gather enough interest from the Espressif community, we plan to reach out to the Espressif company and request a reconsideration of support for PlatformIO.

@thijstriemstra
Copy link

thijstriemstra commented Nov 8, 2023

I've been trying this with following config:

; see https://docs.espressif.com/projects/arduino-esp32/en/latest/installing.html#how-to-update-to-the-latest-code
[env:arduino-latest]
platform = https://github.com/platformio/platform-espressif32.git
board = az-delivery-devkit-v4
framework = arduino
platform_packages =
    framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32#master

but the build fails with KeyError: 'framework-arduinoespressif32-libs':

Processing arduino-latest (platform: https://github.com/platformio/platform-espressif32.git; board: az-delivery-devkit-v4; framework: arduino)
--------------------------------------------------------------------------------
Platform Manager: Installing git+https://github.com/platformio/platform-espressif32.git
git version 2.42.0
Cloning into '/home/runner/.platformio/.cache/tmp/pkg-installing-uz1yxjz0'...
Platform Manager: [email protected]+sha.f6ec392 has been installed!
Tool Manager: Installing git+https://github.com/espressif/arduino-esp32#master
git version 2.42.0
Cloning into '/home/runner/.platformio/.cache/tmp/pkg-installing-kih232ry'...
Tool Manager: [email protected]+sha.bf822ef has been installed!
Library Manager: Installing fortyseveneffects/MIDI Library @ ^5.0.2
Unpacking 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Library Manager: MIDI [email protected] has been installed!
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/az-delivery-devkit-v4.html
PLATFORM: Espressif 32 (6.4.0+sha.f6ec392) > AZ-Delivery ESP-32 Dev Kit C V4
HARDWARE: ESP32 240MHz, 520KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 3.0.0+sha.bf822ef 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Converting edrumulus.ino
/tmp/tmpj5gstour/src/edrumulus.ino:45: warning: "MYMIDI" redefined
 #  define MYMIDI                     usbMIDI
 
/tmp/tmpj5gstour/src/edrumulus.ino:39: note: this is the location of the previous definition
 #  define MYMIDI                     MIDI
 
/tmp/tmpj5gstour/src/edrumulus.ino:46: warning: "MIDI_CONTROL_CHANGE_TYPE" redefined
 #  define MIDI_CONTROL_CHANGE_TYPE   usbMIDI.ControlChange
 
/tmp/tmpj5gstour/src/edrumulus.ino:40: note: this is the location of the previous definition
 #  define MIDI_CONTROL_CHANGE_TYPE   midi::ControlChange
 
/tmp/tmpj5gstour/src/edrumulus.ino:47: warning: "MIDI_SEND_AFTER_TOUCH" redefined
 #  define MIDI_SEND_AFTER_TOUCH      sendAfterTouchPoly
 
/tmp/tmpj5gstour/src/edrumulus.ino:41: note: this is the location of the previous definition
 #  define MIDI_SEND_AFTER_TOUCH      sendAfterTouch
 
KeyError: 'framework-arduinoespressif32-libs':
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/runner/.platformio/platforms/espressif32@src-5f117260f75b328038ec9d3fd0e14a68/builder/main.py", line 346:
    target_elf = env.BuildProgram()
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 61:
    env.ProcessProgramDeps()
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 121:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 342:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 662:
    return method(*args, **kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/runner/.platformio/platforms/espressif32@src-5f117260f75b328038ec9d3fd0e14a68/builder/frameworks/arduino.py", line 41:
    SConscript(
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 662:
    return method(*args, **kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/runner/.platformio/packages/framework-arduinoespressif32/tools/platformio-build.py", line 40:
    FRAMEWORK_LIBS_DIR = platform.get_package_dir("framework-arduinoespressif32-libs")
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/platform/_packages.py", line 32:
    pkg = self.get_package(name)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/platform/_packages.py", line 29:
    return self.pm.get_package(spec or self.get_package_spec(name))
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/platform/_packages.py", line 21:
    owner=self.packages[name].get("owner"),
========================== [FAILED] Took 4.31 seconds ==========================

@ivankravets
Copy link
Member Author

See the updated comment #1225 (comment)

@Jason2866
Copy link
Contributor

Jason2866 commented Nov 10, 2023

Adding Arduino 3.0.0 support is the only way to make the ESP32-C6 available for Arduino projects.
The needed changes are not that many, i have added Arduino 3.0.x alpha support in my Fork and bulding Arduino and IDF projects with the C6 are working.
We have ported Arduino project Tasmota to the C6 and it runs already surprisingly stable.

@Jason2866
Copy link
Contributor

Jason2866 commented Nov 10, 2023

If you're an Espressif user who relies on PlatformIO for your projects, kindly share your thoughts in the comments below and cast your vote on this matter. Once we gather enough interest from the Espressif community, we plan to reach out to the Espressif company and request a reconsideration of support for PlatformIO.

@ivankravets Does this mean that Espressif has dropped any support? Looking in current Arduino 3.0.0 code (branch master) everything regarding Platformio support is there.

@VojtechBartoska
Copy link

Hello all,

I do work in Arduino Core ESP32 project as a Project Manager so I would like to clarify this from our Team point of view.

Please take a look on this comment in original Pull request.

We also update a warning note in our documentation which should be a bit confusing.

To not duplicate what is written in original PR, we will be glad if PlatformIO supports our latest version and we are open to any contributions.

@ivankravets Feel free to contact us any time directly, we will be happy to discuss this if you see any points we can do differently.

Thanks

@0x0fe
Copy link

0x0fe commented Nov 22, 2023

Adding Arduino 3.0.0 support is the only way to make the ESP32-C6 available for Arduino projects.
The needed changes are not that many, i have added Arduino 3.0.x alpha support in my Fork and bulding Arduino and IDF projects with the C6 are working.
We have ported Arduino project Tasmota to the C6 and it runs already surprisingly stable.

@Jason2866
Do you know how i can setup the .ini file in a new project for using your 3.0.0 fork and 5.1 IDF at the same time (to keep the SDK menuConfig)?

@0x0fe
Copy link

0x0fe commented Nov 22, 2023

@ivankravets
there is something i dont understand, why is there a separate platformespressif32 core? It is based on arduinoespressif32, but always lags behind, and arduinoesp32 already lags behind IDF. So what is the point of this separate core? Why is PIO not using arduinoesp32 core as is directly?

I think a reason why many users use PIO is also to keep some of the arduinoesp32 APIs while having the possibility to modify the IDF SDK configuration (menuconfig) which is necessary in many cases and not possible in arduino, unless using the libbuilder (buggy and not convenient).

Maybe it would be wise for PIO to directly use arduinoesp32 core as is, and just focus on allowing users to run arduinoesp32 core along with IDF to keep access to menuconfig.

@mhaberler
Copy link

I am concerned about the viability of this repo long-term as folks like @Jason2866 have started to roll their own and this repo not catching up

I am currently using his https://github.com/tasmota/platform-espressif32/tree/Arduino/IDF5 branch and it works mostly great for me - even if I feel uneasy about relying on a private effort with unclear future - how do you see the longer-term perspective of your fork?

This fork has diverged from this repo massively - not sure this can brought back here easily

the other issue I have is the abysmal debugging experience esp32/pio/arduino compared to using straight IDF

@0x0fe
Copy link

0x0fe commented Nov 23, 2023

@mhaberler i did the same yesterday and used https://github.com/tasmota/platform-espressif32/tree/Arduino/IDF5

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#3.0.0-alpha2
framework = 
	arduino
	espidf
board = esp32dev

But then when building it complained of missing SPIFFS, i added it manually from previous version (in components directory) but then i get a dependancy error with esp_partition.h, despite it is well present in "components".

   17 | #include "esp_partition.h"
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
Compiling .pio\build\esp32dev\tcp_transport\transport.o
Compiling .pio\build\esp32dev\tcp_transport\transport_ssl.o
Compiling .pio\build\esp32dev\tcp_transport\transport_internal.o
*** [.pio\build\esp32dev\spiffs\esp_spiffs.o] Error 1
*** [.pio\build\esp32dev\spiffs\spiffs_api.o] Error 1

Did you met this issue too?

@Jason2866
Copy link
Contributor

Jason2866 commented Nov 23, 2023

@0x0fe @mhaberler My fork has not the goal to be compatible to anything official.
The fork is made for project Tasmota. Everything is optimized for that use. The sdkconfig settings differs a lot from the official espressif ones. There are components added and others are removed (for example SPIFFS).
Breaking changes can and will happen.
Imho no reason to be concerned about support of Arduino Espressif. Platformio always do a release when a RELEASED core is available. Core 3.0.0 is in ALPHA state!

Since project Tasmota is always in first row for new stuff, and is active in contributing to espressif Arduino Core, there was the need to have the development version of Arduino Core available with Platformio.
Project Tasmota is only possible with Platformio. Thx again to the Team of Platformio (@ivankravets ) for this awesome IDE!!

The needed changes for Arduino 3.0 in Platformio are not many. A small job for @valeros

@0x0fe
Copy link

0x0fe commented Nov 23, 2023

@Jason2866 oh, i see, that explains the error with SPIFFS

@mhaberler
Copy link

alpha stage - fair enough.

Now assume Core 3.0.0 gets released - does this have any bearing on the debugging situation?

who has the ticket to make that work?

I'm talking about openocd + gdb versions and matching debug configs which work - at least somewhat like Nordic Connect and Segger

I admit I am slightly confused about players and the who-does-what-and-why-not - I might be barking up the wrong tree (or repo) for that matter

@Jason2866
Copy link
Contributor

@mhaberler Does debugging work in espressif IDF env setup? If yes imho Platformio has the job for the devices which are currently supported with IDF 5.1. Not saying it will be easy to solve or it will be done soon. The debug issue for the C6 is known and there is an open issue.

@mhaberler
Copy link

Did you met this issue too?

@0x0fe yes, same here `Failed to resolve component 'spiffs'

@mhaberler
Copy link

@mhaberler Does debugging work in espressif IDF env setup? If yes imho Platformio has the job for the devices which are currently supported with IDF 5.1. Not saying it will be easy to solve or it will be done soon. The debug issue for the C6 is known and there is an open issue.

you mean like command line/shell? yes, repeatably

@0x0fe
Copy link

0x0fe commented Nov 23, 2023

@mhaberler It makes sense, well SPIFFS yo ucan add it manually from previous version but then you will get an error with esp_partition, and i am not sure where to re-enable the SPIFFS module and dependencies correctly.

@ivankravets
Copy link
Member Author

I'm a bit confused. Why is there a separate platform-espressif32 core? It seems to be based on arduino-espressif32, but it always seems to lag behind...

https://github.com/platformio/platform-espressif32 isn't a core itself. It's a software piece that makes the PlatformIO Build API work seamlessly with Espressif 32 products within the PlatformIO ecosystem. If you check the source code of this repository, you'll find other integrations, like ESP-IDF. If we support Zephyr/ESP32, it'll also be found here.

Would it be smarter for PIO to directly use the arduinoesp32 core as it is?

We do use it directly (https://registry.platformio.org/tools/platformio/framework-arduinoespressif32). There's no common ground between Arduino IDE and PlatformIO. The "arduino" in PlatformIO is just a framework. PlatformIO supports over 10+ different frameworks. The reason many developers see PlatformIO as a "replacement for Arduino IDE" is due to source code compatibility and advanced project management with lots of configuration and collaboration features based on PlatformIO's unique declarative principles.

If you've tried another framework besides "arduino", you'd notice there's no difference in the configuration aspect. You and your team can focus on project development without wrestling with bootstrapping and collaboration issues.

Lastly, for a VSCode-based solution with native ESP32/Arduino core support, I recommend the Arduino IDE for VSCode extension backed by Microsoft. It's similar to Arduino IDE 2.0 but uses VSCode as the editor. Both leverage "arduino-cli", so the build system and workflows are the same.

@0x0fe
Copy link

0x0fe commented Nov 23, 2023

@ivankravets I see, glue logic layer. the totally different versioning between these platforms and the cores they implement is rather annoying, though.

@mhaberler
Copy link

@ivankravets make me understand what your recommendation means - are you saying:

When I prefer to use VSCode, I should take the Microsoft Arduino extension for VSCode and drop Platformio?

@ivankravets
Copy link
Member Author

@mhaberler, we value openness in our community and aim to give you the freedom to choose. If there's a newer version of the software not yet supported by PlatformIO, we won't hide it.

The current supported version is Arduino Core v2.x for ESP32. Our collaboration with Espressif, including discussions about renewal, is ongoing. It's worth noting that we have @VojtechBartoska, a project manager from Espressif, in this thread. We're all working together to ensure you receive the best features and support. We'll keep everyone posted on any updates to ensure a smooth continuation of our services.

@hoeken
Copy link

hoeken commented Nov 25, 2023

I can see there's a bit of political project stuff happening, but as an end-user, all I want to do is be able to use the latest Arduino framework. Consider this a vote of support for making that happen, however it needs to be.

@ivankravets
Copy link
Member Author

PlatformIO is a commercial open-source project. In the past, it used to be a paid service before 2020, following a business-to-consumer (B2C) model. Unexpectedly, PlatformIO gained widespread popularity among millions of developers globally. Consequently, we shifted our strategy to make powerful tools for professional embedded development freely accessible to everyone.

The active development and maintenance of PlatformIO, along with its infrastructure, are now supported by technology partners dedicated to delivering an excellent developer experience. Espressif was one such partner, and we appreciate their long-standing collaboration.

Currently, Espressif has ceased support for new products in PlatformIO, but rest assured, we are committed to providing support for existing Espressif products integrated before this change, as per our technology licensing policy. Your projects won't face disruptions, and services will continue as usual.

If there's significant interest from Espressif customers in PlatformIO, we'll reach out to Espressif and explore the possibility of continuing our collaboration.

If you have any questions or need clarification, feel free to ask. We're here to provide all the answers you need.

@mhaberler
Copy link

ok, so Espressif jumped ship

@ivankravets what does that mean for the subject topic - Arduino3?

your wording would suggest to me "no new Espressif releases - be it Arduino or ESP-IDF, existing releases go on life support"

are you saying this repo stops at ESP-IDF 5.1.1/Arduino2 until Espressif changes its mind ?

@Jason2866
Copy link
Contributor

Jason2866 commented Nov 28, 2023

@mhaberler Looks like: as long there is not enough feedback from Platformio users wanting actual espressif framework support, there is no good position from Platformio side to argument why espressif should invest money having Platformio support.
espressif is thinking supporting the ArduinoIDE toy is enough :-(

rtrbt added a commit to Tinkerforge/esp32-firmware that referenced this issue Feb 25, 2025
rtrbt added a commit to Tinkerforge/esp32-firmware that referenced this issue Feb 26, 2025
rtrbt added a commit to Tinkerforge/esp32-firmware that referenced this issue Mar 4, 2025
@valeros valeros marked this as a duplicate of #1587 May 17, 2025
@gitslav
Copy link

gitslav commented Jun 1, 2025

Hi @ivankravets and thank you for the great work!
I would really like for Platformio to come on top of this situation but after 2 years is there any update on:

"Once we gather enough interest from the Espressif community, we plan to reach out to the Espressif company and request a reconsideration of support for PlatformIO."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests