Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Arduino ESP32 v3.0 based on ESP-IDF v5.1 #1225

Open
ivankravets opened this issue Nov 4, 2023 · 317 comments
Open

Support Arduino ESP32 v3.0 based on ESP-IDF v5.1 #1225

ivankravets opened this issue Nov 4, 2023 · 317 comments
Labels

Comments

@ivankravets
Copy link
Member

ivankravets commented Nov 4, 2023

The ESP32 Core for Arduino 2.x is the most recent major version currently recommended for use with PlatformIO. The decision to discontinue support was made by the Espressif company, as indicated in their official statement: espressif/arduino-esp32#8606

If you're an Espressif user who relies on PlatformIO for your projects, kindly share your thoughts in the comments below and cast your vote on this matter. Once we gather enough interest from the Espressif community, we plan to reach out to the Espressif company and request a reconsideration of support for PlatformIO.

@thijstriemstra
Copy link

thijstriemstra commented Nov 8, 2023

I've been trying this with following config:

; see https://docs.espressif.com/projects/arduino-esp32/en/latest/installing.html#how-to-update-to-the-latest-code
[env:arduino-latest]
platform = https://github.com/platformio/platform-espressif32.git
board = az-delivery-devkit-v4
framework = arduino
platform_packages =
    framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32#master

but the build fails with KeyError: 'framework-arduinoespressif32-libs':

Processing arduino-latest (platform: https://github.com/platformio/platform-espressif32.git; board: az-delivery-devkit-v4; framework: arduino)
--------------------------------------------------------------------------------
Platform Manager: Installing git+https://github.com/platformio/platform-espressif32.git
git version 2.42.0
Cloning into '/home/runner/.platformio/.cache/tmp/pkg-installing-uz1yxjz0'...
Platform Manager: espressif32@6.4.0+sha.f6ec392 has been installed!
Tool Manager: Installing git+https://github.com/espressif/arduino-esp32#master
git version 2.42.0
Cloning into '/home/runner/.platformio/.cache/tmp/pkg-installing-kih232ry'...
Tool Manager: framework-arduinoespressif32@3.0.0+sha.bf822ef has been installed!
Library Manager: Installing fortyseveneffects/MIDI Library @ ^5.0.2
Unpacking 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Library Manager: MIDI Library@5.0.2 has been installed!
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/az-delivery-devkit-v4.html
PLATFORM: Espressif 32 (6.4.0+sha.f6ec392) > AZ-Delivery ESP-32 Dev Kit C V4
HARDWARE: ESP32 240MHz, 520KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 3.0.0+sha.bf822ef 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Converting edrumulus.ino
/tmp/tmpj5gstour/src/edrumulus.ino:45: warning: "MYMIDI" redefined
 #  define MYMIDI                     usbMIDI
 
/tmp/tmpj5gstour/src/edrumulus.ino:39: note: this is the location of the previous definition
 #  define MYMIDI                     MIDI
 
/tmp/tmpj5gstour/src/edrumulus.ino:46: warning: "MIDI_CONTROL_CHANGE_TYPE" redefined
 #  define MIDI_CONTROL_CHANGE_TYPE   usbMIDI.ControlChange
 
/tmp/tmpj5gstour/src/edrumulus.ino:40: note: this is the location of the previous definition
 #  define MIDI_CONTROL_CHANGE_TYPE   midi::ControlChange
 
/tmp/tmpj5gstour/src/edrumulus.ino:47: warning: "MIDI_SEND_AFTER_TOUCH" redefined
 #  define MIDI_SEND_AFTER_TOUCH      sendAfterTouchPoly
 
/tmp/tmpj5gstour/src/edrumulus.ino:41: note: this is the location of the previous definition
 #  define MIDI_SEND_AFTER_TOUCH      sendAfterTouch
 
KeyError: 'framework-arduinoespressif32-libs':
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/runner/.platformio/platforms/espressif32@src-5f117260f75b328038ec9d3fd0e14a68/builder/main.py", line 346:
    target_elf = env.BuildProgram()
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 61:
    env.ProcessProgramDeps()
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 121:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 342:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 662:
    return method(*args, **kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/runner/.platformio/platforms/espressif32@src-5f117260f75b328038ec9d3fd0e14a68/builder/frameworks/arduino.py", line 41:
    SConscript(
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 662:
    return method(*args, **kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/runner/.platformio/packages/framework-arduinoespressif32/tools/platformio-build.py", line 40:
    FRAMEWORK_LIBS_DIR = platform.get_package_dir("framework-arduinoespressif32-libs")
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/platform/_packages.py", line 32:
    pkg = self.get_package(name)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/platform/_packages.py", line 29:
    return self.pm.get_package(spec or self.get_package_spec(name))
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/platform/_packages.py", line 21:
    owner=self.packages[name].get("owner"),
========================== [FAILED] Took 4.31 seconds ==========================

@ivankravets
Copy link
Member Author

See the updated comment #1225 (comment)

@Jason2866
Copy link
Contributor

Jason2866 commented Nov 10, 2023

Adding Arduino 3.0.0 support is the only way to make the ESP32-C6 available for Arduino projects.
The needed changes are not that many, i have added Arduino 3.0.x alpha support in my Fork and bulding Arduino and IDF projects with the C6 are working.
We have ported Arduino project Tasmota to the C6 and it runs already surprisingly stable.

@Jason2866
Copy link
Contributor

Jason2866 commented Nov 10, 2023

If you're an Espressif user who relies on PlatformIO for your projects, kindly share your thoughts in the comments below and cast your vote on this matter. Once we gather enough interest from the Espressif community, we plan to reach out to the Espressif company and request a reconsideration of support for PlatformIO.

@ivankravets Does this mean that Espressif has dropped any support? Looking in current Arduino 3.0.0 code (branch master) everything regarding Platformio support is there.

@VojtechBartoska
Copy link

Hello all,

I do work in Arduino Core ESP32 project as a Project Manager so I would like to clarify this from our Team point of view.

Please take a look on this comment in original Pull request.

We also update a warning note in our documentation which should be a bit confusing.

To not duplicate what is written in original PR, we will be glad if PlatformIO supports our latest version and we are open to any contributions.

@ivankravets Feel free to contact us any time directly, we will be happy to discuss this if you see any points we can do differently.

Thanks

@0x0fe
Copy link

0x0fe commented Nov 22, 2023

Adding Arduino 3.0.0 support is the only way to make the ESP32-C6 available for Arduino projects.
The needed changes are not that many, i have added Arduino 3.0.x alpha support in my Fork and bulding Arduino and IDF projects with the C6 are working.
We have ported Arduino project Tasmota to the C6 and it runs already surprisingly stable.

@Jason2866
Do you know how i can setup the .ini file in a new project for using your 3.0.0 fork and 5.1 IDF at the same time (to keep the SDK menuConfig)?

@0x0fe
Copy link

0x0fe commented Nov 22, 2023

@ivankravets
there is something i dont understand, why is there a separate platformespressif32 core? It is based on arduinoespressif32, but always lags behind, and arduinoesp32 already lags behind IDF. So what is the point of this separate core? Why is PIO not using arduinoesp32 core as is directly?

I think a reason why many users use PIO is also to keep some of the arduinoesp32 APIs while having the possibility to modify the IDF SDK configuration (menuconfig) which is necessary in many cases and not possible in arduino, unless using the libbuilder (buggy and not convenient).

Maybe it would be wise for PIO to directly use arduinoesp32 core as is, and just focus on allowing users to run arduinoesp32 core along with IDF to keep access to menuconfig.

@mhaberler
Copy link

I am concerned about the viability of this repo long-term as folks like @Jason2866 have started to roll their own and this repo not catching up

I am currently using his https://github.com/tasmota/platform-espressif32/tree/Arduino/IDF5 branch and it works mostly great for me - even if I feel uneasy about relying on a private effort with unclear future - how do you see the longer-term perspective of your fork?

This fork has diverged from this repo massively - not sure this can brought back here easily

the other issue I have is the abysmal debugging experience esp32/pio/arduino compared to using straight IDF

@0x0fe
Copy link

0x0fe commented Nov 23, 2023

@mhaberler i did the same yesterday and used https://github.com/tasmota/platform-espressif32/tree/Arduino/IDF5

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#3.0.0-alpha2
framework = 
	arduino
	espidf
board = esp32dev

But then when building it complained of missing SPIFFS, i added it manually from previous version (in components directory) but then i get a dependancy error with esp_partition.h, despite it is well present in "components".

   17 | #include "esp_partition.h"
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
Compiling .pio\build\esp32dev\tcp_transport\transport.o
Compiling .pio\build\esp32dev\tcp_transport\transport_ssl.o
Compiling .pio\build\esp32dev\tcp_transport\transport_internal.o
*** [.pio\build\esp32dev\spiffs\esp_spiffs.o] Error 1
*** [.pio\build\esp32dev\spiffs\spiffs_api.o] Error 1

Did you met this issue too?

@Jason2866
Copy link
Contributor

Jason2866 commented Nov 23, 2023

@0x0fe @mhaberler My fork has not the goal to be compatible to anything official.
The fork is made for project Tasmota. Everything is optimized for that use. The sdkconfig settings differs a lot from the official espressif ones. There are components added and others are removed (for example SPIFFS).
Breaking changes can and will happen.
Imho no reason to be concerned about support of Arduino Espressif. Platformio always do a release when a RELEASED core is available. Core 3.0.0 is in ALPHA state!

Since project Tasmota is always in first row for new stuff, and is active in contributing to espressif Arduino Core, there was the need to have the development version of Arduino Core available with Platformio.
Project Tasmota is only possible with Platformio. Thx again to the Team of Platformio (@ivankravets ) for this awesome IDE!!

The needed changes for Arduino 3.0 in Platformio are not many. A small job for @valeros

@0x0fe
Copy link

0x0fe commented Nov 23, 2023

@Jason2866 oh, i see, that explains the error with SPIFFS

@mhaberler
Copy link

alpha stage - fair enough.

Now assume Core 3.0.0 gets released - does this have any bearing on the debugging situation?

who has the ticket to make that work?

I'm talking about openocd + gdb versions and matching debug configs which work - at least somewhat like Nordic Connect and Segger

I admit I am slightly confused about players and the who-does-what-and-why-not - I might be barking up the wrong tree (or repo) for that matter

@Jason2866
Copy link
Contributor

@mhaberler Does debugging work in espressif IDF env setup? If yes imho Platformio has the job for the devices which are currently supported with IDF 5.1. Not saying it will be easy to solve or it will be done soon. The debug issue for the C6 is known and there is an open issue.

@mhaberler
Copy link

Did you met this issue too?

@0x0fe yes, same here `Failed to resolve component 'spiffs'

@mhaberler
Copy link

@mhaberler Does debugging work in espressif IDF env setup? If yes imho Platformio has the job for the devices which are currently supported with IDF 5.1. Not saying it will be easy to solve or it will be done soon. The debug issue for the C6 is known and there is an open issue.

you mean like command line/shell? yes, repeatably

@0x0fe
Copy link

0x0fe commented Nov 23, 2023

@mhaberler It makes sense, well SPIFFS yo ucan add it manually from previous version but then you will get an error with esp_partition, and i am not sure where to re-enable the SPIFFS module and dependencies correctly.

@ivankravets
Copy link
Member Author

I'm a bit confused. Why is there a separate platform-espressif32 core? It seems to be based on arduino-espressif32, but it always seems to lag behind...

https://github.com/platformio/platform-espressif32 isn't a core itself. It's a software piece that makes the PlatformIO Build API work seamlessly with Espressif 32 products within the PlatformIO ecosystem. If you check the source code of this repository, you'll find other integrations, like ESP-IDF. If we support Zephyr/ESP32, it'll also be found here.

Would it be smarter for PIO to directly use the arduinoesp32 core as it is?

We do use it directly (https://registry.platformio.org/tools/platformio/framework-arduinoespressif32). There's no common ground between Arduino IDE and PlatformIO. The "arduino" in PlatformIO is just a framework. PlatformIO supports over 10+ different frameworks. The reason many developers see PlatformIO as a "replacement for Arduino IDE" is due to source code compatibility and advanced project management with lots of configuration and collaboration features based on PlatformIO's unique declarative principles.

If you've tried another framework besides "arduino", you'd notice there's no difference in the configuration aspect. You and your team can focus on project development without wrestling with bootstrapping and collaboration issues.

Lastly, for a VSCode-based solution with native ESP32/Arduino core support, I recommend the Arduino IDE for VSCode extension backed by Microsoft. It's similar to Arduino IDE 2.0 but uses VSCode as the editor. Both leverage "arduino-cli", so the build system and workflows are the same.

@0x0fe
Copy link

0x0fe commented Nov 23, 2023

@ivankravets I see, glue logic layer. the totally different versioning between these platforms and the cores they implement is rather annoying, though.

@mhaberler
Copy link

@ivankravets make me understand what your recommendation means - are you saying:

When I prefer to use VSCode, I should take the Microsoft Arduino extension for VSCode and drop Platformio?

@ivankravets
Copy link
Member Author

@mhaberler, we value openness in our community and aim to give you the freedom to choose. If there's a newer version of the software not yet supported by PlatformIO, we won't hide it.

The current supported version is Arduino Core v2.x for ESP32. Our collaboration with Espressif, including discussions about renewal, is ongoing. It's worth noting that we have @VojtechBartoska, a project manager from Espressif, in this thread. We're all working together to ensure you receive the best features and support. We'll keep everyone posted on any updates to ensure a smooth continuation of our services.

@hoeken
Copy link

hoeken commented Nov 25, 2023

I can see there's a bit of political project stuff happening, but as an end-user, all I want to do is be able to use the latest Arduino framework. Consider this a vote of support for making that happen, however it needs to be.

@ivankravets
Copy link
Member Author

PlatformIO is a commercial open-source project. In the past, it used to be a paid service before 2020, following a business-to-consumer (B2C) model. Unexpectedly, PlatformIO gained widespread popularity among millions of developers globally. Consequently, we shifted our strategy to make powerful tools for professional embedded development freely accessible to everyone.

The active development and maintenance of PlatformIO, along with its infrastructure, are now supported by technology partners dedicated to delivering an excellent developer experience. Espressif was one such partner, and we appreciate their long-standing collaboration.

Currently, Espressif has ceased support for new products in PlatformIO, but rest assured, we are committed to providing support for existing Espressif products integrated before this change, as per our technology licensing policy. Your projects won't face disruptions, and services will continue as usual.

If there's significant interest from Espressif customers in PlatformIO, we'll reach out to Espressif and explore the possibility of continuing our collaboration.

If you have any questions or need clarification, feel free to ask. We're here to provide all the answers you need.

@mhaberler
Copy link

ok, so Espressif jumped ship

@ivankravets what does that mean for the subject topic - Arduino3?

your wording would suggest to me "no new Espressif releases - be it Arduino or ESP-IDF, existing releases go on life support"

are you saying this repo stops at ESP-IDF 5.1.1/Arduino2 until Espressif changes its mind ?

@Jason2866
Copy link
Contributor

Jason2866 commented Nov 28, 2023

@mhaberler Looks like: as long there is not enough feedback from Platformio users wanting actual espressif framework support, there is no good position from Platformio side to argument why espressif should invest money having Platformio support.
espressif is thinking supporting the ArduinoIDE toy is enough :-(

@TD-er
Copy link

TD-er commented Dec 22, 2024

@jeremiahrose
Just given how much effort has been put into making Arduino 3.x/ESP-IDF5.x work as PioArduino I doubt it can 'just' be merged back into PlatformIO.
Also after such a merge, you need to support it or else it will be the same situation as it is now.

As far as I understood is that there is a business disagreement between Espressif and PlatformIO.
Nobody is stopping you from handing PlatformIO money, but I doubt it will be enough for the PlatformIO team to re-establish relations with Espressif given Espressif also just removed all references to PlatformIO from their repo.
Also PlatformIO did make it quite clear there won't be any new platform build made by PlatformIO supporting the latest Espressif SoC's. (which needs Espressif/Arduino 3.x and ESP-IDF5.x)

That's the sole reason why PioArduino was started.

@jeremiahrose
Copy link

Thanks @TD-er for the context.

TL;DR

PlatformIO and Espressif are no longer friends, and PlatformIO is refusing to merge community PRs for new Espressif devices. Community development is now happening at https://github.com/pioarduino/platform-espressif32, and PRs can be submitted there to add new boards :)

@mathieucarbou
Copy link

mathieucarbou commented Dec 22, 2024

@jeremiahrose also, the way it works this is not only one repository but a complete set of repos which work together. Merging back will just transform platformio to pioarduino, so useless. They work differently and source their artifact at different locations.
Pioarduino does not depend on platformio tooling.
And pioarduino has its own vscode extension.

There are also several things that are added to pioarduino and works well like hybrid compilation which allows to add some esp-idf compile flag and exclude esp-idf modules through the platformio.ini file.

@ivankravets
Copy link
Member Author

As far as I understood is that there is a business disagreement between Espressif and PlatformIO

While this thread focuses on technical aspects, we are in discussions with Espressif about the business value of the Arduino framework for their goals. Recently, Espressif shared that they don't see significant business value in the ESP32/Arduino solution. As a result, they have decided that Arduino IDE will remain the only official tool for starting with their products. The reason behind this approach is straightforward: they aim to guide users towards ESP-IDF. For silicon vendors, business priorities like revenue often outweigh the developer experience.

Nobody is stopping you from handing PlatformIO money

If the PlatformIO/Arduino community isn't a priority for Espressif, why should it be for PlatformIO Labs? Supporting Espressif's customers requires significant effort and investment from us. You might argue: "This could cost you some developers". However, our success isn't measured by the number of users; this is not a key performance indicator (KPI) for us.

Our mission is to change how modern embedded systems are developed. Since 2014, we have introduced new ideas like declarative principles and test-driven development to the semiconductor industry. Initially, many silicon vendors ignored or dismissed us, but today these ideas have become industry standards. For example, Arm recently introduced support for multi-toolchains and dependencies in their CMSIS solution for VS Code - something we pioneered years ago.

Espressif also just removed all references to PlatformIO from their repo

Yes, we received an email from Espressif last week. They have decided to remove all mentions of PlatformIO from their website and documentation. Their stated reason is that "Arduino IDE is the only official tool for working with the ESP32/Arduino framework".

Also PlatformIO did make it quite clear there won't be any new platform build made by PlatformIO supporting the latest Espressif SoC's. (which needs Espressif/Arduino 3.x and ESP-IDF5.x)

This development platform is still active. We continue to provide support for the ESP-IDF framework, which is widely used by businesses. In fact, the number of businesses depending on PlatformIO/ESP-IDF is growing every month.

PlatformIO is refusing to merge community PRs

We are closely monitoring the situation with multiple forks of this development platform. Unfortunately, the results so far have been negative. Contributors to these forks are aware of the challenges and are ready to deal with them, but many end users are confused. They don't realize these forks are unofficial and unrelated to Espressif or PlatformIO.

As a result, we have been receiving complaints about instability and bug reports in our repositories, even though these issues originate from the forked platforms. For example, see platformio/platformio-core#5055

We kindly ask the authors of these forks to add a clear NOTICE/WARNING to their dev-platforms builder/main.py, stating that they are unofficial and that all issues should be reported directly to them. This will save us time and protect our reputation.


Wishing you a wonderful holiday season! 🎄🎉

@b314b
Copy link

b314b commented Dec 27, 2024

Lol -- Both Arduino IDE and ESP-IDF is so abysmally horrible compared to PlatformIO + VS code + arduino, you'd think Espressif would want to keep the PlatformIO/Arduino thing going.

As I'm working on products and have the future in mind, I must ask:

Is platformio/platform-espressif32 being no longer developed from here on?

or

Is platformio/platform-espressif32 under continuing development and expected to continue into the future; but espressif/arduino-esp32 is not being continued by PlatformIO (so we would have to rely on other groups' versions of it)?

@ivankravets
Copy link
Member Author

@b314b do you work on commercial products?

@italocjs
Copy link

Its very sad to see that this will be the outcome of an previously awesome toolchain. Unfortunately i can't keep using outdated or arduino community stuff as they are not being supported by espidf. the only way forward seems to be using espidf directly, none of the attempts to run the pioarduino or other suggested methods have worked for me, every time i add something (like an component) i get an hell of undefined or broken stuff coming from apparently nowhere.

@Vigeant
Copy link

Vigeant commented Dec 31, 2024 via email

@mars000
Copy link

mars000 commented Dec 31, 2024 via email

@0x0fe
Copy link

0x0fe commented Dec 31, 2024

@b314b but...IDF uses VSCODE as IDE.

@ivankravets
Copy link
Member Author

@0x0fe , do you mean this extension? Reading the latest reviews gives us a better understanding of why there is so much interest in the PlatformIO/ESP-IDF integration. Next year, we will focus more on this integration and work on adding new features.

In the meantime, if anyone wants to get started with PlatformIO/ESP-IDF, we recommend checking out our PlatformIO/ESP-IDF Configuration Guide.

Happy New Year! 🎄

@hitecSmartHome
Copy link

Hehe. These reviews are hilarious. Idf extension is still a mess. With platformio or now with pioarduino I just install the extension and it works.

@FlavioZanoni
Copy link

FlavioZanoni commented Jan 8, 2025

So lets get this straight.

  • Espressif doesn't show PlatformIO as a supported anymore.
  • Espressif supports only the official Arduino IDE.
  • PlatformIO sees that Espressif does not see the PlatformIO community as a priority.
  • PlatformIO stops updating the platform-espressif32 (layer that lets you code the esp32 with Arduino) code, with - the "framework = arduino" directory, saying "If the PlatformIO/Arduino community isn't a priority for Espressif, why should it be for PlatformIO Labs?".
  • PlatformIO does not announce the dropped support anywhere (at least i didn't find it) they just comment it on this thread.
  • pioarduino emerges and does the updates that PlatformIO isn't doing anymore (updates for the Arduino framework) , basically a drop-in-replacement for the package... to use it you just replace the "platform" on patformio.ini for their link.

So this package is still being maintained but not for the Arduino framework.

Correct?

@kylefmohr
Copy link

@FlavioZanoni I'd say that's a pretty solid summary of the situation, at least from my perspective 👍

@mathieucarbou
Copy link

pioarduino emerges and does the updates that PlatformIO isn't doing anymore (updates for the Arduino framework)

  • pioarduino is more than a replacement. There are more undocumented features. Hybrid compilation is one: where you can select esp-idf modules to remove and control sdkconfig flags from within platformio.ini file. So this nearly like using arduino as a esp-idf component, for those who need more tweaking.

  • pioarduino has its own vscode extension (forked from the very good PlatformIO one) in order to leverage all the pioarduino tools and repos.

  • Espressif devs of Arduino Core have also mentioned that they will support the pioarduino community initiative, and they do so already (like by adding pioarduino-related scripts in arduino builder).

@sprior
Copy link

sprior commented Jan 9, 2025

@FlavioZanoni You missed one point: PlatformIO uses a business model where they charge the platform providers to be included and the PlatformIO users get it for free. So I'd assume that the PlatformIO wanted to have a little money chat with the Espressif folks who weren't interested in forking over money when they'd really prefer developers use the ESP-IDF they provide.

@TD-er
Copy link

TD-er commented Jan 9, 2025

@FlavioZanoni You missed one point: PlatformIO uses a business model where they charge the platform providers to be included and the PlatformIO users get it for free. So I'd assume that the PlatformIO wanted to have a little money chat with the Espressif folks who weren't interested in forking over money when they'd really prefer developers use the ESP-IDF they provide.

PlatformIO always has had costs to operate its business, so I really can't imagine they were not being paid before by Espressif et.al. Having devs on the payrole and buying bandwidth does cost money.
So the only conclusion we can probably draw right now is that there is likely some business disagreement here.
We can only guess for the exact grounds of such a disagreement.
However I don't think we as users should put in much energy in guessing. If any of the involved parties would have wanted to officially disclose the real reason, they would have done so in the past 12 - 18 months (or has it been longer?)
Guessing would only lead to wrong assumptions and divide the OpenSource community.

@genemars
Copy link

genemars commented Jan 9, 2025

Because of this situation, I use Arduino more as a utility library rather than a framework, and for many things I already had to use ESP-IDF directly.
So, I guess, the only options are to migrate the code with Arduino includes to ESP-IDF where applicable, or deal with the fact that only Arduino 2.x is supported.
In the short term this might be ok, but I am already considering migrating to ESP-IDF-only because the lack of support for newer SoCs such as ESP32-C6 which I would have liked to integrate in my project because the additional network stacks it supports.
Unless support for new boards can be backported to arduino 2.8.x, I see no other way to grant a future to a PlatformIO project than removing Arduino code. If there's a way, please advice.

@mathieucarbou
Copy link

mathieucarbou commented Jan 9, 2025

Because of this situation, I use Arduino more as a utility library rather than a framework, and for many things I already had to use ESP-IDF directly.
So, I guess, the only options are to migrate the code with Arduino includes to ESP-IDF where applicable, or deal with the fact that only Arduino 2.x is supported.
In the short term this might be ok, but I am already considering migrating to ESP-IDF-only because the lack of support for newer SoCs such as ESP32-C6 which I would have liked to integrate in my project because the additional network stacks it supports.
Unless support for new boards can be backported to arduino 2.8.x, I see no other way to grant a future to a PlatformIO project than removing Arduino code. If there's a way, please advice.

@genemars : PlatformIO is tooling, you usually do not chose a framework or development base depending on a tooling, which is more about user preference. I.e. you could go CLI, with 3 different vscode extensions (pioarduino, plaformio, esp-idf), Arduino IDE, etc Each having pros and cons. PlatformIO as it is right now is a valid option for esp8266, esp32 arduino 2, esp-idf, and other platforms. They just stopped supporting Arduino platform but their tooling also supports other languages and platforms.

Tooling should not be the main driver decision for your project ;-)

@TD-er
Copy link

TD-er commented Jan 9, 2025

If there's a way, please advice.

As mentioned by several people before, you can try pioarduino.
Just keep in mind, quite a lot has changed in ESP-IDF5.x/Arduino3.x, so you may need to change some code to make it work again with these newer SDK versions. Also quite a lot of libraries may not yet have been updated to work with these newer SDK versions.
So it will be more work than simply replacing a platform link in your .ini files.

@sblantipodi
Copy link

From my side I can tell only one thing.
ESP devices are less interesting than ever before with a lot of fragmentation due to new launches exacerbated by the Core 2/3 support and breaking changes and the breakup with PlatformIO.

It remains to be proven whether pioarduino will be the free answer to PlatformIO on esp32. I have a lot of confidence in the pioarduino team, but it is clear that, being a project without a solid business model, it is more susceptible to the wind than PlatformIO has been.

I sincerely can't wait to find another reason to abandon Espressif.

@genemars
Copy link

genemars commented Jan 9, 2025

If there's a way, please advice.

As mentioned by several people before, you can try pioarduino. Just keep in mind, quite a lot has changed in ESP-IDF5.x/Arduino3.x, so you may need to change some code to make it work again with these newer SDK versions. Also quite a lot of libraries may not yet have been updated to work with these newer SDK versions. So it will be more work than simply replacing a platform link in your .ini files.

@TD-er, I already did and my project also works with pioarduino, but I'd prefer to stay on an official solution and I don't know if the pioarduino project is going to be active in the long term.

@mathieucarbou, I know it's just tooling but this is the first thing you choose when you start a project. And after lot of research (back in 2019) I choosed platformio because of the existsing codebase and examples I could reuse from arduino framework.

@TD-er
Copy link

TD-er commented Jan 9, 2025

ESP devices are less interesting than ever before with a lot of fragmentation due to new launches exacerbated by the Core 2/3 support and breaking changes and the breakup with PlatformIO.

The part of breaking changes I do understand as that's something that is unavoidable if you want to move forward.
And IMHO there are too many ESP32-variants: ESP32-classic, S2/S3, C2/C3/C6 (upcoming C5/C61), H2, P4
However now with ESP-IDF5.x/Arduino3.x a lot of the architectural changes were absolutely needed to keep it managable.
Also the peripheral support now actually makes sense from a software development perspective.

I'm not sure if for pioarduino a financial businessmodel is required.
At least 2 larger projects (ESPEasy/Tasmota) depend on it and I cannot imagine the others will not follow as they also would want to support the newer ESP32-variants.
This should make it more interesting for those teams to also help maintain it.
Among the teams of the various firmwares there is actually quite a lot of coorporation.
For example, I have received bugfixes from devs of Espruna and Tasmota and vice verse.
Not 100% sure if I personally have ever sent in a bugfix for ESPHome, but I know they have helped with some build issue we all had at some point due to broken tool chains.
And we "meet" eachother often in discussing issues or PRs on the Espressif repos or via other channels (email, discord, or even around the coffee table :) )

If it were only a single person maintaining an important tool you depend on, then I would also have my doubts.

@genemars
Copy link

genemars commented Jan 9, 2025

I sincerely can't wait to find another reason to abandon Espressif.

@sblantipodi, Espressif SoCs are the most versatile, cheap and powerful solution out there. All ESP products have been amazing since the very beginning. I don't see a reason to back out.

@softhack007
Copy link

softhack007 commented Jan 9, 2025

Actually I find it somewhat confusing that espressif seems to be quite proud of the high number of esp-based projects on github,
while still denying any business value from supporting platformIO 🤔

image

I guess that more than half of these "115K+" have a platformio.ini or package.json/library.json/library.properties files in their repo.

@mathieucarbou
Copy link

while still denying any business value from supporting platformIO 🤔

PlatformIO is a tooling, not a platform.
PlatformIO is used to help develop with many platforms / frameworks.

To use PlatformIO tooling, someone needs to develop the glue (integration) between the framework (arduino core, esp-idf, etc) and the tooling (vscode extension, CLI, etc).

So what has been stopped since Arduino Core 3 is that PlatformIO does not take care of this "glue" anymore. But pioarduino does, with the help of Espressif Arduino Core devs.

Making a parallel with other languages, Nebeans and JetBrains are not paid by Oracle or Azure to add support for Java in their IDE. Pyhon does not pay vscode or others to have support for their languages in vscode or other IDE. Maven and Gradle tooling for Java projects are not paid by the different things they integrate to support them. So this is normal that Espressif does not pay any third party to add support for their platform.

What Espressif can do though, but not obligatad to, is help building (or, even more build) the glue between existing tooling and their stuff, existing tooling being PlatformIO, IDEs, etc.

But most of this stuff, in other languages ecosystem, comes from the community, like pioarduino. The more people get involved with and in pioarduino, the more it has chances to survive and be resilient.

@Jason2866
Copy link
Contributor

Jason2866 commented Jan 9, 2025

I'm not sure if for pioarduino a financial businessmodel is required. At least 2 larger projects (ESPEasy/Tasmota) depend on it and I cannot imagine the others will not follow as they also would want to support the newer ESP32-variants. This should make it more interesting for those teams to also help maintain it. Among the teams of the various firmwares there is actually quite a lot of coorporation. For example, I have received bugfixes from devs of Espruna and Tasmota and vice verse. Not 100% sure if I personally have ever sent in a bugfix for ESPHome, but I know they have helped with some build issue we all had at some point due to broken tool chains. And we "meet" eachother often in discussing issues or PRs on the Espressif repos or via other channels (email, discord, or even around the coffee table :) )

If it were only a single person maintaining an important tool you depend on, then I would also have my doubts.

esphome is using pioarduino since a while now too. There are enough skilled developers in background of pioarduino to keep it alive as long espressif MCUs are widely used.

@0x0fe
Copy link

0x0fe commented Jan 9, 2025

I sincerely can't wait to find another reason to abandon Espressif

You'll "abandon" espressif SoCs because a third party package manager drops the support of arduino core, alright, sounds perfectly reasonable.

espressif seems to be quite proud of the high number of esp-based projects on github,
while still denying any business value from supporting platformI
I guess that more than half of these "115K+" have a platformio.in

guess != facts, not sure how one can assume these projects are only possible thanks to PIO. It is just a package manager, the editor is microsoft VSCode, the toolchain is Espressif IDF. the majority of commercial projects are not using pio but just the official IDF with eclipse IDE, or IDF with VScode or even arduino with its IDE.

I wonder how all these peoples so disturbed and concerned about a third party package manager dropping support of Arduino, another third party SDK, can work on actual commercial projects with exotic SoCs or MCU where there is only one possible IDE (often Eclipse based) and toolchain, no "libraries", no chatgpt, or when you can only work on keil UV5 (no syntax highlight, no code browse etc).

Come on guys, its just a package manager, there are plenty of options to not use it and still work on IDF, and even vscode, even if you want to use Arduino core, and contrary to what has been said the official VScode extension works perfectly well, there is like 700 closed tickets and 20 open, and if you have a problem you just need to reach out to Espressif by email or with your allocated FAE. If PIO has so much value, they can sell it and peoples will buy, i guess ;)

I think the lesson to be learned here is to not thrust third parties that pretend to be "open source" and "community", because as you can see, if there is no money to make, then the "community" part will vanish very fast, lol.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests