ESP8266/ESP32

How to fix PlatformIO ValueError: Invalid simple block ‘^0.4.6.4’

Problem:

You want to build your PlatformIO project which has library dependency like

lib_deps =
    nanopb/Nanopb@^0.4.6.4

but you see an error message like

Error: Traceback (most recent call last):
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/__main__.py", line 102, in main
    cli()  # pylint: disable=no-value-for-parameter
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/cli.py", line 71, in invoke
    return super().invoke(ctx)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/run/cli.py", line 144, in cli
    process_env(
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/run/cli.py", line 201, in process_env
    result = {"env": name, "duration": time(), "succeeded": ep.process()}
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/run/processor.py", line 83, in process
    install_project_env_dependencies(
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/commands/install.py", line 132, in install_project_env_dependencies
    _install_project_env_libraries(project_env, options),
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/commands/install.py", line 241, in _install_project_env_libraries
    spec = PackageSpec(library)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/meta.py", line 184, in __init__
    self._parse(self.raw)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/meta.py", line 291, in _parse
    raw = parser(raw)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/meta.py", line 316, in _parse_requirements
    self.requirements = tokens[1].strip()
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/meta.py", line 231, in requirements
    else semantic_version.SimpleSpec(str(value))
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/semantic_version/base.py", line 647, in __init__
    self.clause = self._parse_to_clause(expression)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/semantic_version/base.py", line 1043, in _parse_to_clause
    return cls.Parser.parse(expression)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/semantic_version/base.py", line 1063, in parse
    raise ValueError("Invalid simple block %r" % block)
ValueError: Invalid simple block '^0.4.6.4'

Solution:

You are using the library version 0.4.6.4 but the library version specifier does not support versions with 4 levels (a.b.c.d) – the correct version specifier is just using the first three digits: a.b.c. In our example, this would be

lib_deps =
  nanopb/Nanopb@^0.4.6

After that, you have to delete your .pio directory in the project folder in order to fix the issue:

rm -rf .pio

When that is done, rebuild and the issue will be gone.

Posted by Uli Köhler in ESP8266/ESP32, NanoPB, PlatformIO

How to link Angular project dist directory to PlatformIO SPIFFS data directory

For tips how to make the Angular build small enough to fit into the SPIFFS image, see How to make Angular work with ESP32 SPIFFS / ESPAsyncWebserver

When you are building a PlatformIO image, you can easily make the dist/[project_name] directory from the Angular project directory appear in the SPIFFS image by using a symlink.

My config tells the server to serve from the www subdirectory.

server.serveStatic("/", SPIFFS, "/www/").setDefaultFile("index.html");

Therefore, we first need to create the data directory in the same directory where platformio.ini is located:

mkdir data

Now we can create a symlink from the angular dist directory to data/www, for example:

ln -s ../MyUI/dist/myui data/www

PlatformIO will automatically handle the symlink, if the directory exists.

Posted by Uli Köhler in Angular, ESP8266/ESP32, PlatformIO

How to make Angular work with ESP32 SPIFFS / ESPAsyncWebserver

The main issue when using Angular web UIs is that the resulting files get too large, hence building the filesystem image will fail with SPIFFS_write error(-10001): File system is full.

Using these tips, I could get an Angular PrimeNG app to fit into a 4MB flash ESP32 module without any custom partition table and without any other crazy hacks! Even the fonts & PrimeNG icons fit into the SPIFFS easily, with a total of only 380 kB of the approximately 1.5 MB being consumed.

File compression

The number one most important tip is that you can just gzip -9 the files from the angular dist directory and ESPAsyncWebserver will automatically handle decompressing them!

This is my platformio.ini:

[env:esp32dev]
platform = espressif32
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.3
board = esp32dev
framework = arduino
board_build.filesystem = littlefs
lib_deps =
    esphome/AsyncTCP-esphome@^1.2.2
    esphome/ESPAsyncWebServer-esphome@^2.1.0
    [email protected]
upload_speed = 460800
monitor_speed = 115200

This is my angular build script:

#!/bin/sh
ng build --aot --build-optimizer --optimization --progress --output-hashing none
gzip -9 dist/**/*

This is where I tell ESPAsyncWebserver (note that you should use the esphome fork) to serve files statically:

server.serveStatic("/", SPIFFS, "/www/").setDefaultFile("index.html");

Other tips

In order to make your life easier managing the data directory with both Angular files and other files, see How to link Angular project dist directory to PlatformIO SPIFFS data directory

You can use purgecss but compression works so well that it isn’t really worth both the risk of accidentally removing some CSS rules which you manually need to whitelist. Before discovering how well compression worked, I started to manually remove CSS rules from the PrimeNG theme file. This worked fine, but the SPIFFS still wasn’t small enough.

Often you can save space by deleting.

For example, primeicons.svg and primeicons.ttf are two different formats with the same content. Note that some (especially older, and some mobile) browsers don’t support all formats, hence it’s rather risky to remove them if you need to support multiple platforms.

Posted by Uli Köhler in Angular, ESP8266/ESP32, PlatformIO

How to fix SPIFFS_write error(-10001): File system is full.

Are you using Angular and seeing this error message? See How to make Angular work with ESP32 SPIFFS / ESPAsyncWebserver

The error message

SPIFFS_write error(-10001): File system is full.

should be mostly self-explanatory:

The files you are trying to put into the image exceed the maximum size of the filesystem image.

There are two possible solutions:

  • Decrease the size of the files, either by minifying files where this is possible, or removing some files altogether.
  • Increase the allowable size of the filesystem image e.g. by using a ESP32 module with more flash (e.g. 8MB instead of 4MB), and/or using a partition table with a larger file partition.
Posted by Uli Köhler in ESP8266/ESP32, PlatformIO

How to fix SPIFFS_write error(-10010): unknown

Problem:

When trying to build a SPIFFS image, you see an error message like

SPIFFS_write error(-10010): unknown
error adding file!
Error for adding content from www!
/www/Inter-Light.27083fa6375bb9ef.woff2

Solution:

SPIFFS_write error(-10010): unknown means that the filename is too long. SPIFFS only supports filenames with a maximum of 32 characters.

In our example, Inter-Light.27083fa6375bb9ef.woff2 is 32 characters long.

Hence, the solution is to shorten the filename of every file with a filename (including extension) of 32+ characters.

Posted by Uli Köhler in Embedded, ESP8266/ESP32, PlatformIO

How to fix ESPAsyncWebserver undefined reference to `SHA1Init’

Problem:

When using ESPAsyncWebserver with websockets, you see the following error messages while linking:

Linking .pio/build/esp32dev/firmware.elf
/home/uli/.platformio/packages/[email protected]+2021r2-patch3/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/lib67a/libESP Async WebServer.a(AsyncWebSocket.cpp.o):(.literal._ZN22AsyncWebSocketResponseC2ERK6StringP14AsyncWebSocket+0x10): undefined reference to `SHA1Init'
/home/uli/.platformio/packages/[email protected]+2021r2-patch3/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/lib67a/libESP Async WebServer.a(AsyncWebSocket.cpp.o):(.literal._ZN22AsyncWebSocketResponseC2ERK6StringP14AsyncWebSocket+0x18): undefined reference to `SHA1Update'
/home/uli/.platformio/packages/[email protected]+2021r2-patch3/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/lib67a/libESP Async WebServer.a(AsyncWebSocket.cpp.o):(.literal._ZN22AsyncWebSocketResponseC2ERK6StringP14AsyncWebSocket+0x1c): undefined reference to `SHA1Final'
/home/uli/.platformio/packages/[email protected]+2021r2-patch3/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/lib67a/libESP Async WebServer.a(AsyncWebSocket.cpp.o): in function `AsyncWebSocketResponse::AsyncWebSocketResponse(String const&, AsyncWebSocket*)':
/home/uli/dev/My-Firmware/.pio/libdeps/esp32dev/ESP Async WebServer/src/AsyncWebSocket.cpp:1269: undefined reference to `SHA1Init'
/home/uli/.platformio/packages/[email protected]+2021r2-patch3/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/uli/dev/My-Firmware/.pio/libdeps/esp32dev/ESP Async WebServer/src/AsyncWebSocket.cpp:1270: undefined reference to `SHA1Update'
/home/uli/.platformio/packages/[email protected]+2021r2-patch3/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/uli/dev/My-Firmware/.pio/libdeps/esp32dev/ESP Async WebServer/src/AsyncWebSocket.cpp:1271: undefined reference to `SHA1Final'
/home/uli/.platformio/packages/[email protected]+2021r2-patch3/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/lib67a/libESP Async WebServer.a(WebAuthentication.cpp.o):(.literal._ZL6getMD5PhtPc+0x4): undefined reference to `mbedtls_md5_starts'
/home/uli/.platformio/packages/[email protected]+2021r2-patch3/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/lib67a/libESP Async WebServer.a(WebAuthentication.cpp.o): in function `getMD5(unsigned char*, unsigned short, char*)':
/home/uli/dev/My-Firmware/.pio/libdeps/esp32dev/ESP Async WebServer/src/WebAuthentication.cpp:73: undefined reference to `mbedtls_md5_starts'

Solution:

This is a bug in the ESPAsyncWebserver official library, which is not regularly maintained. But you can use use the esphome fork of ESPAsyncWebserver and their fork of AsyncTCP instead.

In platformio.ini, instead of

lib_deps =
    ESP Async [email protected]

use the esphome fork and AsyncTCP:

lib_deps =
    esphome/AsyncTCP-esphome@^1.2.2
    esphome/ESPAsyncWebServer-esphome@^2.1.0

After that, try to build / upload, the linking errors should be gone.

Full platformio.ini example

[env:esp32dev]
platform = espressif32
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.3
board = esp32dev
framework = arduino
lib_deps =
    esphome/AsyncTCP-esphome@^1.2.2
    esphome/ESPAsyncWebServer-esphome@^2.1.0
    [email protected]
upload_speed = 460800
monitor_speed = 115200

 

Posted by Uli Köhler in Embedded, ESP8266/ESP32, PlatformIO

How to compute Ethernet CRC32 on ESP32 (0x04C11DB7)

The ESP32 ROM provides functionality for calculating CRC8, CRC16 & CRC32 checksums, but that functionality is poorly documented.

By educated trial and error and comparing with Arduino_CRC32 (which implements the Ethernet CRC32 algorithm with polynomial 0x04C11DB7 based on the pyCRC library) I found out how to compute the CRC32 checksum using the Ethernet polynomial.

First, include crc.h:

#include <esp32/rom/crc.h>

Now given a buffer with length length, use this code:

uint32_t romCRC = (~crc32_le((uint32_t)~(0xffffffff), (const uint8_t*)buffer, length))^0xffffffff;

Complete example code

This PlatformIO/Arduino code compares the result of the ESP32 ROM CRC32 functiolity by printing out both results on the serial port.

#include <Arduino.h>
#include <esp32/rom/crc.h>
#include <Arduino_CRC32.h>

void setup() {
    Serial.begin(115200);
}

void loop() {
    const char* data = "ABCDEFGHIJ";
  
    // Compute using ESP32 ROM CRC library
    uint32_t romCRC = (~crc32_le((uint32_t)~(0xffffffff), (const uint8_t*)data, 8))^0xffffffFF;

    // Compute using Arduino_CRC32 library (based on pyCRC)
    Arduino_CRC32 crc32;
    uint32_t libCRC = crc32.calc((uint8_t const *)data, 8);

    // Print out btoh v
    char crcBytes[4];
    memcpy(crcBytes, &romCRC, sizeof(uint32_t));
    Serial.printf("ROM CRC: %02X %02X %02X %02X\n", crcBytes[0], crcBytes[1], crcBytes[2], crcBytes[3]);

    memcpy(crcBytes, &libCRC, sizeof(uint32_t));
    Serial.printf("Lib CRC: %02X %02X %02X %02X\n", crcBytes[0], crcBytes[1], crcBytes[2], crcBytes[3]);
    Serial.println("\n");
    delay(500);
}
[env:esp32dev]
platform = espressif32
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.3
board = esp32dev
framework = arduino
monitor_speed = 115200
lib_deps =
    arduino-libraries/Arduino_CRC32@^1.0.0

Example output:

ROM CRC: 1C B6 DC 68
Lib CRC: 1C B6 DC 68

 

Posted by Uli Köhler in Arduino, ESP8266/ESP32

What is SYS_STARTUP_FN() on the ESP32?

SYS_STARTUP_FN() is being called at the end of call_start_cpu0() during the app startup process (after the bootloader has already finished executing).

It essentially just calls the function pointer g_startup_fn for the current core:

#define SYS_STARTUP_FN()  ((*g_startup_fn[(cpu_hal_get_core_id())])())
g_startup_fn, in turn, is defined to basically start_cpu0 and start_cpu1 in this rather convoluted but still conceptually easy code.

 

Posted by Uli Köhler in Embedded, ESP8266/ESP32

Where is esp_startup_start_app_common() started on the ESP32 (ESP-IDF)?

esp_startup_start_app_common() is called in start_cpu0_default(), which is located in components/esp_system/startup.c
Note that start_cpu0_default() is a weak alias for start_cpu0().

For version 4.4.1 of the ESP-IDF, this is the link to the location where it is called.

Posted by Uli Köhler in Embedded, ESP8266/ESP32

Where is esp_startup_start_app_common() started on the ESP32 (ESP-IDF)?

esp_startup_start_app_common() is called in esp_startup_start_app(), which is a function specific to the architecture (xtensa or riscv).

For version 4.4.1 of the ESP-IDF, this is the link to the location where it is called for the xtensa port.

Posted by Uli Köhler in Embedded, ESP8266/ESP32

Where is the bootloader_init() source code for the ESP32?

In the ESP32 bootloader_start.c, you can see that bootloader_init() is called during bootloader startup:

if (bootloader_init() != ESP_OK) {
    bootloader_reset();
}

The implementation of bootloader_init() is sightly harder to find because it is not located in components/bootloader but components/bootloader_support, within a specific platform-specific subdirectory, for example in components/bootloader_support/src/esp32s2/bootloader_esp32s2.c for the ESP32S2 – Github link to the v4.4.1 source code.

 

 

Posted by Uli Köhler in Embedded, ESP8266/ESP32

Where is main_task() started on the ESP32 (ESP-IDF)?

The FreeRTOS main taskmain_task() (which in turn starts app_main()) is  started in esp_startup_start_app_common():

portBASE_TYPE res = xTaskCreatePinnedToCore(&main_task, "main",
                                            ESP_TASK_MAIN_STACK, NULL,
                                            ESP_TASK_MAIN_PRIO, NULL, ESP_TASK_MAIN_CORE);

 

For version 4.4.1 of the ESP-IDF, this is the link to the location where it is called.

Posted by Uli Köhler in Embedded, ESP8266/ESP32

What are the default UART0 RX/TX pins for the ESP32S2?

As shown in the ESP32S2 datasheet:

  • U0TXD (UART 0 transmit) is GPIO43, which is pin 48 in the QFN56 package#
  • U0RXD is GPIO44, which is pin 49 in the QFN56 package
Posted by Uli Köhler in Embedded, ESP8266/ESP32

Which pins are USB D+/D- on the ESP32S2?

As listed in the ESP32S2 datasheet,

  • GPIO19 is USB-D-
  • GPIO20 is USB-D+
Posted by Uli Köhler in Electronics, Embedded, ESP8266/ESP32

How to fix PlatformIO No such file or directory bootloader__80m.bin/bootloader__40m.bin

Problem:

While trying to upload/flash an ESP32(S2/S3) project using PlatformIO with a platformio.ini like this:

[env:esp32-s2-saola-1]
platform = espressif32
board = esp32-s2-saola-1
framework = arduino

you see an error message like

esptool write_flash: error: argument <address> <filename>: [Errno 2] No such file or directory: '/home/uli/.platformio/packages/framework-arduinoespressif32/tools/sdk/esp32s2/bin/bootloader__80m.bin'
*** [upload] Error 2

or

esptool write_flash: error: argument <address> <filename>: [Errno 2] No such file or directory: '/home/uli/.platformio/packages/framework-arduinoespressif32/tools/sdk/esp32/bin/bootloader__40m.bin'

Solution:

This is a bug in version 2.0.4 of the arduino-espressif32 framework. In order to work around it, use version 2.0.3 until this bux is fixed. You can do this by appending

platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.3

to platformio.ini. Full example:

[env:esp32dev]
platform = espressif32
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.3
board = esp32dev
framework = arduino

 

Posted by Uli Köhler in Arduino, ESP8266/ESP32, PlatformIO

How to initialize struct sockaddr_in using initializer lists

This way of initializing a struct sockaddr_in uses modern C and avoids all the raw of days that should long be forgotten, at least for most applications. I use this approach successfully e.g. for ESP32 microcontrollers.

struct sockaddr_in server_addr = {
    .sin_family = AF_INET,
    .sin_port = htons(46118),
    .sin_addr = {.s_addr = htonl(INADDR_ANY)}
};

 

Posted by Uli Köhler in C/C++, ESP8266/ESP32, PlatformIO

ESP32 Filesystem initialization code example (LittleFS)

This example code takes care of mounting the filesystem, or creating a filesystem if none is present.

#pragma once
#include <stddef.h>

// InitFilesystem() sets this to true if the filesystem is available.
extern volatile bool filesystemOK;

void InitFilesystem();
#include "FS.h"
#include "LittleFS.h"

volatile bool filesystemOK = false;

void InitFilesystem() {
  // Initialize LittleFS
  if (!LittleFS.begin(false /* false: Do not format if mount failed */)) {
    Serial.println("Failed to mount LittleFS");
    if (!LittleFS.begin(true /* true: format */)) {
      Serial.println("Failed to format LittleFS");
    } else {
      Serial.println("LittleFS formatted successfully");
      filesystemOK = true;
    }
  } else { // Initial mount success
    filesystemOK = true;
  }
}

 

Posted by Uli Köhler in C/C++, ESP8266/ESP32

How to install ESP32 esptool / esptool.py on Ubuntu

The recommended way is to install the current version using pip since the version installed using apt might be out of date:

sudo pip install esptool

After that, you can use esptool.py – note that you need to call it as esptool.py, not just as esptool!

In case you are missing pip , install python3-dev using apt:

sudo apt -y install python3-pip

 

Posted by Uli Köhler in ESP8266/ESP32, Linux, Python