Electronics

PySerial minimal example: Copy data received from serial port to stdout

This example does not send data to the serial port but only copies data received from the serial port to stdout. Newlines received from the serial port are preserved.

#!/usr/bin/env python3
import serial
ser = serial.Serial("/dev/ttyUSB0", baudrate=115200)

try:
    while True:
        response = ser.read()
        if response:
            print(response.decode("iso-8859-1"), end="")
finally:
    ser.close()

By using iso-8859-1   decoding, we ensure that even binary bytes are decoded in some way and do not cause an exception.

Posted by Uli Köhler in 3D printing, Embedded, Python

How to Serial.println() a std::string

In Arduino, if you have a std::string:

std::string str = "test";

you can’t directly print it – trying to do so leads to the following error messages:

src/main.cpp: In function 'void setup()':
src/main.cpp:122:22: error: no matching function for call to 'HardwareSerial::println(std::__cxx11::string&)'
   Serial.println(cert);
                      ^
In file included from /home/uli/.platformio/packages/[email protected]2/cores/esp32/Stream.h:26,
                 from /home/uli/.platformio/packages/[email protected]2/cores/esp32/Arduino.h:166,
                 from src/main.cpp:1:
/home/uli/.platformio/packages/[email protected]2/cores/esp32/Print.h:96:12: note: candidate: 'size_t Print::println(const __FlashStringHelper*)'
     size_t println(const __FlashStringHelper *);
            ^~~~~~~
/home/uli/.platformio/packages/[email protected]2/cores/esp32/Print.h:96:12: note:   no known conversion for argument 1 from 'std::__cxx11::string' {aka 'std::__cxx11::basic_string<char>'} to 'const __FlashStringHelper*'
/home/uli/.platformio/packages/[email protected]2/cores/esp32/Print.h:97:12: note: candidate: 'size_t Print::println(const String&)'
     size_t println(const String &s);

 

Solution:

You can use .c_str() to convert it to a NUL-terminated char* which can be printed directly:

Serial.println(str.c_str());

 

Posted by Uli Köhler in Arduino, C/C++, Embedded

How to convert Arduino String to std::string

If you have an Arduino String:

String arduinoStr = "test123";

you can easily convert it to a std::string by using:

std::string stdStr(arduinoStr.c_str(), arduinoStr.length());

The std::string constructor will copy the data, therefore you can de-allocate the Arduino String instance safely.

Posted by Uli Köhler in Arduino, C/C++

Should you use SPIFFS oder LittleFS?

For new projects, you should exclusively use LittleFS, period.

The only reason why you would you SPIFFS at all if you have data stored on SPIFFS and can’t easily migrate to LittleFS.

LittleFS is just better than SPIFFS in every regard and SPIFFS will at some point in time be deprecated anyways.

Posted by Uli Köhler in Embedded, ESP8266/ESP32, PlatformIO

How to get filesize in LittleFS (ESP32/PlatformIO)

After you have initialized LittleFS (see ESP32 Filesystem initialization code example (LittleFS)) you can get the filesize by first opening the file, and then calling .size() on the opened file. Don’t forget to close the file afterwards.

auto file = LittleFS.open(filename, "r");
size_t filesize = file.size();
// Don't forget to clean up!
file.close();

Utility function to get the size of a file stored on LittleFS:

size_t LittleFSFilesize(const char* filename) {
  auto file = LittleFS.open(filename, "r");
  size_t filesize = file.size();
  // Don't forget to clean up!
  file.close();
  return filesize;
}

Example usage:

Serial.println(LittleFSFilesize("/cert.pem"));

 

Posted by Uli Köhler in C/C++, ESP8266/ESP32, PlatformIO

How to fix ESP32 PlatformIO esp_https_server.h: No such file or directory

Problem:

When compiling your PlatformIO project, you see a compiler error like

src/main.cpp:6:10: fatal error: esp_https_server.h: No such file or directory

at the following line:

#include <esp_https_server.h>

Solution:

The ESP32 HTTPS server library is not included with the older versions of arduino-espressif32. The solution therefore is to use a more recent version of the platform library such as version 2.0.4. In order to do that, add the following line to your platformio.ini:

platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.4

Complete platformio.ini example:

[env:esp32dev]
platform = espressif32
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.4
board = esp32dev
framework = arduino
monitor_speed = 115200

 

Posted by Uli Köhler in Embedded, ESP8266/ESP32, PlatformIO

How to set static IP address on ESP32

TL;DR

Before you call WiFi.begin(...), call this:

if (!WiFi.config(
      IPAddress(192, 168, 19, 5), // ESP's IP address
      IPAddress(192, 168, 19, 1), // Gateway
      IPAddress(255, 255, 255, 0), // IP Address
      IPAddress(192, 168, 19, 1) // DNS server
    )) {
  Serial.println("Failed to set static IP");
}

and replace the IP address etc with the static

Full example

This is based on our previous post on How to fix ESP32 not connecting to the Wifi network:

#include <Arduino.h>
#include <WiFi.h>

void setup() {
  Serial.begin(115200);
  WiFi.begin("MyWifiSSID", "MyWifiPassword");

  // Set static IP
  if (!WiFi.config(
        IPAddress(192, 168, 19, 5), // ESP's IP address
        IPAddress(192, 168, 19, 2), // Gateway
        IPAddress(255, 255, 255, 0), // IP Address
        IPAddress(192, 168, 19, 1) // DNS server
      )) {
    Serial.println("Failed to set static IP");
  }

  // Wait for wifi to be connected
  uint32_t notConnectedCounter = 0;
  while (WiFi.status() != WL_CONNECTED) {
      delay(100);
      Serial.println("Wifi connecting...");
      notConnectedCounter++;
      if(notConnectedCounter > 150) { // Reset board if not connected after 5s
          Serial.println("Resetting due to Wifi not connecting...");
          ESP.restart();
      }
  }
  Serial.print("Wifi connected, IP address: ");
  Serial.println(WiFi.localIP());
}

void loop() {
  // put your main code here, to run repeatedly:
}

 

Posted by Uli Köhler in Embedded, ESP8266/ESP32, Networking

How to get current task handle in FreeRTOS

Simply use xTaskGetCurrentTaskHandle()

#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

TaskHandle_t myTaskHandle = xTaskGetCurrentTaskHandle();

 

Posted by Uli Köhler in Embedded, FreeRTOS

How to fix STM32 error: expected constructor, destructor, or type conversion before __declspec(dllexport)

Problem:

When trying to compile a firmware for an STM32 microcontroller, you see a compiler error message like

myheader.h:23:12: error: expected constructor, destructor, or type conversion before ‘(’ token
   23 |  __declspec(dllexport) int myfunc(

Solution:

We previously explored the same problem for Linux platforms.

__declspec(dllexport) is a Windows-specific feature and not available on non-Windows platform such as the ARM embedded API platform (e.g. STM32). In order to fix it in a compatible way with both Windows and the STM32, add the following code either in a header that is included in every file containing __declspec(dllexport) or add it in each file where the error occurs:

#ifdef __ARM_EABI__
#define __declspec(v)
#endif

This will basically ignore any __declspec() call on the preprocessor level.

By using __ARM_EABI__ specfically, the definition will not trigger for ARM platforms for Windows.

Posted by Uli Köhler in C/C++, GCC errors, STM32

How to install ESP32 espota.py on Linux

Currently there is no pip install ... way of installing espota.py. The easiest way of installing it is to download it from GitHub:

wget https://raw.githubusercontent.com/espressif/arduino-esp32/master/tools/espota.py

 

Posted by Uli Köhler in Embedded, ESP8266/ESP32, Networking

How to fix ESP32 esptool.py Running stub… StopIteration

Problem:

When flashing an ESP32, especially when flashing remotely, you see the following log message

Serial port rfc2217://10.1.2.3.105:4418
Connecting...
Device PID identification is only supported on COM and /dev/ serial ports.
..
Chip is ESP32-S2
Features: WiFi, No Embedded Flash, No Embedded PSRAM, ADC and temperature sensor calibration in BLK2 of efuse V2
Crystal is 40MHz
MAC: 60:55:f9:f0:3a:16
Uploading stub...
Running stub...
Traceback (most recent call last):
  File "/usr/local/bin/esptool.py", line 34, in <module>
    esptool._main()
  File "/usr/local/lib/python3.8/dist-packages/esptool/__init__.py", line 1004, in _main
    main()
  File "/usr/local/lib/python3.8/dist-packages/esptool/__init__.py", line 684, in main
    esp = esp.run_stub()
  File "/usr/local/lib/python3.8/dist-packages/esptool/loader.py", line 912, in run_stub
    p = self.read()
  File "/usr/local/lib/python3.8/dist-packages/esptool/loader.py", line 307, in read
    return next(self._slip_reader)
StopIteration

Solution:

This issue occurs not because of a hardware defect but because the latency of the connection is quite high and therefore the timeout occurs before the communication between the script and the bootloader can happen.

In order to fix it, you need to edit the hard-coded default timeout in the esptool.py installation!

First, you need to identify the location of loader.py in your installation. You can simply do that from the following part of the stack trace:

File "/usr/local/lib/python3.8/dist-packages/esptool/loader.py", line 912, in run_stub p = self.read()

In this case, loader.py is located at

/usr/local/lib/python3.8/dist-packages/esptool/loader.py

You need to edit the following line:

MEM_END_ROM_TIMEOUT = 0.05  # short timeout for ESP_MEM_END, as it may never respond

and increase the timeout – my recommendation is to increase it to 2.5 seconds.

… or you can simply change the timeout using the following command:

sed -i -e 's/MEM_END_ROM_TIMEOUT = 0.05/MEM_END_ROM_TIMEOUT = 2.5/g' /usr/local/lib/python3.8/dist-packages/esptool/loader.py

After that, your upload should work just fine.

Posted by Uli Köhler in Electronics, Embedded, ESP8266/ESP32, Networking

How to write std::string to ESP32 NVS

Also see: How to read ESP32 NVS value into std::string

Writing a std::string into NVS is pretty easy. You can just use .c_str() to obtain a const char* pointer and then proceed as shown in How to write string (const char*) to ESP32 NVS

You can use this utility function to write a string (const char*) into the NVS:

bool NVSWriteString(nvs_handle_t nvs, const std::string& key, const std::string& value) {
    esp_err_t err;
    if((err = nvs_set_blob(nvs, key.c_str(), value.data(), value.size())) != ESP_OK) {
        Serial.printf("Failed to write NVS key %s: %srn", key, esp_err_to_name(err));
        return false;
    }
    return true;
}

Usage example

This example is based on How to initialize NVS on ESP32. Most importantly, it assumes you have already initialized the NVS and myNVS exists and is valid.

char mySerialNo[128] = {0};

_NVS_WriteString(myNVS, "SerialNo", "0000001");

This will write the value 0000001 to the NVS with key SerialNo

Note that NVS strings are length-delimited (as in: The length is stored in the NVS) and not neccessarily NUL-terminated. This code does not store the NUL terminator in NVS.

Posted by Uli Köhler in ESP8266/ESP32

How to read ESP32 NVS value into std::string

In our previous post, we discussed How to get the length / size of NVS value on ESP32. Based on that, we can read a NVS value into a std::string.

Strategy

  1. Determine size of value in NVS
  2. Allocate temporary buffer of the determined size
  3. Read value from NVS into temporary buffer
  4. Create std::string from value
  5. Cleanup temporary buffer

Utility function to read NVS value as std::string

In case the key does not exist in NVS, this function will return the empty string ("").

#include <nvs.h>
#include <string>

std::string ReadNVSValueAsStdString(nvs_handle_t nvs, const char* key) {
    /**
     * Strategy:
     *  1. Determine size of value in NVS
     *  2. Allocate temporary buffer of determined size
     *  3. Read value from NVS into temporary buffer
     *  4. Create std::string from value
     *  5. Cleanup
     */
    // Step 1: Get size of key
    esp_err_t err;
    size_t value_size = 0;
    if((err = nvs_get_str(nvs, _key.c_str(), nullptr, &value_size)) != ESP_OK) {
        if(err == ESP_ERR_NVS_NOT_FOUND) {
            // Not found, no error
            return "";
        } else {
            printf("Failed to get size of NVS key %s: %s\r\n", key, esp_err_to_name(err));
            return;
        }
    }
    // Step 2: Allocate temporary buffer to read from
    char* buf = (char*)malloc(value_size);
    // Step 3: Read value into temporary buffer.
    esp_err_t err;
    if((err = nvs_get_str(nvs, _key.c_str(), buf, &value_size)) != ESP_OK) {
        // "Doesn't exist" has already been handled before, so this is an actual error.
        // We assume that the value did not change between reading the size (step 1) and now.
        // In case that assumption is value, this will fail with ESP_ERR_NVS_INVALID_LENGTH.
        // This is extremely unlikely in all usage scenarios, however.
        printf("Failed to read NVS key %s: %s\r\n", key, esp_err_to_name(err));
        free(buf);
        return "";
    }
    // Step 4: Make string
    std::string value = std::string(buf, value_size);
    // Step 5: cleanup
    free(buf);
    
    return value;
}

Usage example

This assumes that you have setup myNvs as we have shown in our previous post How to initialize NVS on ESP32

std::string value = ReadNVSValueAsStdString(myNvs, "MyKey");

C++17 optimizations

Starting from C++17, you can possibly create a std::string directly instead of using the temporary buffer, since there is an overload of .data() that returns a non-const pointer – so you can write directly to the std::string‘s buffer.

However, since my PlatformIO-based toolchain currently doesn’t support that, I have not written that code yet.

Posted by Uli Köhler in C/C++, ESP8266/ESP32

How to get length / size of NVS value on ESP32

You can find the size of a value stored in ESP32 NVS by running nvs_get_blob() with length set to a pointer to a variable with content 0, and out_value set to nullptrnvs_get_str() works as well, the only difference is the type of the out_value argument, which does not matter at all.

It’s easier to understand if you look at this example function:

size_t NVSGetSize(nvs_handle_t nvs, const char* key) {
    esp_err_t err;
    size_t valueSize = 0;
    if((err = nvs_get_str(nvs, key, nullptr, &valueSize)) != ESP_OK) {
        if(err == ESP_ERR_NVS_NOT_FOUND) {
            // Not found, no error
            return 0;
        } else {
             // Actual error, log & return 0
            printf("Failed to get size of NVS key %s: %s\r\n", key, esp_err_to_name(err));
            return 0;
        }
    }
    return valueSize;
}

Usage example

This assumes that you have setup myNvs as we have shown in our previous post How to initialize NVS on ESP32

size_t myKeySize = NVSGetSize(myNvs, "MyKey");

 

Posted by Uli Köhler in ESP8266/ESP32

How to write string (const char*) to ESP32 NVS

Also see: How to read string (const char*) from ESP32 NVS

You can use this utility function to write a string (const char*) into the NVS:

bool _NVS_WriteString(nvs_handle_t nvs, const char* key, const char* value) {
    esp_err_t err;
    if((err = nvs_set_str(nvs, key, value)) != ESP_OK) {
        Serial.printf("Failed to write NVS key %s: %s\r\n", key, esp_err_to_name(err));
        return false;
    }
    return true;
}

Usage example

This example is based on How to initialize NVS on ESP32. Most importantly, it assumes you have already initialized the NVS and myNVS exists and is valid.

char mySerialNo[128] = {0};

_NVS_WriteString(myNVS, "SerialNo", "0000001");

This will write the value 0000001 to the NVS with key SerialNo

Note that NVS strings are length-delimited (as in: The length is stored in the NVS) and not neccessarily NUL-terminated. This code does not store the NUL terminator in NVS.

Posted by Uli Köhler in ESP8266/ESP32

How to read string (const char*) from ESP32 NVS

Also see: How to write string (const char*) to ESP32 NVS

 

You can use this utility function to read a string:

bool _NVS_ReadString(nvs_handle_t nvs, const char* key, char* value, size_t maxSize) {
    esp_err_t err;
    if((err = nvs_get_str(nvs, key, value, &maxSize)) != ESP_OK) {
        Serial.printf("Failed to read NVS key %s: %s\r\n", key, esp_err_to_name(err));
        return false;
    }
    return true;
}

Usage example

This example is based on How to initialize NVS on ESP32. Most importantly, it assumes you have already initialized the NVS and myNVS exists and is valid.

char mySerialNo[128] = {0};

_NVS_ReadString(myNVS, "SerialNo", mySerialNo, 128-1);

This will read the NVS entry with key SerialNo into the buffer mySerialNo. Note that we are using 128-1 as size, even though the buffer has a size of 128. This is to ensure that there is ALWAYS a terminating NUL character at the last value of the buffer.

Note that NVS strings are length-delimited (as in: The length is stored in the NVS) and not neccessarily NUL-terminated. I recommend to not store the NUL terminator in the NVS.

Posted by Uli Köhler in ESP8266/ESP32

How to initialize NVS on ESP32

First, include the ESP32 NVS library using:

#include <nvs.h>

Globally, declare

nvs_handle_t myNVS = 0;

Then you can

void InitNVS() {
    esp_err_t err;
    if((err = nvs_open("MyLabel", NVS_READWRITE, &myNVS)) != ESP_OK) {
        Serial.printf("Failed to open NVS: %s\r\n", esp_err_to_name(err));
        return;
    }
}

You can choose MyLabel arbitrarily, as long as the string isn’t too long. My recommendation is to choose a unique identifier for your application.

Posted by Uli Köhler in ESP8266/ESP32

How to add extra include directory to PlatformIO project

In platformio.ini, add the following build flags:

build_flags = -I../my-extra-include-dir

By using this method, you can also add multiple include directories:

build_flags = -I../my-first-include-dir -I../my-second-include-dir

All paths are relative to the directory where platformio.ini resides in.

Posted by Uli Köhler in PlatformIO

How to fix NanoPB MyProtocol.proto: Option “(nanopb)” unknown.

Problem:

When trying to compile a project using NanoPB, you see a warning like

MyProtocol.proto:11:27: Option "(nanopb)" unknown. Ensure that your proto definition file imports the proto which defines the option.

Solution:

At the top of the .proto file mentioned in the warning message, just after syntax = "proto3"; add the following line:

import "nanopb.proto";

After that, the error should be gone.

Posted by Uli Köhler in NanoPB

How to fix PlatformIO ValueError: Invalid simple block ‘^0.4.6.4’

Problem:

You want to build your PlatformIO project which has library dependency like

lib_deps =
    nanopb/[email protected]^0.4.6.4

but you see an error message like

Error: Traceback (most recent call last):
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/__main__.py", line 102, in main
    cli()  # pylint: disable=no-value-for-parameter
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/cli.py", line 71, in invoke
    return super().invoke(ctx)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/run/cli.py", line 144, in cli
    process_env(
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/run/cli.py", line 201, in process_env
    result = {"env": name, "duration": time(), "succeeded": ep.process()}
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/run/processor.py", line 83, in process
    install_project_env_dependencies(
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/commands/install.py", line 132, in install_project_env_dependencies
    _install_project_env_libraries(project_env, options),
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/commands/install.py", line 241, in _install_project_env_libraries
    spec = PackageSpec(library)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/meta.py", line 184, in __init__
    self._parse(self.raw)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/meta.py", line 291, in _parse
    raw = parser(raw)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/meta.py", line 316, in _parse_requirements
    self.requirements = tokens[1].strip()
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/platformio/package/meta.py", line 231, in requirements
    else semantic_version.SimpleSpec(str(value))
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/semantic_version/base.py", line 647, in __init__
    self.clause = self._parse_to_clause(expression)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/semantic_version/base.py", line 1043, in _parse_to_clause
    return cls.Parser.parse(expression)
  File "/home/uli/.platformio/penv/lib/python3.10/site-packages/semantic_version/base.py", line 1063, in parse
    raise ValueError("Invalid simple block %r" % block)
ValueError: Invalid simple block '^0.4.6.4'

Solution:

You are using the library version 0.4.6.4 but the library version specifier does not support versions with 4 levels (a.b.c.d) – the correct version specifier is just using the first three digits: a.b.c. In our example, this would be

lib_deps =
  nanopb/[email protected]^0.4.6

After that, you have to delete your .pio directory in the project folder in order to fix the issue:

rm -rf .pio

When that is done, rebuild and the issue will be gone.

Posted by Uli Köhler in ESP8266/ESP32, NanoPB, PlatformIO