Embedded

heatshrink compression library: How to compress data from a buffer

This is a starting point for how to compress data from a buffer using heatshrink and write the data into an output buffer:

Note that this has not been thoroughly tested, but at least it doesn’t crash 🙂

// TODO Your code goes here
const size_t outbuf_size = filesize + 128;
char* outbuf = new char[outbuf_size];
heatshrink_encoder* encoder = heatshrink_encoder_alloc(10, 4);
size_t tosink = filesize;
size_t output_size = 0;
while(tosink > 0) {
    size_t sunk = 0;
    auto err = heatshrink_encoder_sink(encoder, reinterpret_cast<uint8_t*>(buf), (size_t)filesize, &sunk);
    if(err != HSER_SINK_OK) {
        std::cerr << "Error sinking data" << std::endl;
        break;
    }
    if(sunk == 0) {
        std::cerr << "No data sunk" << std::endl;
        break;
    }
    // Check how much has been sunk & update tosink
    tosink -= sunk;
    // Poll for output
    size_t polled = 0;
    auto err2 = heatshrink_encoder_poll(encoder, reinterpret_cast<uint8_t*>(outbuf + output_size), outbuf_size - output_size, &polled);
    if(err2 == HSER_POLL_ERROR_NULL || err2 == HSER_POLL_ERROR_MISUSE) {
        std::cerr << "Error polling data" << std::endl;
        break;
    }
    output_size += polled;
}
// Input data finished
auto err3 = heatshrink_encoder_finish(encoder);
// Poll for final output
// Poll for output
size_t polled = 0;
auto err2 = heatshrink_encoder_poll(encoder, reinterpret_cast<uint8_t*>(outbuf + output_size), outbuf_size - output_size, &polled);
if(err2 == HSER_POLL_ERROR_NULL || err2 == HSER_POLL_ERROR_MISUSE) {
    std::cerr << "Error finally polling data" << std::endl;
}
output_size += polled;

cout << "Original size: " << filesize << ", compressed size: " << output_size << endl;

 

Posted by Uli Köhler in C/C++, Embedded

Systemd service to use a DS3231 RTC on the Raspberry Pi

The following systemd service will automatically. See this guide for more information on the setup and ensure sudo i2cdetect -y 1 detects the RTC with address 0x68.

This is an automatic service installation & enable script based on A simple systemd service autoinstall script . This script will automatically enable the service on boot:

#!/bin/bash
# This script installs and enables/starts a systemd service
# It also installs the service file
export NAME=ConfigureRTC

cat >/etc/systemd/system/${NAME}.service <<EOF
[Unit]
Description=${NAME}

[Service]
ExecStart=/bin/bash -c 'echo ds1307 0x68 > /sys/class/i2c-adapter/i2c-1/new_device && hwclock -s'
Restart=always

[Install]
WantedBy=multi-user.target
EOF

# Enable and start service
systemctl enable --now ${NAME}.service

This is just the systemd service:

[Unit]
Description=ConfigureRTC

[Service]
ExecStart=/bin/bash -c 'echo ds1307 0x68 > /sys/class/i2c-adapter/i2c-1/new_device && hwclock -s'
Restart=always

[Install]
WantedBy=multi-user.target

 

Posted by Uli Köhler in Linux, Raspberry Pi

How to enable or disable NTP time synchronization on the Raspberry Pi

Disable NTP:

sudo systemctl disable --now systemd-timesyncd

Enable NTP:

sudo systemctl enable --now systemd-timesyncd

Verifying if NTP is active

You can verify if NTP is active or not by running

timedatectl

Then look for these lines:

System clock synchronized: yes
NTP service: active

System clock synchronized will tell you if the NTP service has successfully synchronized the system time to a NTP time server: yes if synchronized, no if not synchronized.

NTP service will tell you if the NTP service is running, i.e. if it is trying to synchronize the system time to a NTP time server: active if running, inactive when not running

Output with NTP active:

               Local time: Tue 2023-03-14 16:49:28 CET
           Universal time: Tue 2023-03-14 15:49:28 UTC
                 RTC time: Tue 2023-03-14 15:49:28
                Time zone: Europe/Berlin (CET, +0100)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

Output with NTP inactive:

               Local time: Tue 2023-03-14 16:48:01 CET
           Universal time: Tue 2023-03-14 15:48:01 UTC
                 RTC time: Tue 2023-03-14 15:48:01
                Time zone: Europe/Berlin (CET, +0100)
System clock synchronized: no
              NTP service: inactive
          RTC in local TZ: no

 

Posted by Uli Köhler in Linux, Raspberry Pi

How to enable I2C port on the Raspberry Pi using raspi-config

sudo raspi-config nonint do_i2c 0

Now load the relevant modules:

sudo modprobe "i2c-bcm2835"
sudo modprobe "i2c-dev"
sudo modprobe "rtc-ds1307"

and now check if the I2C device file exists using stat /dev/i2c-1:

$ stat /dev/i2c-1
  File: /dev/i2c-1
  Size: 0               Blocks: 0          IO Block: 4096   character special file
Device: 5h/5d   Inode: 169         Links: 1     Device type: 59,1
Access: (0660/crw-rw----)  Uid: (    0/    root)   Gid: (  998/     i2c)
Access: 2023-03-14 16:23:06.643999999 +0100
Modify: 2023-03-14 16:23:06.643999999 +0100
Change: 2023-03-14 16:23:06.643999999 +0100
 Birth: -

If you instead see

ls: cannot access '/dev/i2c-1': No such file or directory

either the driver is not loaded properly or I2C is disabled. Try rebooting the system and repeating the commands above and possibly checking dmesg for any error messages.

Posted by Uli Köhler in Raspberry Pi

How to fix Raspberry Pi i2cdetect: command not found

Problem:

When trying to detect I2C devices on the Raspberry Pi (Raspbian) using i2cdetect, you see the following error:

$ i2cdetect
bash: i2cdetect: command not found

Solution:

Install i2c-tools using

sudo apt -y install i2c-tools

After installing i2c-tools , you can use i2cdetect and other related tools such as i2cget.

Posted by Uli Köhler in Embedded, Linux, Raspberry Pi

ESP-IDF HTTP webserver minimal ArduinoJson serialization example

static const httpd_uri_t valueHandler = {
    .uri       = "/api/value",
    .method    = HTTP_GET,
    .handler   = [](httpd_req_t *req) {
        httpd_resp_set_type(req, "application/json");
        // create json docuemnt
        DynamicJsonDocument json(1024);
        json["value"] = 1.0;
        // Serialize JSON to string
        std::string buf;
        serializeJson(json, buf);
        // Send response
        httpd_resp_send(req, buf.c_str(), buf.length());
        return ESP_OK;
    }
};

 

Posted by Uli Köhler in Arduino, ESP8266/ESP32

How to fix erpc ValueError: The generated shim code version … is different to the rest of eRPC code

Problem:

When trying to import your erpc project generated Python code using e.g.

from erpc_myproject import *

You see the following error message:

Traceback (most recent call last)
Cell In [1], line 1
----> 1 from erpc_myproject import *

File ./erpc_myproject/__init__.py:13
     11     version = "unknown"
     12 if version != "1.9.1":
---> 13     raise ValueError("The generated shim code version (1.9.1) is different to the rest of eRPC code (%s). \
     14 Install newer version by running \"python setup.py install\" in folder erpc/erpc_python/." % repr(version))
     16 from . import common
     17 from . import client

ValueError: The generated shim code version (1.9.1) is different to the rest of eRPC code ('unknown'). Install newer version by running "python setup.py install" in folder erpc/erpc_python/.

Solution:

Either you have not installed the erpc Python library (if the error message lists ... different to the rest of eRPC code ('unknown')) or you have installed the wrong version (e.g. ... (1.9.1) is different to the rest of eRPC code ('1.10.0')).

If you have not installed erpc at all, simply use

pip install erpc

and retry running your script.

If you have installed the wrong version, you have two options:

Option 1 (preferred): Re-generate the code

Just use the original command (some erpcgen call) you’ve used to re-generate the code using the currently installed version.

Option 2: Install the correct version

For this, you need to determine what the correct version is. Let’s consider the following error message:

ValueError: The generated shim code version (1.9.1) is different to the rest of eRPC code ('1.10.0'). Install newer version by running "python setup.py install" in folder erpc/erpc_python/.

From this message, we can read that the shim code version is 1.9.1 whereas you have 1.10.0 installed. Therefore, in order to make it work, we need to install erpc version 1.9.1.

Install it using

pip install -U erpc==1.9.1

and then retry your command. If you are using jupyter notebooks or similar, you need to restart your kernel to load the new library!

Posted by Uli Köhler in Embedded, Python

How to read length-prefixed binary message from Serial using Arduino

The following function allows you to read a binary message, prefixed by a single length byte, from Serial:

#include <Arduino.h>

void setup() {
    Serial.begin(115200);
}

void HandleMessage(String msg) {
    // TODO: Your code to handle the message goes here.
    // See https://techoverflow.net/2022/11/15/how-to-print-string-as-sequence-of-hex-bytes-in-arduino/
    // for an example of how to print the message as a sequence of hex bytes.
}

void ReadMessageFromSerial() {
    // Wait until the length byte is available on Serial
    while (Serial.available() == 0);

    // Read the length of the message
    int length = Serial.read();

    // Read the rest of the message
    String message = "";
    for (int i = 0; i < length; i++) {
      while (Serial.available() == 0);
      message += char(Serial.read());
    }

    // Handle the message
    HandleMessage(message);
}

void loop() {
    ReadMessageFromSerial();
}

 

 

Posted by Uli Köhler in Allgemein, Arduino, Electronics, Embedded

How to fix ESP32 error: ‘ESP_LOGE’ was not declared in this scope

Problem:

When trying compile your ESP32 project, you see an error message such as

.pio/libdeps/esp32dev/HumanESPHTTP/src/QueryURLParser.cpp:29:9: error: 'ESP_LOGE' was not declared in this scope
         ESP_LOGE("Query URL parser", "parsing URL");
         ^~~~~~~~

Solution:

At the top of the file where the error occurs (QueryURLParser.cpp in this example), add the following line:

#include <esp_log.h>

 

 

Posted by Uli Köhler in C/C++, ESP8266/ESP32

How to fix ESP32 Last error reported from esp-tls: 0x8008

Problem:

While trying to use TLS such as MQTTS or HTTPS on the ESP32, you see an error message like

E (333183) MQTT_CLIENT: mqtt_message_receive: transport_read() error: errno=119
[328153][E][MyMQTT.cpp:80] log_error_if_nonzero(): [MQTT] Last error reported from esp-tls: 0x8008
E (333191) MQTT_CLIENT: mqtt_process_receive: mqtt_message_receive() returned -1

Solution:

0x8008 means  ESP_ERR_ESP_TLS_TCP_CLOSED_FIN. In other words, a TCP connection had been established successfully but unexpectedly, the connection has been closed by the server.

This is often caused by the server software crashing, or restarting in some way. When a server process is terminated, the operating system will cleanup after it and close all connections.

In order to debug the issue, start by checking the log of your server message and/or system log to check for unintended crashes. If that doesn’t help, it’s sometimes helpful to packet capture the communication between the ESP32 and the server. You can also write a software script doing the same communication with the server as the ESP32. This will often allow you to try out changes much more easily than on the microcontroller and observe what’s happening using a debugger.

Posted by Uli Köhler in C/C++, Embedded, ESP8266/ESP32, Networking

How to fix ESP32 TRANSPORT_WS: Sec-WebSocket-Accept not found

Problem:

Your ESP32 running a MQTT client is printing the following error messages:

E (285025) TRANSPORT_WS: Sec-WebSocket-Accept not found
E (285025) MQTT_CLIENT: Error transport connect

Solution:

You’re using MQTT over websockets (ws:// or wss://) but on the given MQTT URL, no MQTT-over-websocket server is running.

This is often caused by using a wrong URL (possibly the URL is missing the path), but it might also be caused by a misconfiguraton of the server or the reverse proxy.

It’s often best to try using a software websocket client to test the correct settings.

Posted by Uli Köhler in ESP8266/ESP32, MQTT, Networking

How to fix ESP32 Last error reported from esp-tls: 0x8001

Problem:

While trying to use TLS such as MQTTS or HTTPS on the ESP32, you see an error message like

[MQTT] Last error reported from esp-tls: 0x8001

Solution:

0x8001 means ESP_ERR_ESP_TLS_CANNOT_RESOLVE_HOSTNAME. In other words, the ESP32 is unable to resolve the hostname of the host you’re trying to connect to using DNS.

Typically, this is a DNS problem, so check the DNS settings of your network. Also check if the ESP32 has a correct DNS server set – for example, if the ESP32 has 0.0.0.0 as a DNS server, this explains why it isn’t able to resolve the hostname.

Sometimes this issue is also caused by the hostname not existing at all (i.e. there is no DNS entry for that hostname). You can easily check this by resolving the hostname you’re trying to connect

Posted by Uli Köhler in C/C++, Embedded, ESP8266/ESP32, Networking

Which pins on the ESP32-S2 are DAC outputs?

Only two pins on the ESP32-S2 are capable of being DAC outputs:

  • GPIO23 is DAC_1
  • GPIO24 is DAC_2

Source: ESP32-S2 datasheet

Posted by Uli Köhler in ESP8266/ESP32

How to output 2.048MHz clock on any pin on the ESP32

You can use the LEDC timer (typically used for PWM) to output a 50% duty cycle clock with 3.3V P-P amplitude on any output-capable GPIO pin.

First,

#include <driver/ledc.h>

then setup the timer. You only need to do this once on startup, no code in your loop function is required.

/**
 * Setup 2.048MHz clock output on GPIO33
 */
ledc_timer_config_t ledc_timer = {
    .speed_mode = LEDC_HIGH_SPEED_MODE,
    .bit_num    = LEDC_TIMER_2_BIT,
    .timer_num  = LEDC_TIMER_0,
    .freq_hz    = 2048000
};
ledc_channel_config_t ledc_channel = {
    .gpio_num   = GPIO_NUM_33,
    .speed_mode = LEDC_HIGH_SPEED_MODE,
    .channel    = LEDC_CHANNEL_0,
    .timer_sel  = LEDC_TIMER_0,
    .duty       = 2
};
ledc_timer_config(&ledc_timer);
ledc_channel_config(&ledc_channel);

 

Posted by Uli Köhler in ESP8266/ESP32

How does the ESP32 DAC cosine generator waveform look on an Oscilloscope?

The ESP32 DAC has a built-in cosine waveform generator. Even though it’s an 8-bit DAC, the waveform looks pretty clean.

For an example on how to generate this wavefrm in firmware, see How to use the ESP32 DAC sine/cosine waveform generator using Arduino / PlatformIO

Posted by Uli Köhler in Electronics, ESP8266/ESP32

How to use the ESP32 DAC sine/cosine waveform generator using Arduino / PlatformIO

The ESP32 and its derivatives such as the ESP32-S2 have a built-in sine/cosine waveform generator for the built-in 8-bit DAC.

Using it requires ESP-IDF v5.1+ (see the official example). Using it with Arduino is slightly harder, since the stable version of the arduino-esp32 framework at the time of writing this post is based on ESP-IDF v4.4 which does not provide the DAC cosine generator API.

Therefore, we have to explicitly specify the arduino-espressif32 version (git commit) in platformio.ini:

[env:esp32dev]
platform = espressif32
# Commit f9cddfde697b659b9e818ec514f1505d2bd4a8ae is branch esp-idf-v5.1-libs @2022-02-01
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#f9cddfde697b659b9e818ec514f1505d2bd4a8ae
board = esp32dev
framework = arduino

The example main source code is pretty simple:

#include <Arduino.h>
#include <driver/dac_cosine.h>

void setup() {
    dac_cosine_handle_t chan0_handle;
    dac_cosine_config_t cos0_cfg = {
        .chan_id = DAC_CHAN_1, // GPIO26
        .freq_hz = 1000,
        .clk_src = DAC_COSINE_CLK_SRC_DEFAULT,
        .atten = DAC_COSINE_ATTEN_DEFAULT,  
        .phase = DAC_COSINE_PHASE_0,  
        .offset = 0,
        //.flags.force_set_freq = false,
    };
    ESP_ERROR_CHECK(dac_cosine_new_channel(&cos0_cfg, &chan0_handle));
    ESP_ERROR_CHECK(dac_cosine_start(chan0_handle));
}

void loop() {
  // put your main code here, to run repeatedly:
  delay(1000);
}

If you want to see how the generated waveform looks on an oscilloscope, see How does the ESP32 DAC cosine generator waveform look on an Oscilloscope?

Posted by Uli Köhler in Arduino, ESP8266/ESP32, PlatformIO

Minimal PlatformIO ESP32 ArduinoOTA example

Based on our Minimal PlatformIO ESP8266 ArduinoOTA example, this is a minimal starting point for your ESP32 program running ArduinoOTA.

#include <Arduino.h>
#include <WiFi.h>
#include <ArduinoOTA.h>


void setup() {
  Serial.begin(115200);
  /**
   * Connect to Wifi
   */
  WiFi.begin("MyWifi", "abc123abc");
  uint32_t notConnectedCounter = 0;
  while (WiFi.status() != WL_CONNECTED) {
      delay(100);
      Serial.println("Wifi connecting...");
      notConnectedCounter++;
      if(notConnectedCounter > 150) { // Reset board if not connected after 15s
          Serial.println("Resetting due to Wifi not connecting...");
          ESP.restart();
      }
  }
  Serial.print("Wifi connected, IP address: ");
  Serial.println(WiFi.localIP());
  /**
   * Enable OTA update
   */
  ArduinoOTA.begin();
}

void loop() {
  // Check for over the air update request and (if present) flash it
  ArduinoOTA.handle();
}

platformio.ini

You can leave your platformio.ini at default values, the only aspect you need to change is to set monitor_speed = 115200

[env:esp32dev]
board = esp32dev
platform = espressif32
framework = arduino
monitor_speed = 115200
Posted by Uli Köhler in ESP8266/ESP32, PlatformIO

STM32 HAL equivalent of Arduino millis()

The equivalent of Arduino’s millis() function when using the STM32 HAL is

HAL_GetTick()

The ticks occur once every millisecond, so this will also give you a millisecond timer that will overflow after some time equivalently to how millis() overflows.

Posted by Uli Köhler in Arduino, STM32

How to print WiFi MAC address to serial on ESP32 (Arduino)?

It’s as simple as

Serial.println(WiFi.macAddress());

Full example

#include <Arduino.h>

void setup() {
    Serial.begin(115200);
    Serial.println(WiFi.macAddress());
}

void loop() {
    // ...
}

 

Posted by Uli Köhler in Arduino, ESP8266/ESP32

How to get WiFi MAC address as binary on the ESP32 (Arduino)?

uint8_t mac[6];
WiFi.macAddress(mac);

 

Posted by Uli Köhler in Arduino, ESP8266/ESP32, Networking