How to capture & decode SNMP traffic using tcpdump

Using the following command, you can capture & decode SNMP traffic going in and out of a specific interface (enp3s0 in this example):

tcpdump -v -i enp3s0 'udp port 161'

This works by just filtering for UDP packets on port 161 – the SNMP port. The -v flag tells tcpdump to decode the traffic in real time.

Example output:

01:28:27.915505 IP (tos 0x0, ttl 64, id 28865, offset 0, flags [DF], proto UDP (17), length 151)
    10.9.2.199.snmp > 10.9.2.15.47669:  { SNMPv3 { F=a } { USM B=1 T=608 U="admin" } { ScopedPDU E=_80_00_6a_92_03_44_a5_6e_43_5e_4c C="" { GetResponse(33) R=1015374419  105.1.1.1.5.1.2=1 } } } 
01:28:27.915646 IP (tos 0x0, ttl 64, id 38290, offset 0, flags [DF], proto UDP (17), length 150)
    10.9.2.15.47669 > 10.9.2.199.snmp:  { SNMPv3 { F=ar } { USM B=1 T=608 U="admin" } { ScopedPDU E=_80_00_6a_92_03_44_a5_6e_43_5e_4c C="" { GetNextRequest(32) R=1015374420  105.1.1.1.5.1.2 } } } 

 

Posted by Uli Köhler in Networking

How to fix MikroTik RouterOS NTP client: using local clock

Problem:

in System -> NTP client in your MikroTik router, the Status always tells you using local clock and it won’t synchronize with any NTP server.

Solution:

You can’t disable the local clock in System -> NTP client. Instead, you’ll be able to find the relevant setting in System -> NTP server.

Ensure that Use Local Clock is unchecked and click Apply

 

After that, your NTP client will be active and able to synchronize:

 

Posted by Uli Köhler in MikroTik, Networking

How to fix MikroTik RouterOS DoH server connection error: SSL: ssl: certificate not yet valid (6)

Problem:

The DNS server integrated into your MikroTik router doesn’t work and the log shows a lot of

DoH server connection error: SSL: ssl: certificate not yet valid (6)

messages:

Reason for the error:

The issue here is that the clock in your MikroTik router does not (yet) know the correct time.

For example, the clock might be set to 1st of January, 1970 – however, the TLS certificate of the DNS-over-HTTPS server is only valid from, for example, 1st of November, 2022. This is why the MikroTik router tells you that the certificate isn’t valid.

Preferred solution: Fix the time using NTP

Just tell the MikroTik server to get the time from a public NTP server.

Open System -> NTP client in WebFig or Winbox. Typically, you want to use the upstream router as an NTP server. In my case, that is 192.168.178.1.

Ensure that Enabled is checked, add the NTP server and click Apply.

After waiting a few seconds, you should see synchronized under Status. This means that the clock of the MikroTik router has been set correctly and the issue should be fixed.

Alternate solution: Disable DNS-over-HTTPs

This solution decreases the security of your system and is hence not preferred. You should always set the time of your router correctly, not doing so will lead to a bunch of issues.

If you, however, still intend to disable DNS-over-HTTPS, open IP -> DNS and remove all servers under Use DoH servers, then click Apply.

After that, your router will use the normal DNS servers – 1.1.1.1 in my case. Ensure to enter some server there to make sure DNS requests work – if in doubt, you can always use 1.1.1.1 (Cloudflare) or 8.8.8.8 (Google).

Note that requests to those servers will neither be encrypted nor authenticated, so requests can be sniffed and/or manipulated by anyone capable of manipulating traffic to your device. Even though DNS-over-HTTPS is slighly slower (which, in turn is alleviated by the caching feature of the MikroTik router’s DNS server), it provides a huge security benefit.

Posted by Uli Köhler in MikroTik, Networking

How to install additional SNMP MIBs in Ubuntu

First, download the standard set of MIBs using

sudo apt -y install snmp-mibs-downloader

This will download all available MIBS files during package installation but if at any time you want to download/update them, use

sudo download-mibs

After that, you’ll need to activate loading the MIBS for the SNMP client (original source for this command: @CameronSparr on medium.com):

sudo sed -i 's/mibs :/# mibs :/g' /etc/snmp/snmp.conf

 

Posted by Uli Köhler in Networking, SNMP

Simple SNMPv3 client example for Netgear GS710TUP

First, configure your Netgear GS710TUP to enable SHA authentication (instead of the less secure MD5 authentication) by opening System -> SNMP -> SNMPv3 , clicking on SHA and clicking Save:

Given that configuration, you can query information such as the switch uptime (which is represented by OID 1.3.6.1.2.1.1.3.0) using snmpwalk:

snmpwalk -v3 -l authNopriv -c public -a SHA1 -u admin -A 'switchAdminPassword' SWITCHIPADDRESS 1.3.6.1.2.1.1.3.0

Remember to replace switchAdminPassword by the admin password of your switch and SWITCHIPADDRESS by the IP address of the switch.

Posted by Uli Köhler in Networking, SNMP

How to git submodule init & git submodule update in a single command

Instead of separately running

git submodule init
git submodule update

you can just

git submodule update --init

In many cases, you also want to git submodule update recursively using:

git submodule update --init --recursive

 

Posted by Uli Köhler in git

Automated script to build MicroPython for the ESP32

The following script installs esp-idf, the required dependencies and then builds MicroPython for the ESP32.

apt -y update && apt -y install python3 python3-pip cmake
# Install esp-idf
git clone --depth 1 -b v4.4.3 https://github.com/espressif/esp-idf.git /opt/esp-idf
cd /opt/esp-idf && git submodule update --init --recursive && ./install.sh

# Clone micropython
git clone --depth 1 -b v1.19.1 https://github.com/micropython/micropython.git /opt/micropython

# Build micropython
cd /opt/micropython/ && make -C mpy-cross
# Build micropython@esp32
cd /opt/micropython/ports/esp32 && source /opt/esp-idf/export.sh && make submodules && make -j4

 

Posted by Uli Köhler in Embedded, ESP8266/ESP32, MicroPython

How to get external IPv4 address using curl

Run the following command to print your external IP address:

curl -fsSL https://api4.ipify.org

Example output:

80.187.71.195

 

Posted by Uli Köhler in Networking

How to get external IPv4 address using wget

Run the following command to print your external IP address:

wget -qO- https://api4.ipify.org

Example output:

80.187.71.195

 

Posted by Uli Köhler in Networking

How to check if system is Windows or Linux using Python

def is_platform_windows()
    return platform.system() == "Windows"

def is_platform_linux():
    return platform.system() == "Linux"

 

Posted by Uli Köhler in Python

How to fix Python msgpack TypeError: write() argument must be str, not bytes

Problem:

While trying to write an object to a file using msgpack using code like

with open("myobj.msgpack", "w") as outfile:
    msgpack.dump(myobj, outfile)

you see an error message like

File /usr/lib/python3/dist-packages/msgpack/__init__.py:26, in pack(o, stream, **kwargs)
     20 """
     21 Pack object `o` and write it to `stream`
     22 
     23 See :class:`Packer` for options.
     24 """
     25 packer = Packer(**kwargs)
---> 26 stream.write(packer.pack(o))

TypeError: write() argument must be str, not bytes

Solution:

msgpack writes binary data, so you need to open the file in binary mode using open(..., "wb"):

with open("myobj.msgpack", "wb") as outfile:
    msgpack.dump(myobj, outfile)

 

Posted by Uli Köhler in Python

How to reload “from mymodule import *” wildcard imports in Jupyter

In our previous post How to reload import in Jupyter we showed how to use reload() from importlib in order to reload a module without restarting the kernel.

import mymodule
# Reload .py file every time we run the cell
from importlib import reload
reload(mymodule)

However, often we are using syntax like

from mymodule import *

to wildcard-load everything imported from mymodule.

We can reload those wildcard imports using the same strategy by additionalling importing mymodule and then reloading using reload(mymodule) – this will also reload the wildcard import from mymodule import *!

from mymodule import *

# This line is just to facilitate reloading
import mymodule
from importlib import reload
reload(mymodule)
# Reload .py file every time we run the cell from importlib import reload reload(mymodule)

 

Posted by Uli Köhler in Python

How to add N months to date using arrow in Python

If you have a date like

import arrow

dt = arrow.get("2021-05-27")

you can add N months (example: 18 months) to said date using

dt.shift(months=+18)

The result will be an arrow representation of 2022-11-27

Posted by Uli Köhler in Python

How to fix lxc launch Failed getting root disk: No root device could be found

Problem:

While trying to launch a lxc container using a command like

lxc launch ubuntu:22.04 mycontainer

you see the following error message:

Creating mycontainer
Error: Failed instance creation: Failed creating instance record: Failed initialising instance: Failed getting root disk: No root device could be found

Solution:

You didn’t initialize your LXD storage properly. Run

lxd init

in order to configure the storage for lxd. For most setups except performance-critical production setups, I recommend to use the dir storage backend because it does not require any further configuration. You can leave all other options at their default values.

Name of the storage backend to use (zfs, btrfs, ceph, cephobject, dir, lvm) [default=zfs]: dir
Posted by Uli Köhler in Container, LXC

ESP32 Arduino HTTPS webserver example using certificate from LittleFS

The following example reads a HTTPS/TLS certificate from LittleFS (cert.pem and privkey.pem – these are just standard X.509 PEM files) and uses those to initialize a HTTPS server. Most of this example is based on the esp-idf https_server example.

Our code to initialize the HTTPS server uses code from the following posts:

The core initialization sequence works like this:

// These need to be GLOBAL variables, they still need to exist
// after initialization!
std::string cert;
std::string privkey;

httpd_ssl_config_t conf = HTTPD_SSL_CONFIG_DEFAULT();
httpd_handle_t server = nullptr;

void InitializeHTTPServer() {
        std::string cert = ReadFileToString("/cert.pem");
        std::string privkey = ReadFileToString("/privkey.pem");

        conf.cacert_pem = (const uint8_t*)this->cert.c_str();
        conf.cacert_len = this->cert.size() + 1;

        conf.prvtkey_pem = (const uint8_t*)this->privkey.c_str();
        conf.prvtkey_len = this->privkey.size() + 1;

        esp_err_t ret = httpd_ssl_start(&server, &conf);
        if (ESP_OK != ret) {
            ESP_LOGI(TAG, "Error starting server!");
            return;
        }
}

However it is typically far more convenient to use our HTTPSServer class:

class HTTPSServer {
public:

    HTTPSServer(): conf(HTTPD_SSL_CONFIG_DEFAULT()) {
    }

    void ReadCertificateInfoFromFilesystem() {
        cert = ReadFileToString("/cert.pem");
        privkey = ReadFileToString("/privkey.pem");
    }

    void StartServer() {
        // Start the httpd server
        ESP_LOGI(TAG, "Starting server");

        ReadCertificateInfoFromFilesystem();

        conf.cacert_pem = (const uint8_t*)this->cert.c_str();
        conf.cacert_len = this->cert.size() + 1;

        conf.prvtkey_pem = (const uint8_t*)this->privkey.c_str();
        conf.prvtkey_len = this->privkey.size() + 1;

        esp_err_t ret = httpd_ssl_start(&server, &conf);
        if (ESP_OK != ret) {
            ESP_LOGI(TAG, "Error starting server!");
            return;
        }
    }

    void RegisterHandler(const httpd_uri_t *uri_handler) {
        httpd_register_uri_handler(this->server, uri_handler);
    }

    httpd_handle_t server = nullptr;

    httpd_ssl_config_t conf;
    std::string cert;
    std::string privkey;
};

Usage example:

void setup() {
    InitFilesystem();

    // TODO setup wifi network

    httpsServer.StartServer();
    // Register your 
    httpsServer.RegisterHandler(&root);
}

Full example:

#include <Arduino.h>
#include <WiFi.h>
#include <LittleFS.h>
#include <string>
#include <esp_https_server.h>
#include "esp_tls.h"

//AsyncWebServer server(80);
static const char *TAG = "https-littlefs-example";

volatile bool filesystemOK = false;

void InitFilesystem() {
  // Initialize LittleFS
  if (!LittleFS.begin(false /* false: Do not format if mount failed */)) {
    Serial.println("Failed to mount LittleFS");
    if (!LittleFS.begin(true /* true: format */)) {
      Serial.println("Failed to format LittleFS");
    } else {
      Serial.println("LittleFS formatted successfully");
      filesystemOK = true;
    }
  } else { // Initial mount success
    filesystemOK = true;
  }
}

/* An HTTP GET handler */
static esp_err_t root_get_handler(httpd_req_t *req)
{
    httpd_resp_set_type(req, "text/html");
    httpd_resp_send(req, "<h1>Hello Secure World!</h1>", HTTPD_RESP_USE_STRLEN);

    return ESP_OK;
}

static const httpd_uri_t root = {
    .uri       = "/",
    .method    = HTTP_GET,
    .handler   = root_get_handler
};

size_t LittleFSFilesize(const char* filename) {
  auto file = LittleFS.open(filename, "r");
  size_t filesize = file.size();
  // Don't forget to clean up!
  file.close();
  return filesize;
}

std::string ReadFileToString(const char* filename) {
  auto file = LittleFS.open(filename, "r");
  size_t filesize = file.size();
  // Read into temporary Arduino String
  String data = file.readString();
  // Don't forget to clean up!
  file.close();
  return std::string(data.c_str(), data.length());
}

class HTTPSServer {
public:

    HTTPSServer(): conf(HTTPD_SSL_CONFIG_DEFAULT()) {
    }

    void ReadCertificateInfoFromFilesystem() {
        cert = ReadFileToString("/cert.pem");
        privkey = ReadFileToString("/privkey.pem");
    }

    void StartServer() {
        // Start the httpd server
        ESP_LOGI(TAG, "Starting server");

        ReadCertificateInfoFromFilesystem();

        conf.cacert_pem = (const uint8_t*)this->cert.c_str();
        conf.cacert_len = this->cert.size() + 1;

        conf.prvtkey_pem = (const uint8_t*)this->privkey.c_str();
        conf.prvtkey_len = this->privkey.size() + 1;

        esp_err_t ret = httpd_ssl_start(&server, &conf);
        if (ESP_OK != ret) {
            ESP_LOGI(TAG, "Error starting server!");
            return;
        }

        // Set URI handlers
        ESP_LOGI(TAG, "Registering URI handlers");
        httpd_register_uri_handler(server, &root);
        return;
    }

    void RegisterHandler(const httpd_uri_t *uri_handler) {
        httpd_register_uri_handler(this->server, uri_handler);
    }

    httpd_handle_t server = nullptr;

    httpd_ssl_config_t conf;
    std::string cert;
    std::string privkey;
};

HTTPSServer httpsServer;

void setup() {
    Serial.begin(115200);
    delay(500);

    InitFilesystem();

    // Connect Wifi, restart if not connecting
    // https://techoverflow.net/2021/01/21/how-to-fix-esp32-not-connecting-to-the-wifi-network/
    WiFi.begin("MyWifi", "Lobae1tie5achokef8riequohgohx");

    uint32_t notConnectedCounter = 0;
    while (WiFi.status() != WL_CONNECTED) {
        delay(100);
        Serial.println("Wifi connecting...");
        notConnectedCounter++;
        if(notConnectedCounter > 150) { // Reset board if not connected after 5s
            Serial.println("Resetting due to Wifi not connecting...");
            ESP.restart();
        }
    }
    Serial.print("Wifi connected, IP address: ");
    Serial.println(WiFi.localIP());

    httpsServer.StartServer();
    httpsServer.RegisterHandler(&root);
}

void loop() {
  // put your main code here, to run repeatedly:
  delay(1000);
}
[env:esp32dev]
platform = espressif32
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.5
board = esp32dev
framework = arduino
board_build.filesystem = littlefs
monitor_speed = 115200

 

Posted by Uli Köhler in Arduino, ESP8266/ESP32, Networking, PlatformIO

How to read LittleFS file to std::string (using Arduino/PlatformIO)

The following utility function reads . As a prerequisite, you need to initialize the LittleFS filesystem and configure PlatformIO to use LittleFS as filesystem when uploading the filesystem image.

NoteBefore using any of the functions below, you need to call InitFilesystem() in setup() in order to mount the filesystem. You can find this function in our previous post on how to initialize LittleFS.

#include <string>

std::string ReadFileToString(const char* filename) {
  auto file = LittleFS.open(filename, "r");
  size_t filesize = file.size();
  // Read into temporary Arduino String
  String data = file.readString();
  // Don't forget to clean up!
  file.close();
  return std::string(data.c_str(), data.length());
}

Example usage:

std::string cert = ReadFileToString("/cert.pem");

 

Posted by Uli Köhler in C/C++, ESP8266/ESP32, PlatformIO

How to configure PlatformIO for LittleFS (ESP32)

You can configure PlatformIO to use the LittleFS filesystem as opposed to the default SPIFFS using

board_build.filesystem = littlefs

If you are ussing arduino-esp32, you might also need to specify to use a recent arduino-esp32 framework version, but I suggest to first try it out without using that option:

platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.5

Full platformio.ini example

[env:esp32dev]
platform = espressif32
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.5
board = esp32dev
framework = arduino
board_build.filesystem = littlefs
monitor_speed = 115200

 

Posted by Uli Köhler in ESP8266/ESP32, PlatformIO

How to fix esp-tls-mbedtls: mbedtls_x509_crt_parse returned -0x2180

Problem:

While trying to use TLS on the ESP32, you are using a certificate and private key e.g. from NVS or from the filesystem. However when you try to connect using SSL, you see error messages like

E (9774) esp-tls-mbedtls: mbedtls_x509_crt_parse returned -0x2180
E (9775) esp-tls-mbedtls: Failed to set server pki context
E (9775) esp-tls-mbedtls: Failed to set server configurations, returned [0x8015] (ESP_ERR_MBEDTLS_X509_CRT_PARSE_FAILED)
E (9786) esp-tls-mbedtls: create_ssl_handle failed, returned [0x8015] (ESP_ERR_MBEDTLS_X509_CRT_PARSE_FAILED)
E (9795) esp_https_server: esp_tls_create_server_session failed

Solution:

-0x2180 means MBEDTLS_ERR_X509_INVALID_FORMAT

The most common issue here is that conf.cacert_len and conf.prvtkey_len must include the NUL terminator in the length.

Working Example:

conf.cacert_pem = (const uint8_t*)cert.c_str();
conf.cacert_len = this->cert.size() + 1;

conf.prvtkey_pem = (const uint8_t*)privkey.c_str();
conf.prvtkey_len = this->privkey.size() + 1;

Note the + 1 here: Without the + 1, you’ll see the mbedtls_x509_crt_parse returned -0x2180

Other causes:

If the length isn’t the issue, you likely have a malformed certificate. I suggest to print out the certificate’s content via the serial port, saving it to file we’ll call cert_esp32.pem and then running

openssl x509 -in cert_esp32.pem -noout -text

to verify that the certificate is correct. You can do the same for the private key, but typically either both the private key and the certificate are fine or both are not.

Posted by Uli Köhler in C/C++, Embedded, ESP8266/ESP32, Networking

How to initialize LittleFS in Arduino on the ESP32 (PlatformIO)

In order to use the LittleFS library that comes with Arduino on the ESP32, use the following function

#include <LittleFS.h>

bool filesystemOK = false;

void InitFilesystem() {
  // Initialize LittleFS
  if (!LittleFS.begin(false /* false: Do not format if mount failed */)) {
    Serial.println("Failed to mount LittleFS");
    if (!LittleFS.begin(true /* true: format */)) {
      Serial.println("Failed to format LittleFS");
    } else {
      Serial.println("LittleFS formatted successfully");
      filesystemOK = true;
    }
  } else { // Initial mount success
    filesystemOK = true;
  }
}

Additionally, you need to configure PlatformIO to use LittleFS:

board_build.filesystem = littlefs

Now, in setup(), call

InitFilesystem();

If you have errors including LittleFS.h, you might need to explicitly use a recent arduino-esp32 version in platformio.ini:

platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.5

In case you absolutely need to use an older arduino-esp32 version that doesn’t support LittleFS out of the box, you can use the lorol/LittleFS library as outlined in our previous post: How to initialize LittleFS in PlatformIO on the ESP32 using the lorol/LittleFS library

Full platformio.ini example:

[env:esp32dev]
platform = espressif32
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.5
board = esp32dev
framework = arduino
board_build.filesystem = littlefs
monitor_speed = 115200

 

Posted by Uli Köhler in Arduino, Embedded, ESP8266/ESP32, PlatformIO

Does ESP32 (ESP-IDF/Arduino/mbed-tls) support secp384r1 certificates?

Yes, I verified that mbed-tls (which is used in the esp-idf framework, which is in turn used by Arduino/PlatformIO) works fine when using a secp384r1-keyed Let’s encrypt TLS server certificate.

The version used in arduino-esp32 v2.0.5 with PlatformIO 6.1.5.

Posted by Uli Köhler in ESP8266/ESP32, mbed
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Cookie settingsACCEPTPrivacy &amp; Cookies Policy