C/C++

How to copy boost::urls::url in C++ (Boost.Url)

You can copy a boost::urls::url by using it copy constructor:

boost::urls::url url1 = boost::urls::url::parse("https://example.com/path1");
boost::urls::url url2(url1); // Copy url1 to url2 using the copy constructor

 

Posted by Uli Köhler in Boost, C/C++

How to access JSON nested value in C++ using Boost.JSON

#include <boost/json.hpp>

boost::json::value j = /* ... */;
// Access nested value with key "value"
j.at("value").as_string();

Full example:

#include <boost/json.hpp>
#include <iostream>

int main() {
    // Hardcoded JSON string
    const std::string json_str = R"(
        {
            "key": "1234",
            "value": "Hello, world!"
        }
    )";

    // Parse the JSON string
    boost::json::value j = boost::json::parse(json_str);

    // Access the "value" property and print it to stdout
    std::string value = j.at("value").as_string().c_str();
    std::cout << value << std::endl; // Prints "Hello, world!"

    return 0;
}

Compile using

g++ -o json json.cpp -lboost_json
Posted by Uli Köhler in Boost, C/C++

How to parse JSON from std::string in C++ using Boost.JSON

#include <boost/json.hpp>

boost::json::value j = boost::json::parse(json_str);

Full example:

#include <boost/json.hpp>
#include <iostream>

int main() {
    // Hardcoded JSON string
    const std::string json_str = R"(
        {
            "key": "1234",
            "value": "Hello, world!"
        }
    )";

    // Parse the JSON string
    boost::json::value j = boost::json::parse(json_str);

    // Access the "value" property and print it to stdout
    std::string value = j.at("value").as_string().c_str();
    std::cout << value << std::endl; // Prints "Hello, world!"

    return 0;
}

Compile using

g++ -o json json.cpp -lboost_json
Posted by Uli Köhler in Boost, C/C++

How to encode URL query parameters in C++ using Boost.URL

#include <iostream>
#include <boost/url.hpp>

using namespace boost::urls;

int main() {
    std::string username = "myusername";
    std::string password = "mypassword";
    url_view base_url = "https://example.com/api/login";

    url encoded_url(base_url);
    encoded_url.params().set("user name", username); // With space to test proper encoding
    encoded_url.params().set("password", password);

    // Prints "https://example.com/api/login?username=my%20user%20name&password=mypassword"
    std::cout << "Encoded URL: " << encoded_url << std::endl;

    return 0;
}

Compile using

g++ -o urlhost urlhost.cpp -lboost_url

 

Posted by Uli Köhler in Boost, C/C++

How to parse host from URL in C++ using Boost::URL

If you have an URL such as:

std::string myURL = "https://subdomain.example.com/api/test";

you can parse the hostname from it using Boost::URL (you need at least Boost 1.81.0, previous versions of boost don’t have Boost.URL) using

boost::urls::url_view url(myURL);
std::string host = url.host();

std::cout << host << std::endl; // Prints "subdomain.example.com"

Full example:

#include <string>
#include <iostream>

#include <boost/url.hpp>

int main() {
    std::string myURL = "https://subdomain.example.com/api/test";

    boost::urls::url_view url(myURL);
    std::string host = url.host();

    std::cout << host << std::endl; // Prints "subdomain.example.com"
}

Compile using

g++ -o urlhost urlhost.cpp -lboost_url

 

Posted by Uli Köhler in Boost, C/C++

How to fix GCC fatal error: parquet/arrow/writer.h: No such file or directory

Problem:

While compiling your C++ application, you see an error message like

src/main.cpp:6:10: fatal error: parquet/arrow/writer.h: No such file or directory
    6 | #include <parquet/arrow/writer.h>

Solution:

You need to install the Parquet C++ libraries which are shipped together with the Arrow libraries.

On Ubuntu, you can do that using

sudo apt update
sudo apt install -y -V ca-certificates lsb-release wget
wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
sudo apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
sudo apt update -y
sudo apt -y install libparquet-dev

For other operating systems, see the official Arrow installation guide,

Posted by Uli Köhler in C/C++, GCC errors

How to fix GCC fatal error: arrow/api.h: No such file or directory on Ubuntu

Problem:

While compiling your C++ application, you see an error message like

src/main.cpp:3:10: fatal error: arrow/api.h: No such file or directory
    3 | #include <arrow/api.h>

Solution:

You need to install the Arrow C++ libraries.

On Ubuntu, you can do that using

sudo apt update
sudo apt install -y -V ca-certificates lsb-release wget
wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
sudo apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
sudo apt update -y
sudo apt -y install libarrow-dev

For other operating systems, see the official Arrow installation guide,

Posted by Uli Köhler in C/C++, GCC errors

How to force GCC/G++ to provide colored output

Just add the

-fdiagnostics-color=always

compiler flag

Posted by Uli Köhler in C/C++

Minimal C++ SLIP encoder operating on buffers

The following encoder can be used to encode frames into a byte stream. It operates on complete buffers (i.e. this is not a streaming encoder) but is fast and easy to use.

/*
 * Author: Uli Köhler - techoverflow.net
 * License: CC0 1.0 Universal
 *
 * Description:
 * This is a C++ implementation of a buffer-based SLIP encoder.
 * It can be used to encode frames unambiguously into a stream of bytes.
 * The encoder does not use any memory allocation, but instead requires
 * the caller to provide a buffer for the encoded data - hence being
 * ideally suited for use in embedded systems
 */
#pragma once
#include <cstdint>
#include <cstddef>

#define SLIP_END             ((uint8_t)0300)   // SLIP end of packet character
#define SLIP_ESC             ((uint8_t)0333)   // SLIP escape character
#define SLIP_ESCEND         ((uint8_t)0334)    // Escaped END data byte
#define SLIP_ESCESC         ((uint8_t)0335)    // Escaped ESC data byte

// Return value of slip_encode_packet if the output buffer is not large enough
#define SLIP_ENCODE_ERROR SIZE_MAX // Maximum value of size_t

/**
 * Determine the packet length when sending the given buffer [p] using SLIP.
 * This DOES NOT ENCODE ANYTHING, it just determines the length
 * which an encoded version of the data would have
 * @return The number of bytes a SLIP encoded version of [in] would consume
 */
size_t slip_packet_length(const uint8_t* in, size_t inlen);

/**
 * Encode given input data using SLIP, saving it into the given output buffer.
 * WARNING! The output buffer MUST have a length of at least the return value of
 *    slip_packet_len(in, inlen)
 * In case the output buffer is not large enough, this function will return SLIP_ENCODE_ERROR and
 * the output buffer will be left in an undefined space.
 * Take special care that the input data does not change between the calls to
 * slip_packet_len() and slip_encode_packet()
 * @return The number of bytes in [out], or SLIP_ENCODE_ERROR
 */
size_t slip_encode_packet(const uint8_t* in, size_t inlen, uint8_t* out, size_t outlen);
/*
 * Author: Uli Köhler - techoverflow.net
 * License: CC0 1.0 Universal
 *
 * Description:
 * This is a C++ implementation of a buffer-based SLIP encoder.
 * It can be used to encode frames unambiguously into a stream of bytes.
 * The encoder does not use any memory allocation, but instead requires
 * the caller to provide a buffer for the encoded data - hence being
 * ideally suited for use in embedded systems
 */
#include "SLIPEncoder.hpp"

size_t slip_packet_length(const uint8_t* in, size_t inlen) {
    // This variable contains the length of the data
    size_t outlen = 0;
    const uint8_t* inend = in + inlen; // First character AFTER the
    for (; in < inend ; in++) {
        switch (*in) {
        case SLIP_END: // Need to escape END character to avoid RX seeing end of frame
            // Same as "case SLIP_ESC" so just continue.
        case SLIP_ESC: // Need to escape ESC character, we'll send
            outlen += 2; // Will send ESC + ESCESC
            break;
        default: // Any other character => will just copy
            outlen++; // Will only send the given character
        }
    }
    // + 1: SLIP END bytes
    return outlen + 1;
}

size_t slip_encode_packet(const uint8_t* in, size_t inlen, uint8_t* out, size_t outlen) {
    // This variable contains the length of the data
    uint8_t* out_start = out; // We will increment [out], hence copy the original value.

    // Output buffer must be AT LEAST as long as input data (sanity check)
    if(outlen < inlen) {
        return SLIP_ENCODE_ERROR;
    }

    uint8_t* outend = out + outlen; // First character AFTER the
    const uint8_t* inend = in + inlen; // First character AFTER the
    for (; in < inend ; in++) {
        switch (*in) {
        case SLIP_END: // Need to escape END character to avoid RX seeing end of frame
            // Check out of bounds memory acces
            if(out + 2 >= outend) {
                return SLIP_ENCODE_ERROR;
            }
            // Copy escaped END character
            *out++ = SLIP_ESC;
            *out++ = SLIP_ESCEND;
            break;
        case SLIP_ESC: // Need to escape ESC character, we'll send
            // Check out of bounds memory access
            if(out + 2 >= outend) {
                return SLIP_ENCODE_ERROR;
            }
            // Copy escaped END character
            *out++ = SLIP_ESC;
            *out++ = SLIP_ESCESC;
            break;
        default: // Any other character => just copy the character
            if(out + 1 >= outend) {
                return SLIP_ENCODE_ERROR;
            }
            *out++ = *in;
        }
    }
    // Check out of bounds access for END byte
    if(out + 1 > outend) { // NOTE: > instead of >= since there is only ONE character to be written
        return SLIP_ENCODE_ERROR;
    }
    // Insert END byte
    *out++ = SLIP_END;
    // Return number of bytes
    return (out - out_start);
}

 

 

Posted by Uli Köhler in C/C++

How to fix C++ error: invalid use of template-name ‘std::chrono::time_point’ without an argument list

Problem:

You are trying to use std::chrono::time_point in your C++ code, but the compiler throws an error message like

MyClass.hpp:58:5: error: invalid use of template-name ‘std::chrono::time_point’ without an argument list
   58 |     std::chrono::time_point t0;

Solution:

std::chrono::time_point is a template that requires two template arguments: the clock and the duration. The duration argument, however, defaults to Clock::duration so you only have to explicitly specify the clock.

Typically you can just use std::chrono::system_clock:

std::chrono::time_point<std::chrono::system_clock> t0;

 

Posted by Uli Köhler in C/C++, GCC errors

How to generate filename based on current date & time in C++ using chrono

Problem:

You want to generate a filename, e.g., for datalogging, that contains the current date and time in C++ – for example:

2023-04-22_19-38-34.csv

Solution:

To generate a filename with the current date and time in C++, you can use the chrono library to get the current time and format it as a string. Here’s a code snippet that shows how to do this:

#include <chrono>
#include <iomanip>
#include <sstream>

std::string GetFilenameByCurrentDate(const char* extension=".bin") {
    // Get the current time
    auto now = std::chrono::system_clock::now();

    // Convert to local time
    auto localTime = std::chrono::system_clock::to_time_t(now);

    // Format the timestamp as a string
    std::stringstream ss;
    ss << std::put_time(std::localtime(&localTime), "%F_%H-%M-%S") << extension;
    return ss.str();
}

 

This function takes an optional extension parameter that specifies the file extension to use (by default, “.bin”). It uses the chrono library to get the current time (now), convert it to local time (localTime), and format it as a string (ss). The put_time function is used to format the time as a string in the format “%F_%H-%M-%S”, which represents the date and time in ISO 8601 format (e.g., 2023-04-23_19-13-22).

This will produce a filename such as 2023-04-23_19-13-22.bin, which can be used to create a new file or to store data in an existing file. By including a timestamp in the filename, you can easily keep track of when the file was created and quickly identify files that were created at a particular time.

Posted by Uli Köhler in C/C++

How to fix C++ error: no member named ‘put_time’ in namespace ‘std’

Problem:

You are trying to use the std::put_time function in your C++ application, but you get the following compiler error message:

main.cpp:17:16: error: no member named 'put_time' in namespace 'std'
    ss << std::put_time(std::localtime(&localTime), "%F_%H-%M-%S") << extension;
          ~~~~~^

Solution:

The std::put_time function is part of the <iomanip> header in C++, so you need to include this header in your code to use this function. Add the following code at the top of the file where the error occurs:

#include <iomanip>

Full example:

#include <iostream>
#include <iomanip>
#include <chrono>

int main() {
    auto now = std::chrono::system_clock::now();
    auto localTime = std::chrono::system_clock::to_time_t(now);
    std::cout << std::put_time(std::localtime(&localTime), "%F %T") << std::endl;
    return 0;
}

 

Posted by Uli Köhler in C/C++, GCC errors

How to fix GCC error: implicit instantiation of undefined template ‘std::basic_stringstream<char>’

Problem:

When trying to compile your C++ application, you see an error message such as

main.cpp:15:23: error: implicit instantiation of undefined template 'std::basic_stringstream<char>'
    std::stringstream ss;
                      ^

Solution:

At the top of the file where this error occurs, add the following line:

#include <sstream>

 

Posted by Uli Köhler in C/C++, GCC errors

How to construct C++ chrono::time_point from a milliseconds-since-epoch timestamp

#include <chrono>

size_t myMillisecondsSinceEpochTimestamp = 1680978404374;
std::chrono::time_point<std::chrono::system_clock> myChronoTimestamp(
    std::chrono::milliseconds(myMillisecondsSinceEpochTimestamp)
);

 

Posted by Uli Köhler in C/C++

heatshrink compression library: How to compress data from a buffer

This is a starting point for how to compress data from a buffer using heatshrink and write the data into an output buffer:

Note that this has not been thoroughly tested, but at least it doesn’t crash 🙂

// TODO Your code goes here
const size_t outbuf_size = filesize + 128;
char* outbuf = new char[outbuf_size];
heatshrink_encoder* encoder = heatshrink_encoder_alloc(10, 4);
size_t tosink = filesize;
size_t output_size = 0;
while(tosink > 0) {
    size_t sunk = 0;
    auto err = heatshrink_encoder_sink(encoder, reinterpret_cast<uint8_t*>(buf), (size_t)filesize, &sunk);
    if(err != HSER_SINK_OK) {
        std::cerr << "Error sinking data" << std::endl;
        break;
    }
    if(sunk == 0) {
        std::cerr << "No data sunk" << std::endl;
        break;
    }
    // Check how much has been sunk & update tosink
    tosink -= sunk;
    // Poll for output
    size_t polled = 0;
    auto err2 = heatshrink_encoder_poll(encoder, reinterpret_cast<uint8_t*>(outbuf + output_size), outbuf_size - output_size, &polled);
    if(err2 == HSER_POLL_ERROR_NULL || err2 == HSER_POLL_ERROR_MISUSE) {
        std::cerr << "Error polling data" << std::endl;
        break;
    }
    output_size += polled;
}
// Input data finished
auto err3 = heatshrink_encoder_finish(encoder);
// Poll for final output
// Poll for output
size_t polled = 0;
auto err2 = heatshrink_encoder_poll(encoder, reinterpret_cast<uint8_t*>(outbuf + output_size), outbuf_size - output_size, &polled);
if(err2 == HSER_POLL_ERROR_NULL || err2 == HSER_POLL_ERROR_MISUSE) {
    std::cerr << "Error finally polling data" << std::endl;
}
output_size += polled;

cout << "Original size: " << filesize << ", compressed size: " << output_size << endl;

 

Posted by Uli Köhler in C/C++, Embedded

How to read C++ binary file into a buffer (C++17 way)

#include <iostream>
#include <fstream>
#include <filesystem>

// Get size of file to know how much memory to allocate
std::uintmax_t filesize = std::filesystem::file_size("C037B221110.bin");

// Allocate buffer to hold file
char* buf = new char[filesize];
// Read file
std::ifstream fin("C037B221110.bin", std::ios::binary);
fin.read(buf, filesize);
if(!fin) {
    std::cerr << "Error reading file, could only read " << fin.gcount() << " bytes" << std::endl;
}
// Close file
fin.close();

 

Posted by Uli Köhler in C/C++

How to get filesize in C++ using C++17’s std::filesystem

Recent versions of C++ have a built-in cross-platform filesystem library which supports many useful operations such as getting the filesize.

#include <filesystem>

// Usage example:
std::filesystem::file_size("myfile.bin");

Full example:

#include <filesystem>
#include <iostream>

int main() {
    std::uintmax_t filesize = std::filesystem::file_size("myfile.bin");
    std::cout << "Filesize in bytes is " << filesize << std::endl;
    return 0;
}

 

Posted by Uli Köhler in C/C++

How to fix ESP32 error: ‘ESP_LOGE’ was not declared in this scope

Problem:

When trying compile your ESP32 project, you see an error message such as

.pio/libdeps/esp32dev/HumanESPHTTP/src/QueryURLParser.cpp:29:9: error: 'ESP_LOGE' was not declared in this scope
         ESP_LOGE("Query URL parser", "parsing URL");
         ^~~~~~~~

Solution:

At the top of the file where the error occurs (QueryURLParser.cpp in this example), add the following line:

#include <esp_log.h>

 

 

Posted by Uli Köhler in C/C++, ESP8266/ESP32

How to fix ESP32 Last error reported from esp-tls: 0x8008

Problem:

While trying to use TLS such as MQTTS or HTTPS on the ESP32, you see an error message like

E (333183) MQTT_CLIENT: mqtt_message_receive: transport_read() error: errno=119
[328153][E][MyMQTT.cpp:80] log_error_if_nonzero(): [MQTT] Last error reported from esp-tls: 0x8008
E (333191) MQTT_CLIENT: mqtt_process_receive: mqtt_message_receive() returned -1

Solution:

0x8008 means  ESP_ERR_ESP_TLS_TCP_CLOSED_FIN. In other words, a TCP connection had been established successfully but unexpectedly, the connection has been closed by the server.

This is often caused by the server software crashing, or restarting in some way. When a server process is terminated, the operating system will cleanup after it and close all connections.

In order to debug the issue, start by checking the log of your server message and/or system log to check for unintended crashes. If that doesn’t help, it’s sometimes helpful to packet capture the communication between the ESP32 and the server. You can also write a software script doing the same communication with the server as the ESP32. This will often allow you to try out changes much more easily than on the microcontroller and observe what’s happening using a debugger.

Posted by Uli Köhler in C/C++, Embedded, ESP8266/ESP32, Networking

How to fix ESP32 Last error reported from esp-tls: 0x8001

Problem:

While trying to use TLS such as MQTTS or HTTPS on the ESP32, you see an error message like

[MQTT] Last error reported from esp-tls: 0x8001

Solution:

0x8001 means ESP_ERR_ESP_TLS_CANNOT_RESOLVE_HOSTNAME. In other words, the ESP32 is unable to resolve the hostname of the host you’re trying to connect to using DNS.

Typically, this is a DNS problem, so check the DNS settings of your network. Also check if the ESP32 has a correct DNS server set – for example, if the ESP32 has 0.0.0.0 as a DNS server, this explains why it isn’t able to resolve the hostname.

Sometimes this issue is also caused by the hostname not existing at all (i.e. there is no DNS entry for that hostname). You can easily check this by resolving the hostname you’re trying to connect

Posted by Uli Köhler in C/C++, Embedded, ESP8266/ESP32, Networking