Programming languages

How to prevent pyinvoke “UnexpectedExit: Encountered a bad command exit code!”

Problem:

When you try running your command using pyinvoke’s invoke.run(), the native command returns a non-zero exit code making your script fail instead of just continuing (e.g. to evaluate the output):

Traceback (most recent call last):
File "/home/user/myscript.py", line 7, in <module>
result = invoke.run("mycommand", out_stream=outStream)
# ...
File "/usr/local/lib/python3.10/dist-packages/invoke/runners.py", line 518, in _finish
raise UnexpectedExit(result)
invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!

Solution:

Use warn=True as an argument to invoke.run()

This will tell invoke to just warn you about the command exiting with non-zero exit status instead of throwing an UnexpectedExit

Posted by Uli Köhler in Python

How to compute non-inverting OpAmp amplifier gain using UliEngineering in Python

In this example, we’ll use the UliEngineering library to compute the gain of a non-inverting OpAmp amplifier, given the two feedback resistor values R1 and R2

In order to install UliEngineering (a Python 3 library) run:

sudo pip3 install -U UliEngineering

We can now use noninverting_amplifier_gain() from the UliEngineering.Electronics.OpAmp package to convert between the units of temperature:

Tip: You can pass both numbers (like 100e3) or strings (such as 100 kΩ) to most UliEngineering functions. SI prefixes like k and m are automatically decoded.

Example:

from UliEngineering.Electronics.OpAmp import noninverting_amplifier_gain

# Gain of a non-inverting amplifier with 100k & 10k feedback resistor
gain = noninverting_amplifier_gain(100e3, 10e3)
# gain = 11.0

# ... or you can use strings
gain = noninverting_amplifier_gain("100k", "10k")
 
# ... or strings with units
gain = noninverting_amplifier_gain("100kΩ", "10kΩ")

# You can also automatically format the result
from UliEngineering.EngineerIO import auto_format
print(auto_format(noninverting_amplifier_gain, "100k", "10k"))
# prints 11.0 V/V
Posted by Uli Köhler in Electronics, Python

How to access RouterOS API in Python using the ‘routeros’ library (minimal example)

This example uses the routeros library from PyPI (GitHub) to access the MikroTik API and extract the system identity.

#!/usr/bin/env python3
from routeros import login

routeros = login('admin', 'abc123abc', '192.168.88.1')

output = routeros('/system/identity/print')
# Extract the one identity string from the list of dictionaries
print(output[0]['name'])

 

Posted by Uli Köhler in MikroTik, Python

Netmiko MikroTik RouterOS SSH key login example

In our previous example Netmiko MikroTik RouterOS minimal example we showed how to login to a RouterOS device using netmikoand password-based login.

#!/usr/bin/env python3  
from netmiko import ConnectHandler
import os.path
mikrotik = {
    'device_type': 'mikrotik_routeros',
    'host':   '192.168.88.1',
    'username': 'admin',
    'key_file': os.path.expanduser("~/.ssh/id_mikrotik"),
}
with ConnectHandler(**mikrotik) as mikrotik_connection:
    print(mikrotik_connection.send_command(f'/system/identity/print', cmd_verify=False))

 

Example output:

name: MySwitch01

 

Posted by Uli Köhler in MikroTik, Networking, Python

How to pass string to executable stdin using invoke.run() in Python

In our previous example How to capture stdout to string using invoke.run() in Python we showcased how to use invoke.run(..., out_stream=...) to capture stdout to a StringIO which can then be converted to a string.

Similarly, we can pass a string to the sub-process stdin by first converting it into a StringIO and then using invoke.run(..., in_stream=...).

stdin_str = "abc123" # what we want to pass to stdint

stdin_io = StringIO(stdin_str)
result = invoke.run("myexecutable", in_stream=stdin_in)
# ...
Posted by Uli Köhler in Python

How to capture stdout to string using invoke.run() in Python

This example showcases how to capture stdout to a string instead of printing it to system stdout:

The basic trick is to initialize a StringIO (a file-like object where pyinvoke can write the stdout from the sub-process to) and pass it to the out_stream argument of invoke.run()

import invoke
from io import StringIO

outStream = StringIO()
result = invoke.run("wg genkey", out_stream=outStream)
stdout_str = outStream.getvalue()
print("Result: ", stdout_str)

 

Posted by Uli Köhler in Python

How to fix Arduino / PlatformIO undefined reference to `loop()’

Problem:

While trying to compile your Arduino or PlatformIO project, you see an error message such as
/home/uli/.platformio/packages/[email protected]+2021r2-patch5/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp32dev/libFrameworkArduino.a(main.cpp.o):(.literal._Z8loopTaskPv+0x8): undefined reference to `loop()'

Solution:

You have not declared a loop() function in your source code. Open main.cpp or your .ino source code file and start with the following (empty) loop() function which does nothing:
void loop() {
    // Nothing to do here since HTTPServer
    // is running in a separate thread
    delay(1000);
}
After you’ve added any void loop() { /* ... */} function to your sourcecode try to build/upload again and the error message should have disappeared.
If you want, you can also add code such as to print a message to the serial port every time the loop is run:
void loop() {
    // Nothing to do here since HTTPServer
    // is running in a separate thread
    Serial.println("Hello world!");
    delay(1000);
}

 

Posted by Uli Köhler in Arduino, C/C++, PlatformIO

Dynamic short pulse generator using Arduino Mega2560 with serial control

This is a simplified version of our hybrid static/dynamic pulse generator. On the Arduino Mega2560 platform, it allows generating pulses from 750ns up to almost arbitrary lengths (limited only by the integer width) with a resolution of approx. 437.5ns

In order to control the pulse width, open a serial interface and type + to increase the pulse width or - to decrease the pulse width.

#include <Arduino.h>
#include <avr/io.h>
#include <ArduinoJson.h>

const int pulseAPin = 11;

#define PORT11 PORTB
#define PIN11 5
#define PIN11_MASK (1 << PIN11)

int pulseLength = 0;

/**
 * Pulse the output pin, for very short pulses.
 */
void PulseDynamic(int n) {
    cli();
    PORT11 |= PIN11_MASK;
    // Dynamic for loop
    for (;n >= 0; n--) {
        _NOP();
    }
    PORT11 &= ~PIN11_MASK;
    sei();
}

void setup()
{
    Serial.begin(115200);
    Serial.setTimeout(25);

    pinMode(pulseAPin, OUTPUT);
    pinMode(LED_BUILTIN, OUTPUT);
}

void loop()
{
    PulseDynamic(pulseLength);
    // Process serial
    while(Serial.available() > 0) {
        int c = Serial.read();
        if(c == '+') {
            pulseLength++;
            Serial.println(pulseLength);
        } else if(c == '-') {
            pulseLength--;
            if(pulseLength < 0) {pulseLength = 0;}
            Serial.println(pulseLength);
        }
    }
    delay(50);
}

 

Posted by Uli Köhler in Arduino, C/C++

Hybrid static/dynamic Arduino Mega2560 single channel short pulse generator

This pulse generator firmware uses methods discussed e.g. in Short pulse generation with Arduino Uno Part 4: NOP for loops to generate both dynamic (for loops of NOP instructions) and static (template-based automatically unrolled) in a combined manner so you can select more or less any pulse width using a high resolution.

It doesn’t work 100% since there are still some pulse width jumps. However, it is still useful for many microsecond 5V pulse generation applications and might also be a useful technology demonstrator for hybrid static/dynamic pulsers using optimized integer template loop unrolling approaches.

In order to control the pulse width, open a serial interface and type + to increase the pulse width or - to decrease the pulse width.

This might work on Arduino Unos as well but my Uno had a broken serial interface chip so I used a ATMega2560 board.

// License: CC0 1.0 Universal
// By Uli Köhler (techoverflow.net)
#include <Arduino.h>
#include <avr/io.h>
#include <ArduinoJson.h>

#define PORT13 PORTB
#define PIN13 7
#define PIN13_MASK (1 << PIN13)

#define PORT11 PORTB
#define PIN11 5
#define PIN11_MASK (1 << PIN11)

int pulseLength = 1;

using PulseFunc = void (*)(int);

// Function pointer to void(int)
PulseFunc pulser = nullptr;
int pulserParam = 0; // Pre-computed parameter for the pulser function

void DoNothingPulser(int _) {
}

/**
 * Pulse the output pin, for very short pulses.
 * Will ONLY perform static (optimized) NOPs.
 * Will NOT perform dynamic (slower) NOP cycles
 * 
 * Hence, this function does not have any overhead from for loops.
 */
template<int nNOPs>
void PulseStatic(int _) {
    cli();
    PORT11 |= PIN11_MASK;
    // Static for loop (optimized out - template-driven)
    // Force unrolling of the loop
    // NOTE: Compiler will only unroll for n < 8
    for (int i = 0; i < min(nNOPs, 6); i++) {
        _NOP();
    }
    if(nNOPs > 6) {
        for (int i = 0; i < nNOPs - 6; i++) {
            _NOP();
        }
    }
    PORT11 &= ~PIN11_MASK;
    sei();
}

/**
 * Pulse the output pin, for very short pulses.
 * Will perform static (optimized) NOPs and
 * also dynamic (slower) NOP cycles
 */
template<int nNOPs>
void PulseDynamic(int dynamicNOPs) {
    cli();
    PORT11 |= PIN11_MASK;
    // Dynamic for loop (NOT optimized out - template-driven)
    for (int i = 0; i < dynamicNOPs; i++)
    {
        _NOP();
    }
    // Static for loop (optimized out - template-driven)
    // Force unrolling of the loop
    // NOTE: Compiler will only unroll for n < 8
    for (int i = 0; i < min(nNOPs, 6); i++) {
        _NOP();
    }
    if(nNOPs > 6) {
        for (int i = 0; i < nNOPs - 6; i++) {
            _NOP();
        }
    }
    PORT11 &= ~PIN11_MASK;
    sei();
}

PulseFunc staticPulsers[] = {
    DoNothingPulser,
    &PulseStatic<0>, // 0 NOPs
    &PulseStatic<1>, // 1 NOPs ...
    &PulseStatic<2>,
    &PulseStatic<3>,
    &PulseStatic<4>,
    &PulseStatic<5>,
    &PulseStatic<6>,
    &PulseStatic<7>,
    &PulseStatic<8>,
    &PulseStatic<9>,
    &PulseStatic<10>,
    &PulseStatic<11>
};

PulseFunc dynamicPulsers[] = {
    &PulseDynamic<0>,
    &PulseDynamic<1>,
    &PulseDynamic<2>,
    &PulseDynamic<3>,
    &PulseDynamic<4>,
    &PulseDynamic<5>,
    &PulseDynamic<6>,
    &PulseDynamic<7>,
    &PulseDynamic<8>,
    &PulseDynamic<9>,
    &PulseDynamic<10>,
    &PulseDynamic<11>
};

constexpr int A = 7;
constexpr int B = 6;

void ReconfigurePulse() {
    // Very short pulses are performed using only static
    if(pulseLength < A) {
        pulser = staticPulsers[pulseLength];
    } else {
        pulser = dynamicPulsers[(pulseLength - A) % B];
        pulserParam = (pulseLength - A) / B;
    }
}

void setup()
{
    Serial.begin(115200);
    Serial.setTimeout(25);

    ReconfigurePulse(); // with default pulseLength

    pinMode(11, OUTPUT);
    pinMode(LED_BUILTIN, OUTPUT);
}

void loop()
{
    pulser(pulserParam);
    Serial.println(pulseLength);
    //ProcessSerialInput();
    // Wait until 50ms has passed since start
    while(Serial.available() > 0) {
        int c = Serial.read();
        if(c == '+') {
            pulseLength++;
            ReconfigurePulse();
        } else if(c == '-') {
            pulseLength--;
            if(pulseLength < 0) {pulseLength = 0;}
            ReconfigurePulse();
        }
    }
    delay(50);
}

 

 

Posted by Uli Köhler in Arduino, C/C++

How to list all C++ header files included by a source or header file recursively

You can use g++ -M to list all files included by a header file recursively:

gcc -M myheader.hpp

Often you want to ignore system header files from /usr/include etc – in that case, use -MM:

gcc -MM myheader.hpp

For example:

gcc -M /usr/include/boost/circular_buffer/base.hpp

will print:

base.o: /usr/include/boost/circular_buffer/base.hpp \
 /usr/include/stdc-predef.h /usr/include/boost/config.hpp \
 /usr/include/boost/config/user.hpp \
 /usr/include/boost/config/detail/select_compiler_config.hpp \
 /usr/include/boost/config/compiler/gcc.hpp /usr/include/c++/11/cstddef \
 /usr/include/x86_64-linux-gnu/c++/11/bits/c++config.h \
 /usr/include/x86_64-linux-gnu/c++/11/bits/os_defines.h \
 /usr/include/features.h /usr/include/features-time64.h \
 /usr/include/x86_64-linux-gnu/bits/wordsize.h \
 /usr/include/x86_64-linux-gnu/bits/timesize.h \
 /usr/include/x86_64-linux-gnu/sys/cdefs.h \
 /usr/include/x86_64-linux-gnu/bits/long-double.h \
 /usr/include/x86_64-linux-gnu/gnu/stubs.h \
 /usr/include/x86_64-linux-gnu/gnu/stubs-64.h \
 /usr/include/x86_64-linux-gnu/c++/11/bits/cpu_defines.h \
 /usr/include/c++/11/pstl/pstl_config.h \
 /usr/lib/gcc/x86_64-linux-gnu/11/include/stddef.h \
 /usr/include/boost/config/detail/select_stdlib_config.hpp \
 /usr/include/c++/11/version \
 /usr/include/boost/config/stdlib/libstdcpp3.hpp /usr/include/unistd.h \
 /usr/include/x86_64-linux-gnu/bits/posix_opt.h \
 /usr/include/x86_64-linux-gnu/bits/environments.h \
 /usr/include/x86_64-linux-gnu/bits/types.h \
 /usr/include/x86_64-linux-gnu/bits/typesizes.h \
 /usr/include/x86_64-linux-gnu/bits/time64.h \
 /usr/include/x86_64-linux-gnu/bits/confname.h \
 /usr/include/x86_64-linux-gnu/bits/getopt_posix.h \
 /usr/include/x86_64-linux-gnu/bits/getopt_core.h \
 /usr/include/x86_64-linux-gnu/bits/unistd_ext.h \
 /usr/include/linux/close_range.h \
 /usr/include/boost/config/detail/select_platform_config.hpp \
 /usr/include/boost/config/platform/linux.hpp /usr/include/c++/11/cstdlib \
 /usr/include/stdlib.h \
 /usr/include/x86_64-linux-gnu/bits/libc-header-start.h \
 /usr/include/x86_64-linux-gnu/bits/waitflags.h \
 /usr/include/x86_64-linux-gnu/bits/waitstatus.h \
 /usr/include/x86_64-linux-gnu/bits/floatn.h \
 /usr/include/x86_64-linux-gnu/bits/floatn-common.h \
 /usr/include/x86_64-linux-gnu/bits/types/locale_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/__locale_t.h \
 /usr/include/x86_64-linux-gnu/sys/types.h \
 /usr/include/x86_64-linux-gnu/bits/types/clock_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/clockid_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/time_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/timer_t.h \
 /usr/include/x86_64-linux-gnu/bits/stdint-intn.h /usr/include/endian.h \
 /usr/include/x86_64-linux-gnu/bits/endian.h \
 /usr/include/x86_64-linux-gnu/bits/endianness.h \
 /usr/include/x86_64-linux-gnu/bits/byteswap.h \
 /usr/include/x86_64-linux-gnu/bits/uintn-identity.h \
 /usr/include/x86_64-linux-gnu/sys/select.h \
 /usr/include/x86_64-linux-gnu/bits/select.h \
 /usr/include/x86_64-linux-gnu/bits/types/sigset_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/__sigset_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/struct_timeval.h \
 /usr/include/x86_64-linux-gnu/bits/types/struct_timespec.h \
 /usr/include/x86_64-linux-gnu/bits/pthreadtypes.h \
 /usr/include/x86_64-linux-gnu/bits/thread-shared-types.h \
 /usr/include/x86_64-linux-gnu/bits/pthreadtypes-arch.h \
 /usr/include/x86_64-linux-gnu/bits/atomic_wide_counter.h \
 /usr/include/x86_64-linux-gnu/bits/struct_mutex.h \
 /usr/include/x86_64-linux-gnu/bits/struct_rwlock.h /usr/include/alloca.h \
 /usr/include/x86_64-linux-gnu/bits/stdlib-float.h \
 /usr/include/c++/11/bits/std_abs.h \
 /usr/include/boost/config/detail/posix_features.hpp \
 /usr/include/boost/config/detail/suffix.hpp \
 /usr/include/boost/config/helper_macros.hpp \
 /usr/include/boost/config/detail/cxx_composite.hpp \
 /usr/include/boost/concept_check.hpp \
 /usr/include/boost/concept/assert.hpp \
 /usr/include/boost/config/workaround.hpp \
 /usr/include/boost/concept/detail/general.hpp \
 /usr/include/boost/preprocessor/cat.hpp \
 /usr/include/boost/preprocessor/config/config.hpp \
 /usr/include/boost/concept/detail/backward_compatibility.hpp \
 /usr/include/boost/concept/detail/has_constraints.hpp \
 /usr/include/boost/type_traits/integral_constant.hpp \
 /usr/include/boost/detail/workaround.hpp \
 /usr/include/boost/type_traits/conditional.hpp \
 /usr/include/c++/11/iterator \
 /usr/include/c++/11/bits/stl_iterator_base_types.h \
 /usr/include/c++/11/type_traits \
 /usr/include/c++/11/bits/stl_iterator_base_funcs.h \
 /usr/include/c++/11/bits/concept_check.h \
 /usr/include/c++/11/debug/assertions.h \
 /usr/include/c++/11/bits/stl_iterator.h \
 /usr/include/c++/11/bits/cpp_type_traits.h \
 /usr/include/c++/11/ext/type_traits.h /usr/include/c++/11/bits/move.h \
 /usr/include/c++/11/bits/ptr_traits.h /usr/include/c++/11/iosfwd \
 /usr/include/c++/11/bits/stringfwd.h \
 /usr/include/c++/11/bits/memoryfwd.h /usr/include/c++/11/bits/postypes.h \
 /usr/include/c++/11/cwchar /usr/include/wchar.h \
 /usr/lib/gcc/x86_64-linux-gnu/11/include/stdarg.h \
 /usr/include/x86_64-linux-gnu/bits/wchar.h \
 /usr/include/x86_64-linux-gnu/bits/types/wint_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/mbstate_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/__mbstate_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/__FILE.h \
 /usr/include/x86_64-linux-gnu/bits/types/FILE.h \
 /usr/include/c++/11/bits/stream_iterator.h \
 /usr/include/c++/11/debug/debug.h \
 /usr/include/c++/11/bits/streambuf_iterator.h \
 /usr/include/c++/11/streambuf /usr/include/c++/11/bits/localefwd.h \
 /usr/include/x86_64-linux-gnu/c++/11/bits/c++locale.h \
 /usr/include/c++/11/clocale /usr/include/locale.h \
 /usr/include/x86_64-linux-gnu/bits/locale.h /usr/include/c++/11/cctype \
 /usr/include/ctype.h /usr/include/c++/11/bits/ios_base.h \
 /usr/include/c++/11/ext/atomicity.h \
 /usr/include/x86_64-linux-gnu/c++/11/bits/gthr.h \
 /usr/include/x86_64-linux-gnu/c++/11/bits/gthr-default.h \
 /usr/include/pthread.h /usr/include/sched.h \
 /usr/include/x86_64-linux-gnu/bits/sched.h \
 /usr/include/x86_64-linux-gnu/bits/types/struct_sched_param.h \
 /usr/include/x86_64-linux-gnu/bits/cpu-set.h /usr/include/time.h \
 /usr/include/x86_64-linux-gnu/bits/time.h \
 /usr/include/x86_64-linux-gnu/bits/timex.h \
 /usr/include/x86_64-linux-gnu/bits/types/struct_tm.h \
 /usr/include/x86_64-linux-gnu/bits/types/struct_itimerspec.h \
 /usr/include/x86_64-linux-gnu/bits/setjmp.h \
 /usr/include/x86_64-linux-gnu/bits/types/struct___jmp_buf_tag.h \
 /usr/include/x86_64-linux-gnu/bits/pthread_stack_min-dynamic.h \
 /usr/include/x86_64-linux-gnu/c++/11/bits/atomic_word.h \
 /usr/include/x86_64-linux-gnu/sys/single_threaded.h \
 /usr/include/c++/11/bits/locale_classes.h /usr/include/c++/11/string \
 /usr/include/c++/11/bits/char_traits.h \
 /usr/include/c++/11/bits/stl_algobase.h \
 /usr/include/c++/11/bits/functexcept.h \
 /usr/include/c++/11/bits/exception_defines.h \
 /usr/include/c++/11/ext/numeric_traits.h \
 /usr/include/c++/11/bits/stl_pair.h \
 /usr/include/c++/11/bits/predefined_ops.h /usr/include/c++/11/cstdint \
 /usr/lib/gcc/x86_64-linux-gnu/11/include/stdint.h /usr/include/stdint.h \
 /usr/include/x86_64-linux-gnu/bits/stdint-uintn.h \
 /usr/include/c++/11/bits/allocator.h \
 /usr/include/x86_64-linux-gnu/c++/11/bits/c++allocator.h \
 /usr/include/c++/11/ext/new_allocator.h /usr/include/c++/11/new \
 /usr/include/c++/11/bits/exception.h \
 /usr/include/c++/11/bits/ostream_insert.h \
 /usr/include/c++/11/bits/cxxabi_forced.h \
 /usr/include/c++/11/bits/stl_function.h \
 /usr/include/c++/11/backward/binders.h \
 /usr/include/c++/11/bits/range_access.h \
 /usr/include/c++/11/initializer_list \
 /usr/include/c++/11/bits/basic_string.h \
 /usr/include/c++/11/ext/alloc_traits.h \
 /usr/include/c++/11/bits/alloc_traits.h \
 /usr/include/c++/11/bits/stl_construct.h /usr/include/c++/11/string_view \
 /usr/include/c++/11/bits/functional_hash.h \
 /usr/include/c++/11/bits/hash_bytes.h \
 /usr/include/c++/11/bits/string_view.tcc \
 /usr/include/c++/11/ext/string_conversions.h /usr/include/c++/11/cstdio \
 /usr/include/stdio.h /usr/include/x86_64-linux-gnu/bits/types/__fpos_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/__fpos64_t.h \
 /usr/include/x86_64-linux-gnu/bits/types/struct_FILE.h \
 /usr/include/x86_64-linux-gnu/bits/types/cookie_io_functions_t.h \
 /usr/include/x86_64-linux-gnu/bits/stdio_lim.h \
 /usr/include/c++/11/cerrno /usr/include/errno.h \
 /usr/include/x86_64-linux-gnu/bits/errno.h /usr/include/linux/errno.h \
 /usr/include/x86_64-linux-gnu/asm/errno.h \
 /usr/include/asm-generic/errno.h /usr/include/asm-generic/errno-base.h \
 /usr/include/x86_64-linux-gnu/bits/types/error_t.h \
 /usr/include/c++/11/bits/charconv.h \
 /usr/include/c++/11/bits/basic_string.tcc \
 /usr/include/c++/11/bits/locale_classes.tcc \
 /usr/include/c++/11/system_error \
 /usr/include/x86_64-linux-gnu/c++/11/bits/error_constants.h \
 /usr/include/c++/11/stdexcept /usr/include/c++/11/exception \
 /usr/include/c++/11/bits/exception_ptr.h \
 /usr/include/c++/11/bits/cxxabi_init_exception.h \
 /usr/include/c++/11/typeinfo /usr/include/c++/11/bits/nested_exception.h \
 /usr/include/c++/11/bits/streambuf.tcc \
 /usr/include/boost/type_traits/conversion_traits.hpp \
 /usr/include/boost/type_traits/is_convertible.hpp \
 /usr/include/boost/type_traits/intrinsics.hpp \
 /usr/include/boost/type_traits/detail/config.hpp \
 /usr/include/boost/version.hpp \
 /usr/include/boost/type_traits/is_complete.hpp \
 /usr/include/boost/type_traits/declval.hpp \
 /usr/include/boost/type_traits/add_rvalue_reference.hpp \
 /usr/include/boost/type_traits/is_void.hpp \
 /usr/include/boost/type_traits/is_reference.hpp \
 /usr/include/boost/type_traits/is_lvalue_reference.hpp \
 /usr/include/boost/type_traits/is_rvalue_reference.hpp \
 /usr/include/boost/type_traits/remove_reference.hpp \
 /usr/include/boost/type_traits/is_function.hpp \
 /usr/include/boost/type_traits/detail/is_function_cxx_11.hpp \
 /usr/include/boost/type_traits/detail/yes_no_type.hpp \
 /usr/include/boost/type_traits/is_array.hpp \
 /usr/include/boost/static_assert.hpp \
 /usr/include/boost/type_traits/is_arithmetic.hpp \
 /usr/include/boost/type_traits/is_integral.hpp \
 /usr/include/boost/type_traits/is_floating_point.hpp \
 /usr/include/boost/type_traits/is_abstract.hpp \
 /usr/include/boost/type_traits/add_lvalue_reference.hpp \
 /usr/include/boost/type_traits/add_reference.hpp \
 /usr/include/c++/11/utility /usr/include/c++/11/bits/stl_relops.h \
 /usr/include/boost/type_traits/is_same.hpp \
 /usr/include/boost/concept/usage.hpp \
 /usr/include/boost/concept/detail/concept_def.hpp \
 /usr/include/boost/preprocessor/seq/for_each_i.hpp \
 /usr/include/boost/preprocessor/arithmetic/dec.hpp \
 /usr/include/boost/preprocessor/config/limits.hpp \
 /usr/include/boost/preprocessor/arithmetic/limits/dec_256.hpp \
 /usr/include/boost/preprocessor/arithmetic/inc.hpp \
 /usr/include/boost/preprocessor/arithmetic/limits/inc_256.hpp \
 /usr/include/boost/preprocessor/control/if.hpp \
 /usr/include/boost/preprocessor/control/iif.hpp \
 /usr/include/boost/preprocessor/logical/bool.hpp \
 /usr/include/boost/preprocessor/logical/limits/bool_256.hpp \
 /usr/include/boost/preprocessor/repetition/for.hpp \
 /usr/include/boost/preprocessor/debug/error.hpp \
 /usr/include/boost/preprocessor/facilities/empty.hpp \
 /usr/include/boost/preprocessor/detail/auto_rec.hpp \
 /usr/include/boost/preprocessor/detail/limits/auto_rec_256.hpp \
 /usr/include/boost/preprocessor/repetition/detail/for.hpp \
 /usr/include/boost/preprocessor/control/expr_iif.hpp \
 /usr/include/boost/preprocessor/tuple/eat.hpp \
 /usr/include/boost/preprocessor/repetition/detail/limits/for_256.hpp \
 /usr/include/boost/preprocessor/repetition/limits/for_256.hpp \
 /usr/include/boost/preprocessor/seq/seq.hpp \
 /usr/include/boost/preprocessor/seq/elem.hpp \
 /usr/include/boost/preprocessor/seq/limits/elem_256.hpp \
 /usr/include/boost/preprocessor/seq/size.hpp \
 /usr/include/boost/preprocessor/seq/limits/size_256.hpp \
 /usr/include/boost/preprocessor/seq/detail/is_empty.hpp \
 /usr/include/boost/preprocessor/logical/compl.hpp \
 /usr/include/boost/preprocessor/tuple/elem.hpp \
 /usr/include/boost/preprocessor/facilities/expand.hpp \
 /usr/include/boost/preprocessor/facilities/overload.hpp \
 /usr/include/boost/preprocessor/variadic/size.hpp \
 /usr/include/boost/preprocessor/facilities/check_empty.hpp \
 /usr/include/boost/preprocessor/variadic/has_opt.hpp \
 /usr/include/boost/preprocessor/variadic/limits/size_64.hpp \
 /usr/include/boost/preprocessor/tuple/rem.hpp \
 /usr/include/boost/preprocessor/tuple/detail/is_single_return.hpp \
 /usr/include/boost/preprocessor/variadic/elem.hpp \
 /usr/include/boost/preprocessor/variadic/limits/elem_64.hpp \
 /usr/include/boost/preprocessor/seq/enum.hpp \
 /usr/include/boost/preprocessor/seq/limits/enum_256.hpp \
 /usr/include/boost/preprocessor/comma_if.hpp \
 /usr/include/boost/preprocessor/punctuation/comma_if.hpp \
 /usr/include/boost/preprocessor/punctuation/comma.hpp \
 /usr/include/boost/concept/detail/concept_undef.hpp \
 /usr/include/boost/limits.hpp /usr/include/c++/11/limits \
 /usr/include/boost/core/allocator_access.hpp \
 /usr/include/boost/core/pointer_traits.hpp \
 /usr/include/boost/core/addressof.hpp \
 /usr/include/boost/core/empty_value.hpp \
 /usr/include/boost/type_traits/is_stateless.hpp \
 /usr/include/boost/type_traits/has_trivial_constructor.hpp \
 /usr/include/boost/type_traits/is_pod.hpp \
 /usr/include/boost/type_traits/is_scalar.hpp \
 /usr/include/boost/type_traits/is_enum.hpp \
 /usr/include/boost/type_traits/is_pointer.hpp \
 /usr/include/boost/type_traits/is_member_pointer.hpp \
 /usr/include/boost/type_traits/is_member_function_pointer.hpp \
 /usr/include/boost/type_traits/detail/is_member_function_pointer_cxx_11.hpp \
 /usr/include/boost/type_traits/is_default_constructible.hpp \
 /usr/include/boost/type_traits/is_volatile.hpp \
 /usr/include/boost/type_traits/has_trivial_copy.hpp \
 /usr/include/boost/type_traits/is_copy_constructible.hpp \
 /usr/include/boost/type_traits/is_constructible.hpp \
 /usr/include/boost/type_traits/is_destructible.hpp \
 /usr/include/boost/type_traits/has_trivial_destructor.hpp \
 /usr/include/boost/type_traits/is_class.hpp \
 /usr/include/boost/type_traits/is_empty.hpp \
 /usr/include/boost/type_traits/remove_cv.hpp \
 /usr/include/boost/type_traits/is_nothrow_move_constructible.hpp \
 /usr/include/boost/type_traits/enable_if.hpp \
 /usr/include/boost/type_traits/is_nothrow_move_assignable.hpp \
 /usr/include/boost/type_traits/has_trivial_move_assign.hpp \
 /usr/include/boost/type_traits/is_assignable.hpp \
 /usr/include/boost/type_traits/has_nothrow_assign.hpp \
 /usr/include/boost/type_traits/is_const.hpp \
 /usr/include/boost/move/adl_move_swap.hpp \
 /usr/include/boost/move/detail/workaround.hpp \
 /usr/include/boost/move/utility_core.hpp \
 /usr/include/boost/move/detail/config_begin.hpp \
 /usr/include/boost/move/core.hpp \
 /usr/include/boost/move/detail/config_end.hpp \
 /usr/include/boost/move/detail/meta_utils.hpp \
 /usr/include/boost/move/detail/meta_utils_core.hpp \
 /usr/include/boost/move/detail/addressof.hpp \
 /usr/include/boost/move/move.hpp /usr/include/boost/move/utility.hpp \
 /usr/include/boost/move/traits.hpp \
 /usr/include/boost/move/detail/type_traits.hpp \
 /usr/include/c++/11/cassert /usr/include/assert.h \
 /usr/include/boost/move/iterator.hpp \
 /usr/include/boost/move/detail/iterator_traits.hpp \
 /usr/include/boost/move/detail/std_ns_begin.hpp \
 /usr/include/boost/move/detail/std_ns_end.hpp \
 /usr/include/boost/move/algorithm.hpp \
 /usr/include/boost/move/algo/move.hpp \
 /usr/include/boost/move/detail/iterator_to_raw_pointer.hpp \
 /usr/include/boost/move/detail/to_raw_pointer.hpp \
 /usr/include/boost/move/detail/pointer_element.hpp \
 /usr/include/c++/11/algorithm /usr/include/c++/11/bits/stl_algo.h \
 /usr/include/c++/11/bits/algorithmfwd.h \
 /usr/include/c++/11/bits/stl_heap.h \
 /usr/include/c++/11/bits/stl_tempbuf.h \
 /usr/include/c++/11/bits/uniform_int_dist.h \
 /usr/include/c++/11/pstl/glue_algorithm_defs.h \
 /usr/include/c++/11/functional /usr/include/c++/11/tuple \
 /usr/include/c++/11/array /usr/include/c++/11/bits/uses_allocator.h \
 /usr/include/c++/11/bits/invoke.h /usr/include/c++/11/bits/refwrap.h \
 /usr/include/c++/11/bits/std_function.h \
 /usr/include/c++/11/unordered_map \
 /usr/include/c++/11/ext/aligned_buffer.h \
 /usr/include/c++/11/bits/hashtable.h \
 /usr/include/c++/11/bits/hashtable_policy.h \
 /usr/include/c++/11/bits/enable_special_members.h \
 /usr/include/c++/11/bits/node_handle.h \
 /usr/include/c++/11/bits/unordered_map.h \
 /usr/include/c++/11/bits/erase_if.h /usr/include/c++/11/vector \
 /usr/include/c++/11/bits/stl_uninitialized.h \
 /usr/include/c++/11/bits/stl_vector.h \
 /usr/include/c++/11/bits/stl_bvector.h \
 /usr/include/c++/11/bits/vector.tcc \
 /usr/include/c++/11/pstl/execution_defs.h /usr/include/c++/11/memory \
 /usr/include/c++/11/bits/stl_raw_storage_iter.h \
 /usr/include/c++/11/bits/align.h /usr/include/c++/11/bit \
 /usr/include/c++/11/bits/unique_ptr.h \
 /usr/include/c++/11/bits/shared_ptr.h \
 /usr/include/c++/11/bits/shared_ptr_base.h \
 /usr/include/c++/11/bits/allocated_ptr.h \
 /usr/include/c++/11/ext/concurrence.h \
 /usr/include/c++/11/bits/shared_ptr_atomic.h \
 /usr/include/c++/11/bits/atomic_base.h \
 /usr/include/c++/11/bits/atomic_lockfree_defines.h \
 /usr/include/c++/11/backward/auto_ptr.h \
 /usr/include/c++/11/pstl/glue_memory_defs.h /usr/include/c++/11/deque \
 /usr/include/c++/11/bits/stl_deque.h /usr/include/c++/11/bits/deque.tcc

 

Posted by Uli Köhler in C/C++

How to list only sub-folders (not files) using Python

Whereas Python’s os.listdir("mydirectory") will list both sub-files and sub-directories, you can use this simple list comprehension:

parent_folder = "myfolder"
sub_directories = [
    filename for filename in os.listdir(parent_folder)
    if os.path.isdir(os.path.join(parent_folder, filename))
]

which will only list sub-directories of the parent_fodler

 

Posted by Uli Köhler in Python

How to matplotlib plt.savefig() to a io.BytesIO buffer

You can just pass a io.BytesIO() to plt.savefig() as first parameter. Note that format defaults to "png", but best practice is to explicitly specify it.

# Save plot to BytesIO
bio = io.BytesIO()
plt.savefig(bio, dpi=250, format="png")

Full example:

#!/usr/bin/env python3
import numpy as np
import io
import matplotlib.pyplot as plt

# Generate data
x = np.linspace(0, 2*np.pi, 100)
y = np.sin(x)

# Plot data
plt.plot(x, y)

# Save plot to BytesIO
bio = io.BytesIO()
plt.savefig(bio, dpi=250, format="png")

# Cleanup plot
plt.close(plt.gcf())
plt.clf()

# Write BytesIO to file
with open("plot.png", "wb") as f:
    f.write(bio.getvalue())

Output plot.png:

 

Posted by Uli Köhler in Python

C++ S3 GetObject minimal streaming example using minio-cpp

#include <client.h>
#include <iostream>

using std::cout, std::endl;

int main(int argc, char* argv[]) {
  // Create S3 base URL.
  minio::s3::BaseUrl base_url("minio.mydomain.com");

  // Create credential provider.
  minio::creds::StaticProvider provider(
      "my-access-key", "my-secret-key");

  // Create S3 client.
  minio::s3::Client client(base_url, &provider);
  std::string bucket_name = "my-bucket";

  // Build arguments object
  minio::s3::GetObjectArgs args;
  args.bucket = bucket_name;
  args.object = "my-object.txt";
  args.datafunc = [](minio::http::DataFunctionArgs args) -> bool {
    // This function will be called for every data chunk of the object
    cout << args.datachunk;
    return true;
  };

  // Perform the request (calling datafunc for every chunk)
  minio::s3::GetObjectResponse resp = client.GetObject(args);
  // Handle error (if any)
  if (resp) {
    cout << endl // end line after file content
         << "data of my-object is received successfully" << endl;
  } else {
    cout << "Error during GetObject(): " << resp.Error().String() << endl;
  }

  return EXIT_SUCCESS;
}

 

Posted by Uli Köhler in C/C++, S3

C++ S3 ListObjects minimal example using minio-cpp

#include <client.h>

int main(int argc, char* argv[]) {
  // Create S3 base URL.
  minio::s3::BaseUrl base_url("minio.mydomain.com");

  // Create credential provider.
  minio::creds::StaticProvider provider(
      "my_access_key", "my_secret_key");

  // Create S3 client.
  minio::s3::Client client(base_url, &provider);
  std::string bucket_name = "my-bucket";

  minio::s3::ListObjectsArgs args;
  args.bucket = bucket_name;
  // Optional prefix filter
  args.prefix = "folder/";

  minio::s3::ListObjectsResult result = client.ListObjects(args);
  for (; result; result++) {
      minio::s3::Item item = *result;
      if (!item) {
        throw std::runtime_error("Error during ListObjects(): " + item.Error().String());
      }
      std::cout << item.name << std::endl;
  }

  return EXIT_SUCCESS;
}

 

Posted by Uli Köhler in C/C++, S3

How to fix C++ linking error: undefined reference to symbol ‘crc32’

Problem:

While trying to compile your C++ program, during the linking stage, you see an error message such as

/bin/ld: minio-cpp/src/utils.o: undefined reference to symbol 'crc32'
/bin/ld: /lib/x86_64-linux-gnu/libz.so.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

Solution:

The crc32 symbol is defined in zlib. Hence, you need to link zlib by adding

-lz

to your linker flags. If you don’t know where your linker flags are, try just adding it to your compiler flags.

Posted by Uli Köhler in C/C++, GCC errors

How to fix C++ “fatal error: INIReader.h: No such file or directory” on Ubuntu

Problem:

While trying to compile your C++ project, you see an error message such as

minio-cpp/include/providers.h:19:10: fatal error: INIReader.h: No such file or directory
   19 | #include <INIReader.h>
      |          ^~~~~~~~~~~~~
compilation terminated.

Solution:

You need to install the inih library. On Ubuntu or Debian, you can do that using

sudo apt -y install libinih-dev

 

Posted by Uli Köhler in C/C++, GCC errors

How to fix C++ “fatal error: curlpp/Easy.hpp: No such file or directory” on Ubuntu

Problem:

While trying to compile your C++ project, you see an error message such as

minio-cpp/include/http.h:21:10: fatal error: curlpp/Easy.hpp: No such file or directory
   21 | #include <curlpp/Easy.hpp>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.

Solution:

You need to install the curlpp library, a C++ wrapper around libcurl. On Ubuntu or Debian, you can do that using

sudo apt -y install libcurlpp-dev

 

Posted by Uli Köhler in C/C++, GCC errors

How to fix C++ “fatal error: pugixml.hpp: No such file or directory” on Ubuntu

Problem:

While trying to compile your C++ project, you see an error message such as

minio-cpp/include/select.h:19:10: fatal error: pugixml.hpp: No such file or directory
   19 | #include <pugixml.hpp>

Solution:

You need to install the PugiXML library. On Ubuntu or Debian, you can do that using

sudo apt -y install libpugixml-dev

 

Posted by Uli Köhler in C/C++, GCC errors

How to fix C++ error: ‘put_time’ is not a member of ‘std’

Problem:

While trying to compile your C++ application, you see an error message such as

src/HTTPServer.cpp:12:16: error: 'put_time' is not a member of 'std'
     ss << std::put_time(std::localtime(&localTime), "%FT%H-%M-%SZ");;
                ^~~~~~~~

Solution:

At the top of the file where this error occured, you need to add

#include <iomanip>

iomanip contains std::put_time() among other functionality.

Posted by Uli Köhler in C/C++, GCC errors

Customizable Python INI file parser

I recommend using the configparser library which comes with Python for parsing INI files. However, if you need a highly customizable parser, this code is a good place to start:

def read_ini_config(filename):
    config = {}
    current_section = None

    with open(filename, 'r', encoding="utf-8") as file:
        for line in file:
            line = line.strip()

            # Skip empty lines and comments
            if not line or line.startswith(';') or line.startswith('#'):
                continue

            # Check if it's a section header
            if line.startswith('[') and line.endswith(']'):
                current_section = line[1:-1]
                config[current_section] = {}
            else:
                # Parse key-value pairs
                key, value = line.split('=', 1)
                key = key.strip()
                value = value.strip()
                config[current_section][key] = value

    return config

 

Posted by Uli Köhler in Python