Programming languages

requests: Download file if it doesn’t exist

Problem:

You want to download a URL to a file using the requests python library, but you want to skip the download if it doesn’t exist

Solution:

Use the following functions:

import requests
import os.path

def download_file(filename, url):
    """
    Download an URL to a file
    """
    with open(filename, 'wb') as fout:
        response = requests.get(url, stream=True)
        response.raise_for_status()
        # Write response data to file
        for block in response.iter_content(4096):
            fout.write(block)

def download_if_not_exists(filename, url):
    """
    Download a URL to a file if the file
    does not exist already.

    Returns
    -------
    True if the file was downloaded,
    False if it already existed
    """
    if not os.path.exists(filename):
        download_file(filename, url)
        return True
    return False

 

Posted by Uli Köhler in Python

Removing spans/divs with style attributes from HTML

Occasionally I have to clean up some HTML code – mostly because parts of it were pasted into a CMS like WordPress from rich text editor like Word.

I’ve noticed that the formatting I want to remove is mostly based on span and div elements with a style attribute. Therefore, I’ve written a simple Python script based on BeautifulSoup4 which will replace certain tags with their contents if they have a style attribute. While in some cases some other formatting might be destroyed by such a script, it is very useful for some recurring usecases.

Continue reading →

Posted by Uli Köhler in Python

Advantages and disadvantages of hugepages

In a previous post, I’ve written about how to check and enable transparent hugepages in Linux globally.

Although this post is important if you actually have a usecase for hugepages, I’ve seen multiple people getting fooled by the prospect that hugepages will magically increase performance. However, hugepaging is a complex topic and, if used in the wrong way, might easily decrease overall performance. Continue reading →

Posted by Uli Köhler in C/C++, Performance

In-place trimming/stripping in C

For an explanation of in-place algorithms see my previous post on zero-copy in-place splitting

The problem

You have a C string possibly containing whitespace at the beginning and/or the end.

char* s = " abc   \n\r";

Using an in-place algorithm, you want to remove the whitespace from this string.

Doing this is also possible using boost::algorithm::trim, but it has the same caveats as boost::algorithm::split as discussed in my previous post about C splitting Continue reading →

Posted by Uli Köhler in C/C++

Zero-copy in-place string splitting in C

Let’s assume you have a string:

char* s = "1,23,456,7890";

You want to split said string at each comma in order to obtain its parts as C strings (with the number of parts being variable):

char* s1 = "1";
char* s2 = "23";
char* s3 = "456";
char* s4 = "7890";

Continue reading →

Posted by Uli Köhler in C/C++

Computing the LP2980 adjust resistor using Python

The LP2980ADJ is a 50 mA LDO that be configured for an output voltage from 1.23V to 15V using a pair of resistors.

The datasheet lists a formula for the output voltage, however no easy-to-use customizable software is provided that can be used to directly compute the correct resistor in a reproducible way. Continue reading →

Posted by Uli Köhler in Electronics, Python

Uploading multiple files to the tornado webserver

The following html code can be used to create an html form that allows uploading multiple files at once:

<form enctype="multipart/form-data" method="POST" action="upload.py">
  <table style="width: 100%">
    <tr>
      <td>Choose the files to upload:</td>
      <td style="text-align: right"><input type="file" multiple="" id="files" name="files"></td>
    </tr>
    <tr>
      <td><input id="fileUploadButton" type="submit" value="Upload &gt;&gt;"></td>
      <td></td>
    </tr>
  </table>
</form>

Continue reading →

Posted by Yann Spöri in Python

Normalizing electronics engineering value notations using Python

In electronics engineering there is a wide variety of notations for values that need to be recognized by intuitive user interfaces. Examples include:

  • 1fA
  • 0.1A
  • 0.00001
  • 1e-6
  • 4,5nA
  • 4,500.123 A
  • 4A5
  • 4k0 A

The wide variety of options, including thousands separators, comma-as-decimal-separator and suffix-as-decimal-separator, optional whitespace and scientific notations makes it difficult to normalize values without using specialized libraries. Continue reading →

Posted by Uli Köhler in Electronics, Python

Using Arduino Leonardo as an USB/UART adapter

In contrasts to older designs like the Arduino Uno, the Arduino Leonardo features a separate connection Serial1 for TTLUART whereas Serial is used for the USB CDC UART interface.

This allows one to use the Leonardo as an USB/UART bridge without having to resort to more expensive boards like the Arduino Mega 2560. In order to do this, use this sketch which can also be modified to provide an intelligent UART bridge.

Remember to adjust the baudrate for your application. This version of the sketch does not support automatic baudrate selection via the CDC peripheral.

Continue reading →

Posted by Uli Köhler in C/C++, Embedded

Accessing lwIP struct netif instance in ChibiOS

Problem

You are using the ChibiOS lwIP binding to access the network your microcontroller application.

You need to access the lwIP struct netif structure, for example to get the current DHCP IP address assigned to the network interface. However, the default ChibiOS implementaton in lwipthread.c does not export the interface structure.
Continue reading →

Posted by Uli Köhler in C/C++, Embedded

Engineering for the super-lazy: Solving equations without activating your brain

Preface

In electronics engineering, from time to time you have to use standard formulas to characterize your circuits. To what extent you need to calculate all parameters most often depends on the requirement.

For example, consider the formula for the -3dB cutoff frequency of a 1st order RC lowpass filter:

f_c=\frac{1}{2\pi RC}

Although this equation is fairly simple and most people won’t have any problem solving it for any particular variable in a few seconds, it can serve as a basic example on how to solve an equation symbolically.

One of the easiest ways of performing this task is to use SymPy, a Python library for symbolic mathematics.

Continue reading →

Posted by Uli Köhler in Python

Reading the STM32 unique device ID in C

All STM32 microcontrollers feature a 96-bit factory-programmed unique device ID. However, for me it was hard to find an adequately licensed example on how to read it in a manner compatible with different families and compilers.

Here’s a simple header that defines a macro for the device ID address. While I checked the address for both STM32F4 and STM32F0 families, other families might have slightly different addresses for the device ID. Check the reference manual corresponding to your STM32 family if errors occur.

Continue reading →

Posted by Uli Köhler in C/C++, Embedded