TechOverflow

May

2021

RTC battery lifetime online calculator

Free calculator to estimate how long your battery (e.g. CR2032) will last in your microcontroller RTC application. Note: The default capacity of 220 mAh is a typical capacity for a CR2032 coin cells.

TechOverflow calculators:
You can enter values with SI suffixes like 12.2m (equivalent to 0.012) or 14k (14000) or 32u (0.000032).
The results are calculated while you type and shown directly below the calculator, so there is no need to press return or click on a Calculate button. Just make sure that all inputs are green by entering valid values.

Ah ⚠

A ⚠

May

2021

How to fix OpenWRT: Unknown package ‘…’

Problem:

You are trying to install an OpenWRT package using e.g.

opkg install luci-app-openvpn

but you see an error message like

Unknown package 'luci-app-openvpn'.
Collected errors:
 * opkg_install_cmd: Cannot install package luci-app-openvpn.

Solution:

There are two different reasons why this error can occur:

You forgot to run
```
opkg update
```
Note that you need to run opkg update after each reboot since the package lists are only stored in the RAM
The package actually doesn’t exist. Check
```
opkg list
```
for a list of packages that can be installed for your OpenWRT version and architecture. In very rare cases you might need to build a package from source if it isn’t available, or you might need to upgrade your OpenWRT version if that’s possible for your router.

May

2021

How I fixed OpenWRT LuCI error /etc/config/luci seems to be corrupt, unable to find section ‘main’

Problem:

When you try to open your OpenWRT Web UI, you see an error message like

/usr/lib/lua/luci/dispatcher.lua:427: /etc/config/luci seems to be corrupt, unable to find section 'main'

Solution:

In my case, the issue was a bad rpcd configuration which I restored from an older OpenWRT version that contained the config line

option socket /var/run/ubus.sock

instead of the correct

option socket /var/run/ubus/ubus.sock

You can replace your rpcd config by the default by running the following command over SSH:

cp /rom/etc/config/rpcd /etc/config/rpcd

and then

reboot

May

2021

How to fix GCC error: implicit declaration of function …

Problem:

While trying to compile your C/C++ program, you see an error message like

../src/main.c:48:9: error: implicit declaration of function 'StartBenchmark' [-Werror=implicit-function-declaration]
         StartBenchmark();

Solution:

implicit declaration of function means that you are trying to use a function that has not been declared. In our example above, StartBenchmark is the function that is implicitly declared.

This is how you call a function:

StartBenchmark();

This is how you declare a function:

void StartBenchmark();

The following bullet points list the most common reasons and how to fix them:

Missing #include: Check if the header file that contains the declaration of the function is #included in each file where you call the function (especially the file that is listed in the error message), before the first call of the function (typically at the top of the file). Header files can be included via other headers,
Function name typo: Often the function name of the declaration does not exactly match the function name that is being called. For example, startBenchmark() is declared while StartBenchmark() is being called. I recommend to fix this by copy-&-pasting the function name from the declaration to wherever you call it.
Bad include guard: The include guard that is auto-generated by IDEs often looks like this:
```
#ifndef _EXAMPLE_FILE_NAME_H
#define _EXAMPLE_FILE_NAME_H
// ...
#endif
```
Note that the include guard definition _EXAMPLE_FILE_NAME_H is not specific to the header filename that we are using (for example Benchmark.h). Just the first of all header file names wil
Change the order of the #include statements: While this might seem like a bad hack, it often works just fine. Just move the #include statements of the header file containing the declaration to the top. For example, before the move:
```
#include "Benchmark.h"
#include "other_header.h"
```
after the move:
```
#include "Benchmark.h"
#include "other_header.h"
```

May

2021

How to check / enable DHCP in Alpine Linux installer

Once you have booted from the Alpine Linux installer CD and logged in using root / <no password> as described in What is the Alpine linux default login & password?, one often wants to test if DHCP works before going through the installer and potentially having to repeat the process.

First enable Ethernet using

ifconfig eth0 up

Then run the DHCP client using

udhcpc eth0

This will show you the IP address of the lease that your Alpine Live CD did acquire.

May

2021

How to measure performance on the PIC32

First, use MPLab Harmony Configurator 3 to enable the CORETIMER module for your project. No special configuration is neccessary.

The PIC32 Core Timer always runs at half the CPU frequency. For example, if the CPU is running at 200 MHz, the Core Timer will run at 100 MHz.

You an then use

uint32_t CORETIMER_CounterGet();

to get the current value of the core timer. Additionally, you can get the frequency of the Core Timer in Hz using

uint32_t CORETIMER_FrequencyGet();

Use the following snippet to measure how long it takes to run a specific function:

uint32_t t0 = CORETIMER_CounterGet();
run_my_func();
uint32_t t1 = CORETIMER_CounterGet();

After that, you can compute the number of seconds it took to run the function using e.g.

uint32_t tdelta = t1 - t0;
float seconds = tdelta / (float)CORETIMER_FrequencyGet();

or the number of milliseconds:

uint32_t tdelta = t1 - t0;
float milliseconds = tdelta / (CORETIMER_FrequencyGet() / 1000.0);

or the number of microseconds:

uint32_t tdelta = t1 - t0;
float microseconds = tdelta / (CORETIMER_FrequencyGet() / 1000000.0);

May

2021

How to delay() by milliseconds / microseconds on the PIC32

First, use MPLab Harmony Configurator 3 to enable the CORETIMER module for your project. No special configuration is neccessary.

After that, use

CORETIMER_DelayMs(uint32_t delay_ms)

to delay for a given number of milliseconds or use

CORETIMER_DelayUs(uint32_t delay_us)

to delay for a given number of microseconds.

May

2021

How to fix PIC32 Curiosity Board gray / greyed out in MPLab X

Problem:

You want to program your PIC32 Curiosity board in MPLab X using the PicKit 4 on board (PKoB4) programmer that is included on the Curiosity board.

However, when you try to select the Curiosity board in the project settings, it is greyed out and you can only select the Simulator or No Tool.

Solution:

You need to reprogram the firmware of the PicKit On Board using the MPLab X IPE which is installed alongside the MPLab X IDE.

Open MPLab X IPE, open the Tools menu and select Hardware Tool Emergency Boot Firmware Recovery

After that, closely follow the instructions and reprogram the firmware for the PicKit On Board 4 programmer.

After reflashing the PKoB4 is successful, you will be able to select the Curiosity Board in MPLab X IDE.

May

2021

Where to set MPLab X PIC32 heap size

In order to set the heap size, use MPLab Harmony 3, click on System in the Project Graph, then open the tree as follows:

Device & Project configuration
Project configuration
Tool Chain Selections
XC32 Global Options
Linker
General

and set Heap Size (bytes) to the desired value.

Additionally, you need to set the heap size in the project options => XC32 options => xc32-ld options:

Apr

2021

Should you use static IP addresses or static DHCP leases?

I generally recommend static DHCP leases over static IP addresses for almost any circumstance.

There are three primary reasons for that:

DHCP will automatically configure gateway, DNS servers and other options – if you change your network configuration, this will avoid having to reconfigure your devices with static IP addresses.
Monitoring capability: You’ll be able to see if a device has requested the IP address from the DHCP server. This also means that you have an easy way
Failure tolerance: If you connect the device to the wrong network – for example, if you connect it to the wrong VLAN – it will still try to acquire an IP address. which you can see in the DHCP lease table of the router. This is often relevant when, for example, changing network components or moving equipment between locations. Quite often, it’s hard to access equipment that has an unknown IP address and a complete reset would also delete a lot of configuration work.

There are cases however, where a static IP address should be preferred:

DHCP server needs static IP: If the device is the DHCP server itself, obviously you need to set a static IP address (only on the interface where the DHCP server is enabled, of course)
Reliability: If the device is mission-critical, you might not want to depend on the DHCP server being online. You need to weigh this argument against the failure tolerance and monitoring capability arguments as listed above! Note than some devices will only try to acquire an IP address using DHCP once and will not retry if no DHCP server answers!
Network without DHCP server: If there is no DHCP server on the local network, obviously you need a static IP
Dynamic MAC address: If a device doesn’t use the same MAC address all the time

If your device supports DHCP with fallback, I recomend that you use a static lease plus a static IP fallback with the same IP address. This approach can combine the advantages of both static leases and static IP addresses.

Apr

2021

How to make your Matplotlib plot nicer in 15 seconds

Use plt.style.ggplot():

from matplotlib import pyplot as plt
plt.style.use("ggplot")

Before:

After:

Apr

2021

How to fix pandas to_sql() AttributeError: ‘DataFrame’ object has no attribute ‘cursor’

Problem:

You are trying to save your DataFrame in an SQL database using pandas to_sql(), but you see an exception like

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-21-3788db1a4131> in <module>
      7 db = sqlalchemy.create_engine('sqlite:///timeseries.db' class="ansi-blue-fg">)
      8 
----> 9 df.to_sql('timeseries', df)

~/miniconda3/lib/python3.8/site-packages/pandas/core/generic.py in to_sql(self, name, con, schema, if_exists, index, index_label, chunksize, dtype, method)
   2603         from pandas.io import sql
   2604 
-> 2605         sql.to_sql(
   2606             self,
   2607             name,

~/miniconda3/lib/python3.8/site-packages/pandas/io/sql.py in to_sql(frame, name, con, schema, if_exists, index, index_label, chunksize, dtype, method)
    587         )
    588 
--> 589     pandas_sql.to_sql(
    590         frame,
    591         name,

~/miniconda3/lib/python3.8/site-packages/pandas/io/sql.py in to_sql(self, frame, name, if_exists, index, index_label, schema, chunksize, dtype, method)
   1825             dtype=dtype,
   1826         )
-> 1827         table.create()
   1828         table.insert(chunksize, method)
   1829 

~/miniconda3/lib/python3.8/site-packages/pandas/io/sql.py in create(self)
    719 
    720     def create(self):
--> 721         if self.exists():
    722             if self.if_exists == "fail":
    723                 raise ValueError(f"Table '{self.name}' already exists.")

~/miniconda3/lib/python3.8/site-packages/pandas/io/sql.py in exists(self)
    706 
    707     def exists(self):
--> 708         return self.pd_sql.has_table(self.name, self.schema)
    709 
    710     def sql_schema(self):

~/miniconda3/lib/python3.8/site-packages/pandas/io/sql.py in has_table(self, name, schema)
   1836         query = f"SELECT name FROM sqlite_master WHERE type='table' AND name={wld};"
   1837 
-> 1838         return len(self.execute(query, [name]).fetchall()) > 0
   1839 
   1840     def get_table(self, table_name, schema=None):

~/miniconda3/lib/python3.8/site-packages/pandas/io/sql.py in execute(self, *args, **kwargs)
   1677             cur = self.con
   1678         else:
-> 1679             cur = self.con.cursor()
   1680         try:
   1681             cur.execute(*args, **kwargs)

~/miniconda3/lib/python3.8/site-packages/pandas/core/generic.py in __getattr__(self, name)
   5137             if self._info_axis._can_hold_identifiers_and_holds_name(name):
   5138                 return self[name]
-> 5139             return object.__getattribute__(self, name)
   5140 
   5141     def __setattr__(self, name: str, value) -> None:

AttributeError: 'DataFrame' object has no attribute 'cursor'

Solution:

You’re calling to_sql() with the wrong arguments! The second argument needs to be the database connection (e.g. an sqlalchemy engine)! You’re probably calling it like this:

df.to_sql('timeseries', df)

but the second argument needs to be db (or whatever your database connection object is named), not df!

Full working example for `to_sql()`

import pandas as pd
# Load pre-built time series example dataset
df = pd.read_csv("https://datasets.techoverflow.net/timeseries-example.csv", parse_dates=["Timestamp"])
df.set_index("Timestamp", inplace=True)

import sqlalchemy
db = sqlalchemy.create_engine('sqlite:///timeseries.db')

df.to_sql('timeseries', db, if_exists="replace")

Apr

2021

How to export Pandas dataset to SQLite database

In our previous post we showed how to connect to an SQLite database using sqlalchemy.

In this blogpost, we’ll show how you can export a pandas DataFrame – for example, our time series example dataset – to the SQLite database.

First, we’ll load the example data frame:

import pandas as pd
# Load pre-built time series example dataset
df = pd.read_csv("https://datasets.techoverflow.net/timeseries-example.csv", parse_dates=["Timestamp"])
df.set_index("Timestamp", inplace=True)

Now we can open the SQLite database as shown in our previous post

import sqlalchemy
db = sqlalchemy.create_engine('sqlite:///timeseries.db')

and export the DataFrame to the database:

df.to_sql('timeseries', db, if_exists="replace")

I always recommend using if_exists="replace" (i.e. if the table already exists, replace it) for a quicker development process.

The database looks like this when viewed in an SQLite viewer like HeidiSQL:

Complete code example

import pandas as pd
# Load pre-built time series example dataset
df = pd.read_csv("https://datasets.techoverflow.net/timeseries-example.csv", parse_dates=["Timestamp"])
df.set_index("Timestamp", inplace=True)

import sqlalchemy
db = sqlalchemy.create_engine('sqlite:///timeseries.db')

df.to_sql('timeseries', db, if_exists="replace")

Apr

2021

How to create SQLite database using SQLAlchemy

The following code will create my-sqlite.db in the current directory using sqlalchemy:

import sqlalchemy
db = sqlalchemy.create_engine('sqlite:///my-sqlite.db')

Note that you need three slashes in sqlite:/// in order to use a relative path for the DB. If you want an absolute path, use four slashes: sqlite:////.

Apr

2021

How to make bounding box larger by a percentage in Python

If we have a bounding box (xmin, xmax, ymin, ymax), we can use the following algorithm to resize the bounding box by e.g. 15% on each side:

xmin -= 0.15 * (xmax - xmin)
xmax += 0.15 * (xmax - xmin)
ymin -= 0.15 * (ymax - ymin)
ymax += 0.15 * (ymax - ymin)

Note that this will reisze the bounding box by 15% (=0.15) on each side, i.e. the total width of the resulting bounding box will be 15% * 2 = 30% larger!

Why it works

xmin -= 0.15 * (xmax - xmin)

is the same as

xmin -= 0.15 * [width of the bounding box]

Apr

2021

How to get bounding box of a country using Natural Earth data and Cartopy

In this example, we’ll determine the bounding box of Kenya using the public domain Natural Earth dataset and the Cartopy library.

Rendering just the bounding box of Kenya (with the actual country being highlighted in green) looks like this:

How to get the bounding box

First we use Cartopy’s cartopy.io.shapereader.natural_earth() function that will automatically download Natural Earth data (if it has already been downloaded, the cached data will be used):

shpfilename = shpreader.natural_earth(resolution='10m',
                                      category='cultural',
                                      name='admin_0_countries')
reader = shpreader.Reader(shpfilename)

Now we can filter for Kenya just like we did in our previous post on How to highlight a specific country using Cartopy:

kenya = [country for country in reader.records() if country.attributes["NAME_LONG"] == "Kenya"][0]

and get the bounding box using kenya.bounds:

lon_min, lat_min, lon_max, lat_max = kenya.bounds

Complete example code

This code will render the image shown above:

import cartopy.crs as ccrs
import cartopy.feature as cf
from cartopy.feature import ShapelyFeature
from matplotlib import pyplot as plt

proj = ccrs.PlateCarree()
ax = plt.axes(projection=proj)
# Show only Africa
#ax.set_extent([-23, 55, -35, 40])
ax.stock_img()

ax.add_feature(cf.COASTLINE, lw=2)
# Make figure larger
plt.gcf().set_size_inches(20, 10)

import cartopy.io.shapereader as shpreader
# Read shape file
shpfilename = shpreader.natural_earth(resolution='10m',
                                      category='cultural',
                                      name='admin_0_countries')
reader = shpreader.Reader(shpfilename)
# Filter for a specific country
kenya = [country for country in reader.records() if country.attributes["NAME_LONG"] == "Kenya"][0]
# Determine bounding box
lon_min, lat_min, lon_max, lat_max = kenya.bounds
ax.set_extent([lon_min, lon_max, lat_min, lat_max])

# Display Kenya's shape
shape_feature = ShapelyFeature([kenya.geometry], ccrs.PlateCarree(), facecolor="lime", edgecolor='black', lw=1)
ax.add_feature(shape_feature)

# Save figure as SVG
plt.savefig("Kenya-Bounding-Box-Tight.svg")

Apr

2021

How to plot Shapefile data in Cartopy

In order to display shapefile data in Cartopy, we can first use the cartopy.io.shapereader package to read the shape data and then convert the geometry we want to display to a cartopy.feature.ShapelyFeature.

In the following example, we’ll read the Natural Earth ne_110m_admin_0_countries.shp and
Note that there’s an easier way to plot Natural Earth data using shpreader.natural_earth – see How to highlight a specific country using Cartopy and we’ll use the Natural Earth dataset just as an example!

import cartopy.io.shapereader as shpreader
# Read shape file
reader = shpreader.Reader("ne_110m_admin_0_countries.shp")
# Filter for a specific country
kenya = [country for country in reader.records() if country.attributes["NAME_LONG"] == "Kenya"][0]

# Display Kenya's shape
shape_feature = ShapelyFeature([kenya.geometry], ccrs.PlateCarree(), facecolor="lime", edgecolor='black', lw=1)
ax.add_feature(shape_feature)

Complete code example

import cartopy.crs as ccrs
import cartopy.feature as cf
from cartopy.feature import ShapelyFeature
from matplotlib import pyplot as plt

proj = ccrs.PlateCarree()
ax = plt.axes(projection=proj)
# Show only Africa
ax.set_extent([-23, 55, -35, 40])
ax.stock_img()

ax.add_feature(cf.COASTLINE, lw=2)
# Make figure larger
plt.gcf().set_size_inches(20, 10)

import cartopy.io.shapereader as shpreader
# Read shape file
reader = shpreader.Reader("ne_110m_admin_0_countries.shp")
# Filter for a specific country
kenya = [country for country in reader.records() if country.attributes["NAME_LONG"] == "Kenya"][0]

# Display Kenya's shape
shape_feature = ShapelyFeature([kenya.geometry], ccrs.PlateCarree(), facecolor="lime", edgecolor='black', lw=1)
ax.add_feature(shape_feature)

# Save figure as SVG
plt.savefig("Africa-Highlight-Kenya.svg")

Apr

2021

How to highlight a specific country using Cartopy

In our previous posts, e.g. How to draw Africa map using Cartopy we showed how to draw an overview map of an entire continent using Cartopy. This post provides an example of how to highlight a specific country in that map. In this example, we’ll highlight Kenya

The general approach is:

Use cartopy.io.shapereader.natural_earth to download Natural Earth data that contains the shape of Kenya
Convert it to a cartopy.feature.ShapelyFeature
Display said feature

Displaying Kenya’s Natural Earth shape in cartopy

First, we create a Reader for the Natural Earth data. Cartopy will automatically download the data if it has not been cached.

import cartopy.io.shapereader as shpreader

shpfilename = shpreader.natural_earth(resolution='110m',
                                      category='cultural',
                                      name='admin_0_countries')
reader = shpreader.Reader(shpfilename)

Now we can select Kenya by name from the records:

kenya = [country for country in reader.records() if country.attributes["NAME_LONG"] == "Kenya"][0]

In order to display that geometry, we use

shape_feature = ShapelyFeature([kenya.geometry], ccrs.PlateCarree(), facecolor="lime", edgecolor='black', lw=1)
ax.add_feature(shape_feature)

Complete example code

import cartopy.crs as ccrs
import cartopy.feature as cf
from cartopy.feature import ShapelyFeature
from matplotlib import pyplot as plt

proj = ccrs.PlateCarree()
ax = plt.axes(projection=proj)
# Show only Africa
ax.set_extent([-23, 55, -35, 40])
ax.stock_img()

ax.add_feature(cf.COASTLINE, lw=2)
# Make figure larger
plt.gcf().set_size_inches(20, 10)

# Read Natural Earth data
import cartopy.io.shapereader as shpreader

shpfilename = shpreader.natural_earth(resolution='110m',
                                      category='cultural',
                                      name='admin_0_countries')
reader = shpreader.Reader(shpfilename)
kenya = [country for country in reader.records() if country.attributes["NAME_LONG"] == "Kenya"][0]

# Display Kenya's shape
shape_feature = ShapelyFeature([kenya.geometry], ccrs.PlateCarree(), facecolor="lime", edgecolor='black', lw=1)
ax.add_feature(shape_feature)

# Save figure as SVG
plt.savefig("Africa-Highlight-Kenya.svg")

Apr

2021

How to draw Europe map using Cartopy

We can easily draw an Africa Map using Cartopy by setting the extents to [-13, 45, 30, 70]:

ax.set_extent([-13, 45, 30, 70])

Complete code example

The code above produces the image shown above:

import cartopy.crs as ccrs
import cartopy.feature as cf
from matplotlib import pyplot as plt

proj = ccrs.Miller()
ax = plt.axes(projection=proj)
ax.set_extent([-13, 45, 30, 70])
ax.stock_img()

ax.add_feature(cf.COASTLINE, lw=2)
ax.add_feature(cf.BORDERS)
# Make figure larger
plt.gcf().set_size_inches(20, 10)

# Save figure as SVG
plt.savefig("Europe.svg")

Apr

2021

How to make Cartopy coastline or border lines thicker (line width)

By standard, Cartopy draws every feature with the same line width:

ax.add_feature(cf.COASTLINE)
ax.add_feature(cf.BORDERS)

We can easily increasing the line width by adding e.g. lw=2 to the ax.add_feature() call:

ax.add_feature(cf.COASTLINE, lw=2)
ax.add_feature(cf.BORDERS)

Complete code example

This example produces the image with a wider coast line line width as shown above

import cartopy.crs as ccrs
import cartopy.feature as cf
from matplotlib import pyplot as plt

proj = ccrs.PlateCarree()
ax = plt.axes(projection=proj)
ax.set_extent([-23, 55, -35, 40])
ax.stock_img()

ax.add_feature(cf.COASTLINE, lw=2)
ax.add_feature(cf.BORDERS)
# Make figure larger
plt.gcf().set_size_inches(20, 10)

# Save figure as SVG
plt.savefig("Africa-Standard.svg")

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Full working example for to_sql()

Complete code example

Why it works

How to get the bounding box

Complete example code

Complete code example

Displaying Kenya’s Natural Earth shape in cartopy

Complete example code

Complete code example

Complete code example

Search

Categories

Full working example for `to_sql()`