How to fix ESP32 MicroPython ‘ValueError: pin can only be input’

Problem:

You are trying to initialize an ESP32 pin in MicroPython using

import machine
machine.Pin(34, machine.Pin.OUT)

but you see an error message like

>>> machine.Pin(34, machine.Pin.OUT)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: pin can only be input

Solution:

On the ESP32, pins with numbers >= 34 are input-only pins!

You need to use other pins < 34 if you need output capability!

For reference, see the relevant MicroPython source code section:

// configure mode
if (args[ARG_mode].u_obj != mp_const_none) {
    mp_int_t pin_io_mode = mp_obj_get_int(args[ARG_mode].u_obj);
    if (self->id >= 34 && (pin_io_mode & GPIO_MODE_DEF_OUTPUT)) {
        mp_raise_ValueError("pin can only be input");
    } else {
        gpio_set_direction(self->id, pin_io_mode);
    }
}
Posted by Uli Köhler in Embedded, MicroPython, Python

How to upload files to MicroPython over USB/serial

In this post we will investigate how to connect to a wireless network on boot

First, install ampy – a tool to modify the MicroPython filesystem over a serial connection.

sudo pip3 install adafruit-ampy

Now prepare your script – we’ll use main.py in this example.

Upload the file to the board:

ampy -p /dev/ttyUSB* put main.py

This only takes about 2-4 seconds. In case ampy is still running after 10 seconds, you might need to

  • Stop ampy (Ctrl+C), reset the board using the RESET button and retry the command
  • Stop ampy (Ctrl+C). Detach USB and ensure the board is powered off (and not powered externally). Re-Attach USB and retry the command.
  • In case that doesn’t help, try re-flashing your board with the most recent version of MicroPython. See How to flash MicroPython to your ESP32 board in 30 seconds. This will also clear the internal filesystem and hence remove any file that might cause failure to boot properly.
Posted by Uli Köhler in Embedded, MicroPython, Python

MicroPython ESP32 blink example

This MicroPython code blinks GPIO2 which is connected to the LED on most ESP32 boards.

import machine
import time
led = machine.Pin(2, machine.Pin.OUT)
while True:
    led.value(1)
    time.sleep(1)
    led.value(0)
    time.sleep(1)

Don’t know how to upload the file to MicroPython so it is automatically run on boot?

Posted by Uli Köhler in Embedded, MicroPython, Python

How to upload files to MicroPython using WebREPL using webrepl_cli.py

First, clone the webrepl repository:

git clone https://github.com/micropython/webrepl.git

Now use ampy to initially setup your wifi connection and setup both wifi and WebREPL on on boot (see How to autoconnect to Wifi using MicroPython on your ESP32 board):

import network
station = network.WLAN(network.STA_IF)
station.active(True)
station.connect("MyWifi", "MyWifiPassword")

# Start webrepl
import webrepl
webrepl.start(password="Rua8ohje")

Upload via the serial port using

ampy -p /dev/ttyUSB0 put main.py

You should only have to do this once (except if you break you main.py). If the upload doesn’t work

Go to the directory where webrepl_cli.py is located:

cd webrepl

Now you can upload main.py using

./webrepl_cli.py -p Rua8ohje ../main.py espressif.local:/main.py

You might need to use a different host, but espressif.local seems to work out-of-the-box in many configurations.

The output should look like this:

op:put, host:espressif.local, port:8266, passwd:Rua8ohje.
../main.py -> /main.py
Remote WebREPL version: (1, 12, 0)
Sent 329 of 329 bytes

Now, reset your board using the reset button, so your updated main.py will be executed.

Posted by Uli Köhler in Embedded, MicroPython

How to run WebREPL without webrepl_setup in MicroPython

Problem:

You want to enable WebREPL on your MicroPython board using

import webrepl
webrepl.start()

but it is only showing this error message:

WebREPL is not configured, run 'import webrepl_setup'

However, you want to configure WebREPL programmatically instead of manually running it on every single board.

Solution:

Use

import webrepl
webrepl.start(password="Rua8ohje")

This will circumvent webrepl_setup completely and is compatible with an automated setup process.

Note: At the time of writing this you can only use passwords with 8 characters max! (see How to fix MicroPython WebREPL ValueError in File &#8222;webrepl.py&#8220;, line 72, in start )

Posted by Uli Köhler in Embedded, MicroPython, Python

How to fix MicroPython WebREPL ValueError in File “webrepl.py”, line 72, in start

Problem:

You want to configure your MicroPython WebREPL programmatically using webrepl.start(password="...") but you see a stacktrace like

>>> webrepl.start(password="Rua8ohjedo")
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "webrepl.py", line 72, in start
ValueError:

Solution:

Use a shorter password with 8 characters max:

webrepl.start(password="Rua8ohje")

 

Posted by Uli Köhler in Embedded, MicroPython, Python

How to autoconnect to Wifi using MicroPython on your ESP32 board

In this post we will investigate how to connect to a wireless network on boot

First, install ampy – a tool to modify the MicroPython filesystem over a serial connection.

sudo pip3 install adafruit-ampy

Now download main.py and save it in your current working directory and insert your wifi credentials:

import network
station = network.WLAN(network.STA_IF)
station.active(True)
station.connect("YourWifiName", "EnterYourWifiPasswordHere")

Upload the file to the board:

ampy -p /dev/ttyUSB* put main.py

In case ampy shows no output within 5 seconds, try resetting the board, waiting for 5-10 seconds and retrying the upload using ampy

Note: You can list the files on the board’s filesystem using

ampy -p /dev/ttyUSB0 ls

You can verify the content of main.py using

ampy -p /dev/ttyUSB0 get main.py
Posted by Uli Köhler in Embedded, MicroPython, Python

Lehman scale fee calculator

Calculate broker fees using the origin Lehman scale, double Lehman scale and modern Lehman scale using this online calculator.

TechOverflow calculators:
You can enter values with SI suffixes like 12.2m (equivalent to 0.012) or 14k (14000) or 32u (0.000032).
The results are calculated while you type and shown directly below the calculator, so there is no need to press return or click on a Calculate button. Just make sure that all inputs are green by entering valid values.

$



















Posted by Uli Köhler in Calculators, Economics

How to access COM1, COM2, … serial ports in Windows Subsystem for Linux (WSL)?

Accessing serial ports in WSL is really simple:

  • COM1 is mapped to /dev/ttyS1
  • COM2 is mapped to /dev/ttyS2
  • COM3 is mapped to /dev/ttyS3
  • COM4 is mapped to /dev/ttyS4
Posted by Uli Köhler in Windows

How to connect your ESP32 MicroPython board to your Wifi in 20 seconds

Didn’t flash MicroPython on your ESP32 board yet? See How to flash MicroPython to your ESP32 board in 30 seconds

Copy this script and enter your Wifi credentials:

import network
station = network.WLAN(network.STA_IF)
station.active(True)
station.connect("YourWifiName", "EnterYourWifiPasswordHere")

Now you need to open a REPL to MicroPython via USB.

If your Wifi is within range of the board and the password is correct, your should see output like

>>> import network
>>> station = network.WLAN(network.STA_IF)
I (457700) wifi: wifi driver task: 3ffd2a80, prio:23, stack:3584, core=0
I (459702) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (459712) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (459832) wifi: wifi firmware version: 10f4364
I (459832) wifi: config NVS flash: enabled
I (459832) wifi: config nano formating: disabled
I (459832) wifi: Init dynamic tx buffer num: 32
I (459842) wifi: Init data frame dynamic rx buffer num: 32
I (459842) wifi: Init management frame dynamic rx buffer num: 32
I (459852) wifi: Init management short buffer num: 32
I (459852) wifi: Init static rx buffer size: 1600
I (459862) wifi: Init static rx buffer num: 10
I (459862) wifi: Init dynamic rx buffer num: 32
>>> station.active(True)
W (459872) phy_init: failed to load RF calibration data (0x1102), falling back to full calibration
I (460012) phy: phy_version: 4102, 2fa7a43, Jul 15 2019, 13:06:06, 0, 2
I (460062) wifi: mode : sta (24:6f:28:b0:28:b4)
True
I (460062) wifi: STA_START
>>> station.connect("MyWifi", "ThisIsMyWifiPassword")
>>> I (461782) wifi: new:<5,0>, old:<1,0>, ap:<255,255>, sta:<5,0>, prof:1
I (462632) wifi: state: init -> auth (b0)
I (462632) wifi: state: auth -> assoc (0)
I (462642) wifi: state: assoc -> run (10)
I (462672) wifi: connected with MyWifi, channel 5, BW20, bssid = 9a:c7:44:33:22:11
I (462672) wifi: pm start, type: 1

I (462672) network: CONNECTED
I (463672) tcpip_adapter: sta ip: 192.168.178.42, mask: 255.255.255.0, gw: 192.168.178.1
I (463672) network: GOT_IP
>>>

Note that the output might take a few seconds to appear since it might take some time to connect to the access point.

Note the IP address (192.168.178.42 in this example)

Posted by Uli Köhler in Embedded, MicroPython

How to get into a MicroPython REPL using USB in 10 seconds

On Linux, use picocom:

picocom /dev/ttyUSB0 -b115200

and press enter to get a REPL (or press the RESET button on the board if that does not work).

You should now see the

>>>

prompt.

Don’t have picocom installed?

sudo apt -y install picocom

Got permission denied errors?

sudo usermod -a -G dialout $USER

then logout and log back in or reboot!

Posted by Uli Köhler in MicroPython

How to flash MicroPython to your ESP32 board in 30 seconds

This script does all the neccessary steps to flash MicroPython to your ESP32 board on Linux. Run this script with your board plugged in via USB:

#!/bin/sh
# Call without arguments
# Download esptool
git clone https://github.com/espressif/esptool.git -b v2.8
cd esptool
# Erase flash. Press the reset button while Connecting.
python3 esptool.py --chip esp32 --port /dev/ttyUSB* erase_flash
# Download firmware
wget -qO micropython.bin https://micropython.org/resources/firmware/esp32-idf4-20191220-v1.12.bin
# Upload firmware to board
python3 esptool.py --chip esp32 --port /dev/ttyUSB* --baud 460800 write_flash -z 0x1000 micropython.bin

Remember to press the RESET button on the board if the script is telling you Connecting.... Not sure which button it is? Just press either one, and if the output doesn’t show Chip is ESP32 within 2 seconds, press the other one.

This script was built for my ESP-WROOM-32 board.

Now you can try to get into a REPL using USB.

Posted by Uli Köhler in Embedded, MicroPython

How to fix Nextcloud upgrade Migration step ‘…’ is unknown

Problem:

When upgrading your Nextcloud instance, you see an error message like

Migration step 'OCA\Files\Migration\Version11301Date20191205150729' is unknown

Solution:

This error occurs because you just copied over multiple versions of the apps directory, which resulted in

First, make a backup of your apps directory (located inside your Nextcloud folder).

Then delete your current apps directory and replace it by the apps directory from the new version of Nextcloud you want to upgrade to.

After that, retry the upgrade.

Posted by Uli Köhler in Nextcloud

How to fix NextCloud OnlyOffice MixedContent or ‘Refused to frame ‘http://…’ because it violates the following Content Security Policy directive: “frame-src https://…”.

Problem:

In reverse-proxy setups  forwarding requests to OnlyOffice like our reference setup there you might encounter issues like

Refused to frame 'http://onlyoffice.mydomain.com/' because it violates the following Content Security Policy directive: "frame-src https://onlyoffice.mydomain.com/".

Solution:

Just add

proxy_set_header X-Forwarded-Proto $scheme;

directly after your proxy_pass clause in your nginx config, then run sudo service nginx reload.

The reason for this issue is that OnlyOffice thinks it’s being loaded using HTTP, but the Nextcloud page prevents insecure content from being loaded.

Using a proxy other than nginx? Just ensure that every proxied request (i.e. every request directed towards the OnlyOffice instance) has the X-Forwarded-Proto header set to the protocol of the original request – which should be https.

Posted by Uli Köhler in Nextcloud, nginx

How to setup OnlyOffice using docker-compose, systemd and nginx

In this setup we show how to setup OnlyOffice using nginx as a reverse proxy, docker-compose to run and configure the OnlyOffice image and systemd to automatically start and restart the OnlyOffice instance. Running it in a reverse proxy configuration allows you to have other domains listening on the same IP address and have a central management of Let’s Encrypt SSL certificates.

We will setup the instance in /opt/onlyoffice on port 2291.

Save this file as /opt/onlyoffice/docker-compose.yml and don’t forget to change JWT_SECRET to a random password!

version: '3'
services:
  onlyoffice-documentserver:
    image: onlyoffice/documentserver:latest
    restart: always
    environment:
      - JWT_ENABLED=true
      - JWT_SECRET=Shei9AifuZ4ze7udahG2seb3aa6ool
    ports:
      - 2291:80
    volumes:
      - ./onlyoffice/data:/var/www/onlyoffice/Data
      - ./onlyoffice/lib:/var/lib/onlyoffice
      - ./onlyoffice/logs:/var/log/onlyoffice
      - ./onlyoffice/db:/var/lib/postgresql

Now we can create the systemd service. I created it using TechOverflow’s docker-compose systemd .service generator. Save it in /etc/systemd/system/OnlyOffice.service:

[Unit]
Description=OnlyOffice
Requires=docker.service
After=docker.service

[Service]
Restart=always
User=root
Group=docker
# Shutdown container (if running) when unit is stopped
ExecStartPre=/usr/local/bin/docker-compose -f /opt/onlyoffice/docker-compose.yml down
# Start container when unit is started
ExecStart=/usr/local/bin/docker-compose -f /opt/onlyoffice/docker-compose.yml up
# Stop container when unit is stopped
ExecStop=/usr/local/bin/docker-compose -f /opt/onlyoffice/docker-compose.yml down

[Install]
WantedBy=multi-user.target

Now we can enable & start the service using

sudo systemctl enable OnlyOffice.service
sudo systemctl start OnlyOffice.service

Now let’s create the nginx config in /etc/nginx/sites-enabled/OnlyOffice.conf. Obviously, you’ll have to modify at least the

 server {
    server_name onlyoffice.mydomain.com;

    access_log /var/log/nginx/onlyoffice.access_log;
    error_log /var/log/nginx/onlyoffice.error_log info;

    location / {
        proxy_pass http://127.0.0.1:2291;
        proxy_http_version 1.1;
        proxy_read_timeout 3600s;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        add_header X-Frontend-Host $host;
    }

    listen 80;
}

Check the validity of the nginx config using

sudo nginx -t

and unless it fails, reload nginx using

sudo service nginx reload

Now I recommend to use certbot to enable TLS encryption on your domain. You should be familiar with these steps already ; my approach is to sudo apt -y install python-certbot-nginx, then certbot --nginx --staging to first obtain a staging certificate to avoid being blocked if there are any issues and after you have obtained the staging certificate use certbot --nginx and Renew & replace cert. After that, run sudo service nginx reload and check if you domain works with HTTPS. You should always choose redirection to HTTPS if certbot asks you.

Posted by Uli Köhler in Docker, nginx

How to fix Nextcloud Refused to send form data to /login/v2/grant because it violates the following Content Security Policy directive: form-action ‘self’

Problem:

When trying to connect using the Nextcloud client, during the Flow v2 authorization step where you open a page in the browser to authorize the client, you see an error message in the JS console like

Refused to send form data to 'http://nextcloud.mydomain.com/login/v2/grant' because it violates the following Content Security Policy directive: "form-action 'self'".

Solution:

Add

'overwriteprotocol' => 'https',

after this line:

'version' => '18.0.0.10',

in your nextcloud/config/config.php.

Posted by Uli Köhler in Nextcloud, PHP

Auto-generate nginx forwarding configs using this script

A major hassle for me when setting up lots of docker-based services on a machine is to setup the individual nginx configs to forward the domains to the correct services.

TechOverflow provides a simple way to automatically generate nginx configs for a single domain to configure port forwarding to a specific port.

wget -qO- https://techoverflow.net/scripts/generate-nginx-docker-config.sh | sudo bash /dev/stdin service.mydomain.com 1234

Note: This script was tested on Ubuntu 18.04 and is regularly used by myself and others. However, if used incorrectly or in case there is a major bug, it might damage your webserver configuration, so be sure to be prepared to fix any issues that might arise. This script is provided without any warranty, expressed or implied.

Remember to replace service.mydomain.com by your domain and 1234 by the local port your docker service is listening to.

The script will generate /etc/nginx/sites-enabled/[domain].conf.

Run

sudo nginx -t

to check if the config syntax is OK, and only if that succeeds, run

sudo service nginx reload

Now your domain should be online and I recommend running

sudo certbot --nginx

 

Posted by Uli Köhler in nginx

How to read IDF diabetes statistics in Python using Pandas

The International Diabetes Foundation provides a Data portal with various statistics related to diabetes.

In this post we’ll show how to read the Diabetes estimates (20-79 y) / People with diabetes, in 1,000s data export in CSV format using pandas.

First download IDF (people-with-diabetes--in-1-000s).csv from the data page.

Now we can parse the CSV file:

import pandas as pd

# Download at https://www.diabetesatlas.org/data/en/indicators/1/
df = pd.read_csv("IDF (people-with-diabetes--in-1-000s).csv")
# Parse year columns to obtain floats and multiply by thousands factor. Pandas fails to parse values like "12,345.67"
for column in df.columns:
    try:
        int(column)
        df[column] = df[column].apply(lambda s: None if s == "-" else float(s.replace(",", "")) * 1000)
    except:
        pass

As you can see in the postprocessing step, the number of diabetes patients are given in 1000s in the CSV, so we multiply them by 1000 to obtain the actual numbers.

If you want to modify the data columns (i.e. the columns referring to year), you can use this simple template:

for column in df.columns:
    try:
        int(column) # Will raise ValueError() if column is not a year number
        # Whatever you do here will only be applied to year columns
        df[column] = df[column] * 0.75 # Example on how to modify a column
        # But note that if your code raises an Exception, it will be ignored!
    except:
        pass

Let’s plot some data:

regions = df[df["Type"] == "Region"] # Only regions, not individual countries

from matplotlib import pyplot as plt
plt.style.use("ggplot")
plt.gcf().set_size_inches(20,4)
plt.ylabel("Diabetes patients [millions]")
plt.xlabel("Region")
plt.title("Diabetes patients in 2019 by region")
plt.bar(regions["Country/Territory"], regions["2019"] / 1e6)

Note that if you use a more recent dataset than the version I’m using the 2019 column might not exist in your CSV file. Choose an appropriate column in that case.

Posted by Uli Köhler in Bioinformatics, pandas, Python

How to repair docker-compose MariaDB instances (aria_chk -r)

Problem:

You are trying to run a MariaDB container using docker-compose. However, the database container doesn’t start up and you see error messages like these in the logs:

[ERROR] mysqld: Aria recovery failed. Please run aria_chk -r on all Aria tables and delete all aria_log.######## files
[ERROR] Plugin 'Aria' init function returned error.
[ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
....
[ERROR] Could not open mysql.plugin table. Some plugins may be not loaded
[ERROR] Failed to initialize plugins.
[ERROR] Aborting

Solution:

The log messages already tell you what to do – but they don’t tell you how to do it:

Aria recovery failed. Please run aria_chk -r on all Aria tables and delete all aria_log.######## files

First, backup the entire MariaDB data directory: Check onto which host directory the data directory (/var/lib/mysql) of the container is mapped and copy the entire directory to a backup space. This is important in case the repair process fails.

Now let’s run aria_chk -r to check and repair MySQL table files.

docker-compose run my-db bash -c 'aria_chk -r /var/lib/mysql/**/*'

Replace my-db by the name of your database container. This will attempt to repair a lot of non-table-files as well but aria_chk will happily ignore those.

Now we can delete the log files:

docker-compose run my-db bash -c 'rm /var/lib/mysql/aria_log.*'

Again, replace my-db by the name of your database container.

Posted by Uli Köhler in Databases, Docker

Parsing World Population Prospects (WPP) XLSX data in Python

The United Nations provides the Word Population Prospects (WPP) dataset on geographic and age distribution of mankind as downloadable XLSX files.

Reading these files in Python is rather easy. First we have to find out how many rows to skip. For the 2019 WPP dataset this value is 16 since row 17 contains all the column headers. The number of rows to skip might be different depending on the dataset. We’re using WPP2019_POP_F07_1_POPULATION_BY_AGE_BOTH_SEXES.xlsx in this example.

We can use Pandas read_excel() function to import the dataset in Python:

import pandas as pd

df = pd.read_excel("WPP2019_INT_F03_1_POPULATION_BY_AGE_ANNUAL_BOTH_SEXES.xlsx", skiprows=16, na_values=["..."])

This will take a few seconds until the large dataset has been processed. Now we can check if skiprows=16 is the correct value. It is correct if pandas did recognize the column names correctly:

>>> df.columns
Index(['Index', 'Variant', 'Region, subregion, country or area *', 'Notes',
       'Country code', 'Type', 'Parent code', 'Reference date (as of 1 July)',
       '0-4', '5-9', '10-14', '15-19', '20-24', '25-29', '30-34', '35-39',
       '40-44', '45-49', '50-54', '55-59', '60-64', '65-69', '70-74', '75-79',
       '80-84', '85-89', '90-94', '95-99', '100+'],
      dtype='object')

Now let’s filter for a country:

russia = df[df["Region, subregion, country or area *"] == 'Russian Federation']

This will show us the population data for multiple years in 5-year intervals from 1950 to 2020. Now let’s filter for the most recent year:

russia.loc[russia["Reference date (as of 1 July)"].idxmax()]

This will show us a single dataset:

Index                                                 3255
Variant                                          Estimates
Region, subregion, country or area *    Russian Federation
Notes                                                  NaN
Country code                                           643
Type                                          Country/Area
Parent code                                            923
Reference date (as of 1 July)                         2020
0-4                                                9271.69
5-9                                                9350.92
10-14                                              8174.26
15-19                                              7081.77
20-24                                               6614.7
25-29                                              8993.09
30-34                                              12543.8
35-39                                              11924.7
40-44                                              10604.6
45-49                                              9770.68
50-54                                              8479.65
55-59                                                10418
60-64                                              10073.6
65-69                                              8427.75
70-74                                              5390.38
75-79                                              3159.34
80-84                                              3485.78
85-89                                              1389.64
90-94                                              668.338
95-99                                              102.243
100+                                                 9.407
Name: 3254, dtype: object
​

How can we plot that data? First, we need to select all the columns that contain age data. We’ll do this by manually inserting the name of the first such column (0-4) into the following code and assuming that there are no columns after the last age column:

>>> df.columns[df.columns.get_loc("0-4"):]
Index(['0-4', '5-9', '10-14', '15-19', '20-24', '25-29', '30-34', '35-39',
       '40-44', '45-49', '50-54', '55-59', '60-64', '65-69', '70-74', '75-79',
       '80-84', '85-89', '90-94', '95-99', '100+'],
      dtype='object')

Now let’s select those columns from the russia dataset:

most_recent_russia = russia.loc[russia["Reference date (as of 1 July)"].idxmax()]
age_columns = df.columns[df.columns.get_loc("0-4"):]

russian_age_data = most_recent_russia[age_columns]

Let’s have a look at the dataset:

>>> russian_age_data
0-4      9271.69
5-9      9350.92
10-14    8174.26
15-19    7081.77
20-24     6614.7
25-29    8993.09
30-34    12543.8
35-39    11924.7
40-44    10604.6
45-49    9770.68
50-54    8479.65
55-59      10418
60-64    10073.6
65-69    8427.75
70-74    5390.38
75-79    3159.34
80-84    3485.78
85-89    1389.64
90-94    668.338
95-99    102.243
100+       9.407

That looks useable, note however that the values are in thousands, i.e. we have to multiply the values by 1000 to obtain the actual estimates of the population. Let’s plot it:

from matplotlib import pyplot as plt
plt.style.use("ggplot")

plt.title("Age composition of the Russian population (2020)")
plt.ylabel("People in age group [Millions]")
plt.xlabel("Age group")
plt.gcf().set_size_inches(15,5)
# Data is given in thousands => divide by 1000 to obtain millions
plt.plot(russian_age_data.index, russian_age_data.as_matrix() / 1000., lw=3)

The finished plot will look like this:

Here’s our finished script:

#!/usr/bin/env python3
import pandas as pd
df = pd.read_excel("WPP2019_POP_F07_1_POPULATION_BY_AGE_BOTH_SEXES.xlsx", skiprows=16)
# Filter only russia
russia = df[df["Region, subregion, country or area *"] == 'Russian Federation']

# Filter only most recent estimate (1 row)
most_recent_russia = russia.loc[russia["Reference date (as of 1 July)"].idxmax()]
# Retain only value columns
age_columns = df.columns[df.columns.get_loc("0-4"):]
russian_age_data = most_recent_russia[age_columns]

# Plot!
from matplotlib import pyplot as plt
plt.style.use("ggplot")

plt.title("Age composition of the Russian population (2020)")
plt.ylabel("People in age group [Millions]")
plt.xlabel("Age group")
plt.gcf().set_size_inches(15,5)
# Data is given in thousands => divide by 1000 to obtain millions
plt.plot(russian_age_data.index, russian_age_data.as_matrix() / 1000., lw=3)

# Export as SVG
plt.savefig("russian-demographics.svg")

 

 

Posted by Uli Köhler in Bioinformatics, Data science, pandas, Python
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Cookie settingsACCEPTPrivacy &amp; Cookies Policy