OpenStreetMap

How to filter ways by tag from .osm.pbf file using Python & Osmium

In our previous post Minimal example how to read .osm.pbf file using Python & osmium we investigated how to read a .osm.pbf file and count all nodes, ways and relations.

Today, we’ll investigate how to filter for specific ways by tag using osmium working on .osm.pbf files. In this example, we’ll filter by highwaytrunk

#!/usr/bin/env python3
import osmium as osm
from collections import namedtuple
import itertools

Way = namedtuple("Way", ["id", "nodes", "tags"])

class FindHighwayTrunks(osm.SimpleHandler):
    """
    Find ways with "highway: trunk" tag
    """

    def __init__(self):
        osm.SimpleHandler.__init__(self)
        self.ways = []

    def way(self, way):
        # If this way is a highway: trunk, ...
        if way.tags.get("highway") == "trunk":
            # ... add it to self.ways
            # NOTE: We may not keep a reference to the way object,
            # so we have to create a new Way object
            nodes = [node.ref for node in way.nodes]
            self.ways.append(Way(way.id, nodes, dict(way.tags)))

# Find ways with the given tag
way_finder = FindHighwayTrunks()
way_finder.apply_file("kenya-latest.osm.pbf")

print(f"Found {len(way_finder.ways)} ways")

This example uses kenya-latest.osm.pbf downloaded from Geofabrik, but you can use any .osm.pbf file. Parsing said file takes about 10 seconds on my Desktop.

Note that we can’t just self.ways.append(way) because way is a temporary/internal Protobuf object and may not be used directly since. Hence, we create our own object that just wraps the extracted contents of way: Its id, its list of nodes (node IDs) and a dictionary containing the tags. Otherwise, you’ll see this error message:

Traceback (most recent call last):
  File "run.py", line 32, in <module>
    trunk_handler.apply_file("kenya-latest.osm.pbf")
RuntimeError: Way callback keeps reference to OSM object. This is not allowed.
Posted by Uli Köhler in Geography, Geoinformatics, OpenStreetMap, Python

How to fix Python ModuleNotFoundError: No module named ‘osmium’

Problem:

When you running your python script, you see an error message like

Traceback (most recent call last):
  File "run.py", line 2, in <module>
    import osmium as osm
ModuleNotFoundError: No module named 'osmium

Solution:

Install pyosmium using

pip install osmium

or

pip3 install osmium

 

 

Posted by Uli Köhler in OpenStreetMap, Python

Minimal example how to read .osm.pbf file using Python & osmium

This minimal example uses the osmium python binding to read an .osm.pbf file and count the number of nodes, ways and relations.

First, we need to download a suitable dataset. For this example, we’ll be using kenya-latest.osm.pbf which you can download from Geofabrik:

wget https://download.geofabrik.de/africa/kenya-latest.osm.pbf

Now we can run the script:

#!/usr/bin/env python3
import osmium as osm

class OSMHandler(osm.SimpleHandler):
    def __init__(self):
        osm.SimpleHandler.__init__(self)
        self.node_count = 0
        self.way_count = 0
        self.relation_count = 0

    def node(self, n):
        self.node_count += 1

    def way(self, w):
        self.way_count += 1

    def relation(self, r):
        self.relation_count += 1

osmhandler = OSMHandler()
osmhandler.apply_file("kenya-latest.osm.pbf")
print(f'Number of nodes: {osmhandler.node_count}')
print(f'Number of way: {osmhandler.way_count}')
print(f'Number of relations: {osmhandler.relation_count}')

Example output:

Number of nodes: 29442019
Number of way: 3188550
Number of relations: 2225

 

Posted by Uli Köhler in OpenStreetMap, Python