How to get all Wordpress posts as JSON using Python & the Wordpress REST API

In our previous post How to get WordPress posts as JSON using Python & the WordPress REST API we showed how to fetch a single page of 10 posts using the Wordpress REST API in Python.

In this post, we’ll use the pagination in order to fetch a list of all the posts.

Firstly, we observe that once we query an invalid page such as ?page=1000000, the returned JSON will be

{'code': 'rest_post_invalid_page_number',
 'message': 'The page number requested is larger than the number of pages available.',
 'data': {'status': 400}}

instead of the JSON array representing the list of posts.

Using this information, we can write a fetcher that fetches pages of 100 posts each until this error message is encountered:

from tqdm import tqdm
import requests

def page_numbers():
    """Infinite generate of page numbers"""
    num = 1
    while True:
        yield num
        num += 1

posts = []
for page in tqdm(page_numbers()):
    # Fetch the next [pagesize=10] posts
    posts_page = requests.get("https://mydomain.com/wp-json/wp/v2/posts", params={"page": page, "per_page": 100}).json()
    # Check for "last page" error code
    if isinstance(posts_page, dict) and posts_page["code"] == "rest_post_invalid_page_number": # Found last page
        break
    # No error code -> add posts
    posts += posts_page