How to fix Pyppeteer page.goto() page not loading completely

If you are using Pyppetteer code such as await page.goto('https://www.techoverflow.net') and the page doesn’t load completely, i.e. it looks like

whereas it should look like this

you can fix this by using the waitUntil option of page.goto().

How to fix

Use

await page.goto('https://www.techoverflow.net', {'waitUntil': 'networkidle2'})

as a staring point. Depending on the programming of the page you are loading, you might need to try different values from the available options.

• load: Waits for the load event to be fired.
• domcontentloaded: Waits for the DOMContentLoaded event to be fired.
• networkidle0: Waits until there are no network connections for at least 500 ms.
• networkidle2: Waits until there are no more than 2 network connections for at least 500 ms.

Note that there are other options for what to wait (e.g. wait for a specific selector to appear) which are beyond the scope of this post, but networkidle2 is a good starting point for most pages.