How to download forum pages to HTML using Python & requests
This will download page 1000
… page 1
in descending order and save the HTML to 1000.html
up to 1.html
import requests
for i in range(1000, 0, -1):
with open(f"{i}.html", "w") as outfile:
outfile.write(requests.get(f"https://forum.my-domain.com/showthread.php?123456-my-thread/page{i}").text)
With progress bar
import requests
from tqdm import tqdm
for i in tqdm(range(1000, 0, -1)):
with open(f"{i}.html", "w") as outfile:
outfile.write(requests.get(f"https://forum.my-domain.com/showthread.php?123456-my-thread/page{i}").text)