web botting with python requestsView other answers to this thread
3.4K+ developers have started their personal blogs on Hashnode in the last one month.
Write in Markdown · Publish articles on custom domain · Gain readership on day zero · Automatic GitHub backup and more
That'd be a starting point. Request one page from a server. If that works, take the response text and turn it into Python objects. Look up the data you want inside. Then repeat for multiple pages.
Eventually you may want to make it faster by doing multiple requests at the same time. You can use multiprocessing.Pool. But get it working first, being optimizing.