---
title: "Discussion on \"Web Scraping with Python 101\" | Hashnode"
description: "Discussion on \"Web Scraping with Python 101\". Learn to build a web scraper with Python using Requests and BeautifulSoup libraries. We will cover, step-by-step, a scraping process on a job board."
url: "https://hashnode.com/posts/web-scraping-with-python-101/61e8043a0204273bdcb112c7"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Web Scraping with Python 101](https://anderrv.hashnode.dev/web-scraping-with-python-101?utm_source=hashnode&utm_medium=feed)

Learn to build a web scraper with Python using Requests and BeautifulSoup libraries. We will cover, step-by-step, a scraping process on a job board.

Jan 19, 2022 · 16 min read · anderrv.hashnode.dev  
Read the article

-   [#python](https://hashnode.com/tag/python)
-   [#beginners](https://hashnode.com/tag/beginners)
-   [#python-beginner](https://hashnode.com/tag/python-beginner)
-   [#tutorial](https://hashnode.com/tag/tutorial)
-   [#web-scraping](https://hashnode.com/tag/web-scraping)

## Written by

[Ander Rodriguez](https://hashnode.com/@AnderRV)

@AnderRV · Bilbao, Spain

  
[View profile](https://hashnode.com/@AnderRV)

Co-Founder @ ZenRows

Web developer who has been working for several startups for more than 10 years, having worked with a wide variety of sectors and technologies. Engineer turned entrepreneur.

## Responses 1

Markdown supported · ⌘↵ to post

[Peter Merritt](https://hashnode.com/@PeterMerritt)

[May 13, 2022](https://hashnode.com/posts/web-scraping-with-python-101/61e8043a0204273bdcb112c7/comment/627e0ee9cc45a3de295c6045)

How do you deal with sites where JS rendering is required? I've been using [https://webscraping.ai](https://webscraping.ai/) but maybe there are better ways?

[Ander Rodriguez](https://hashnode.com/@AnderRV)

[May 13, 2022](https://hashnode.com/posts/web-scraping-with-python-101/61e8043a0204273bdcb112c7/comment/627e0ee9cc45a3de295c6045#reply-627e32f85cbe9ede04a911b4)

Hi, there two main approaches:

1.  try to find the data you want without JS (internal XHR endpoints, metadata, different URLs...). It's not always possible.
    
2.  use headless browsers (Puppeteer, Selenium, Playwright). You can wait for certain data or selector. The downside is that they are much slower and harder to maintain.
