Home
Feed

Extend

Hashnode Pro
Hashnode for Agents

Resources

The foreword
Hashnode gql skill

Buildful - AI coding agent that delivers frontier-level results at a fraction of the cost Bug0 - The AI-native e2e QA regression testing Hackathons Changelog Brand @hashnode on X Hashnode on LinkedIn Support - hello+support@hashnode.com Code of Conduct Terms Privacy Sitemap

Sign in

Search Hashnode

Search posts, tags, users, and pages

Discussion on "Scrape News website using Scrapy" | Hashnode

Discussion

Soumendra kumar sahoo

Lead Systems Engineer | Open source contributor

Dec 4, 2021

Scrape News website using Scrapy

Abstract To feed data-hungry NLP models of recent times, I have scraped 5,50,000+ news articles from their websites which constitute an average of 50 lakhs sentences and 2 crores plus words of Monolingual Odia corpus. The dataset consists of a header...

soumendrak.hashnode.dev7 min read

#opensource #web-scraping #python #selenium #python-projects

Responses1

Markdown supported · ⌘↵ to post

Biswajeet Biswal

Software Engineer

Thanks for the awesome post bhai. Can you explain this over a video? It will be really useful for newbies like me. Thanks a lot.

Sure, will try to do it, maybe on YouTube.

SK

Soumendra kumar sahoo

Lead Systems Engineer | Open source contributor