Discussion on "Scrapy - your neighborhood spider!"

Krish · 2021-03-11T06:18:32.923Z

Spider? You are kidding :) Well, spiders are all over the internet. It crawls websites and retrieves information readily available on the wild web. From the Wikipedia: A Web crawler, sometimes called a spider or spiderbot and often shortened to cra...

Interesting read Krish. I will try this one out to see what I come up with. Thank you for explaining what you are actually doing in code. I am wondering if this is also one part of lighthouse searches. Quite trippy!

What do you mean by lighthouse searches?

Very interesting read. I'm curious to give "scrapy" a try. Your article prompted me with an idea. If I could automate the python script to see where the search result of my personal website ranks every week, it could help me with the SEO for my website if need be.

Great thought! So, to know your seo rankings I'm sure you'd be looking it up on some website. Isn't it? You could probably scrape those websites to retrieve the information you are looking for.

Krishanu : Yes, you're right. It usually is based on the Google search result. So, I guess I can scrape the result from Google. I will give this a try. Of course, there may be OOB tools available for this. Google Analytics being the simplest one of them. But, I'm curious to try this as a use case. Thanks for sharing your knowledge.

Discussion

Scrapy - your neighborhood spider!

Responses(2)

Recent in Forum

Search Hashnode

Scrapy - your neighborhood spider!

Responses(2)

Recent in Forum