Aug 4, 2023 · 3 min read · With the development of the Internet, crawler technology plays an important role in data collection and information retrieval. However, in order to protect data security and prevent abuse, many websites have adopted various means to anti-crawl, which...
Join discussion
Aug 4, 2023 · 3 min read · Various anti-crawler technologies appearing on the Internet, especially Cloudflare and CAPTCHA, effectively prevent unauthorized access. However, with the continuous advancement of technology, we ushered in a powerful HTTP API, providing us with new ...
Join discussion
Aug 4, 2023 · 4 min read · With the development and popularization of the Internet, search engines have become one of the main ways for people to obtain information and find resources. Whether it is answering questions in daily life or market research in the business field, th...
Join discussion
Aug 2, 2023 · 4 min read · With the rapid development of e-commerce, the product information on the e-commerce platform is increasingly abundant, which has become an important basis for market competition and consumer decision-making. However, manually collecting large amounts...
Join discussion
Jul 31, 2023 · 4 min read · When collecting network data, Cloudflare's interception often causes problems for crawlers. Cloudflare employs various security measures, including the presence of captchas designed to identify and block bot access. In this article, I will explore tw...
Join discussion
Jul 26, 2023 · 4 min read · In the event that Cloudflare’s firewall rules detect dubious activity emanating from the client or browser utilized for scraping, an error 1020 will be triggered. This error message conveys the fact that the security service has implemented an automa...
Join discussion