---
title: "Discussion on \"How to Scrape Hacker News with Go: Front Page, Comments, Algolia Search, and a Rank Tracker (2026)\" | Hashnode"
description: "Discussion on \"How to Scrape Hacker News with Go: Front Page, Comments, Algolia Search, and a Rank Tracker (2026)\". Build a stdlib-only Go Hacker News scraper on the official Firebase API and Algolia search that fetches the front page concurrently, rebuilds comment trees, gets past the 1,000-hit search cap, and logs front-page ranks to JSON Lines."
url: "https://hashnode.com/posts/how-to-scrape-hacker-news-with-go-front-page-comments-algolia-search-and-a-rank-tracker-2026/6ad993db1254e611c94a5dd1"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [How to Scrape Hacker News with Go: Front Page, Comments, Algolia Search, and a Rank Tracker (2026)](https://viktorhendelmann.hashnode.dev/how-to-scrape-hacker-news-with-go-front-page-comments-algolia-search-and-a-rank-tracker-2026?utm_source=hashnode&utm_medium=feed)

Build a stdlib-only Go Hacker News scraper on the official Firebase API and Algolia search that fetches the front page concurrently, rebuilds comment trees, gets past the 1,000-hit search cap, and logs front-page ranks to JSON Lines.

Oct 22 · 19 min read · viktorhendelmann.hashnode.dev  
Read the article

-   [#tutorial](https://hashnode.com/tag/tutorial)
-   [#go](https://hashnode.com/tag/go)
-   [#golang](https://hashnode.com/tag/golang)
-   [#web-scraping](https://hashnode.com/tag/web-scraping)
-   [#hackernews](https://hashnode.com/tag/hackernews)

## Written by

[Viktor](https://hashnode.com/@viktorhendelmann)

@viktorhendelmann

  
[View profile](https://hashnode.com/@viktorhendelmann)

## Responses 1

Markdown supported · ⌘↵ to post

[OnlineProxy](https://hashnode.com/@onlineproxyio)

[Oct 23](https://hashnode.com/posts/how-to-scrape-hacker-news-with-go-front-page-comments-algolia-search-and-a-rank-tracker-2026/6ad993db1254e611c94a5dd1/comment/6adb7ea1c5fd6a2811bc3417)

the getitem return order is worth fixing, like bind the error first, then return, since evaluation order there isn’t guaranteed. the descendants math only works if one item in each of those threads is both deleted and dead i think. dropbox fits the rule, but i’d want a few more clean threads before calling it settled

the epic vs apple drop is your strongest hint of a penalty or moderation action, tho still a guess. also 586 requests against a crawl-delay of 30 is hard to square so think about how often you actually poll before widening the snapshot
