This Show HN project delivers a 47-million-item, 11.6GB Hacker News archive as Parquet files, updated every 5 minutes, and hit 148 votes by solving a critical data-access problem. It outperforms #2 in data comprehensiveness, offering 30 times more items than typical API-based archives. At 60% smaller than the average CSV dump, the Parquet format enables analysts to run complex queries directly without API limits, reducing query time by over 90% for historical trend analysis.
Comments on "Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m"
Create a free account or sign in to join the discussion.
Sign in to join the conversation