{"data":{"slug":"common-crawl","etype":"dataset","name":"Common Crawl","cat":"Web Corpus","subtitle":"The nonprofit web archive nearly every LLM ate first.","summary":"Common Crawl has archived the open web monthly since 2008 — petabytes of pages that, filtered and deduplicated, form the backbone of most pre-training corpora (GPT-3's largest component; ancestor of C4, RefinedWeb, FineWeb). A tiny nonprofit's crawler quietly became the most economically consequential dataset on Earth — and the center of the consent-and-copyright fight.","attrs":null,"kf":null,"aliases":["CommonCrawl"],"ticker":null,"exchange":null,"hq":null,"founded":null,"website":null,"status":null,"region":null,"asof":"2026-01-15","sources":[{"url":"https://commoncrawl.org/","label":"Common Crawl"}]},"meta":{"found":true,"count":1,"served_from":"site cache (no live upstream calls)","as_of":"2026-10-14T05:45:20+00:00","disclaimer":"Research and information only — not investment advice. Structured records carry basis labels (vendor spec / reported / estimate); market quotes are delayed/indicative.","attribution":"Records compiled from vendor documentation, official filings and reputable public reporting; market quotes via financial data providers. See /data-sources."}}