Common Crawl
数据集
网络语料库
The nonprofit web archive nearly every LLM ate first.
您正在查看一条数据集记录：一个训练数据集合——其中包含什么，以及哪些模型从中学习。
Common Crawl has archived the open web monthly since 2008 — petabytes of pages that, filtered and deduplicated, form the backbone of most pre-training corpora (GPT-3's largest component; ancestor of C4, RefinedWeb, FineWeb). A tiny nonprofit's crawler quietly became the most economically consequential dataset on Earth — and the center of the consent-and-copyright fight.
它如何与整个技术栈相连
本记录的关联链接一览——点击分组可直接跳转。
用于训练
来源
记录审阅截至 2026-01-15 — 查看 方法论.