Home › AI Models › Llama 3.1 405B

Llama 3.1 405B

AI Model Language Prior generation

The first open-weights model at closed-frontier scale.

You're looking at an AI model record: who built it, what it can do, and the hardware and data behind it.

Llama 3.1's 405B dense model (July 2024) was trained on 15.6 trillion tokens across ~16,000 H100s (~3.8×10²⁵ FLOP, disclosed) and matched GPT-4-class results — the moment open weights reached the frontier's neighborhood. Its disclosed training details made it the reference public datapoint for frontier training economics.

Specifications & key facts

Each figure is labeled by basis: vendor spec, vendor claim, reported, estimate, or official. "Not publicly disclosed" means exactly that — we never invent values.

Overview

Training compute ~3.8×10²⁵ FLOP (from disclosed GPU-hours)Published paper

Training hardware ~16,000 × H100 (two 24k clusters built)Published paper

How this connects to the rest of the stack

This record's links at a glance — click a group to jump to it.

Developed by

Trained on

NVIDIA H100

Basis for

NVIDIA Nemotron

Succeeded by

Llama 4

Successor to

Llama 2

Competes with

GPT-4o

Open vs Closed Weights

Sources

Llama 3.1 paper ↗

Record reviewed as of 2026-01-15 — see Methodology.