Megatron-Core
Software
Training
Active
NVIDIA's reference recipe for trillion-parameter training.
You're looking at a software record: a tool or platform in the AI stack — who maintains it and what hardware it runs on.
Megatron pioneered tensor and pipeline parallelism for transformers; Megatron-Core packages those techniques as the library under NVIDIA's NeMo stack and many frontier training codebases — the canonical implementation of 3D parallelism on NVIDIA fabric.
How this connects to the rest of the stack
This record's links at a glance — click a group to jump to it.
Runs on
Maintained by
Related
Sources
Record reviewed as of 2026-01-15 — see Methodology.