Software | AI Infrastructure Database

Field notes

Hardware gets the headlines, but software is where the lock-in lives. One platform here is old enough to vote and still decides which chips people buy; the youngest entries are barely two years old and already load-bearing for the agent era.

Start with these three

CU CUDA Two decades of libraries — the deepest moat in technology. VL vLLM The open engine that serves a huge share of the world's tokens. MC Model Context Protocol (MCP) The agent era's universal port, adopted industry-wide within a year.

Compute platforms & kernels

CU CUDA NVIDIA's programming platform — the deepest moat in the industry. Active NC NCCL The collective-communications library every distributed training run calls. Active RO ROCm AMD's open-source answer to CUDA — the gap that narrowed at last. Active T( Triton (OpenAI) Python-level GPU kernels — the community's crowbar under the CUDA moat. Active X/ XLA / OpenXLA The compiler that makes TPUs programmable — now an open project. Active

Training frameworks

PY PyTorch The framework nearly every modern model is written in. Active DE DeepSpeed Microsoft's scaling toolkit — ZeRO sharding made huge models trainable. Active

JAX Composable function transforms — the framework Gemini is built with. Active MC Megatron-Core NVIDIA's reference recipe for trillion-parameter training. Active

TensorFlow The 2015 giant that industrialized deep learning — now in JAX's shadow. Maintained

Inference & serving

VL vLLM The open-source serving engine that made GPU inference efficient. Active OR ONNX Runtime The portable format for shipping models across hardware. Active OL Ollama One command to run an open model locally. Active SG SGLang The high-performance serving runtime born from RadixAttention research. Active TL TensorRT-LLM & Dynamo NVIDIA's tuned inference stack — peak tokens from its own silicon. Active LL llama.cpp The C++ project that put LLMs on laptops — and invented GGUF. Active

Orchestration & cluster management

Kubernetes The cloud's operating system, retrofitted for GPUs. Active RA Ray Distributed Python for the whole ML pipeline — Anyscale's engine. Active SL Slurm The HPC batch scheduler most big training runs still queue through. Active

Libraries, tooling & the agent layer

MC Model Context Protocol (MCP) The open standard connecting AI models to tools and data — USB-C for agents. Active HF Hugging Face Transformers The library that standardized how models are shared and loaded. Active L& LangChain & LangGraph The scaffolding layer of the agent era. Active W& Weights & Biases Experiment tracking for the training runs on this site — now CoreWeave-owned. Active