NVIDIA Dynamo
Slide decks
Interactive HTML presentations. Use โ โ to navigate and F for full screen inside a deck.
Introduction
-
Introduction to NVIDIA DynamoWhat Dynamo is, who runs it, the open-source community, and a tour of every component with published results.intro.html
-
Deploy and tune DynamoHands-on setup from a laptop to Kubernetes, tested recipes, the tuning levers that matter, and agent skills that optimize for you.setup.html
Use cases
-
RL rolloutWhere Dynamo fits in RL post-training today, what customers are doing with it, and the roadmap across three pillars.rl.html
-
AgentsHow Dynamo serves agent programs: session IDs, tracing and simulation, program-level scheduling with ThunderAgent, and programmatic KV caching.agent.html
-
Omni: multimodal, speech and diffusion servingVision-language, speech and diffusion models on one OpenAI-compatible serving stack, with live demos.omni.html
Components
-
ModelExpress + SnapshotFaster model loading and worker startup with ModelExpress and snapshots.mx-snapshot.html
-
KVCR (KV Cache Runner)Distributed KV cache management: the Dynamo router plans where KV goes, and KVCR moves it between workers and keeps it through failures.kvcr.html
-
AISimSimulating the whole inference stack (traffic, scheduler, KV cache, router, planner, GPU timing) on a laptop, without GPUs.aisim.html
-
SidecarTake Dynamo out of the engine process: a CPU-only sidecar plugs a stock vLLM, SGLang or TRT-LLM server into Dynamo.sidecar.html
-
RouterHow the Dynamo router picks a worker, survives overload and restarts, decides what gets admitted, and how to plug in your own policy โ plus where it's going.router.html
-
Global RouterThe existing router run one level up, where each pool acts as a single super-worker.global-router.html
-
Fault ToleranceKeep serving when something breaks.ft.html
-
Planner + GrovePlanner decides how many engines to run; Grove decides what one engine instance is and where it lands.planner-grove.html
-
AIPerfBenchmark any inference endpoint with production-shaped traffic: architecture, load control, SLA search, goodput, outputs, and who uses it.aiperf.html
-
AITuneOne API that finds the fastest validated backend for every module of a PyTorch model: architecture, strategies, JIT vs AOT and performance.aitune.html
-
FlexTensorRun models bigger than your GPU: weights stream from host RAM just in time, when it's free, how it's built, and results on video and LLM workloads.flextensor.html