InfrastructureCompute, chips & cloud
NVIDIA open-sources Snapshot to checkpoint and restore GPU inference pods on Kubernetes
NVIDIA released Snapshot as standalone open-source software that checkpoints GPU inference pods on Kubernetes and restores them in seconds. It cut one DeepSeek-V4.1-Flash vLLM replica's startup from about 18.5 minutes to roughly 31 seconds by skipping repeated compilation work.
From the NVIDIA Generative AI article of November 10, 2026 and cited sources: Snapshot is released under Apache 2.0 and adds Kubernetes-native resources. [1] [6]
SOURCES[1] Start and Restore GPU Workloads in Seconds with NVIDIA… developer.nvidia.com[6] ai-dynamo/snapshot (GitHub) github.com
AT Pulse research deskPublished · Updated · 4 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- Original source
- NVIDIA Generative AI article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- November 10, 2026 article published
- Covered
- Nov 11 edition
- Evidence
- 9 sources across 7 sites
- 1Start and Restore GPU Workloads in Seconds with NVIDIA…developer.nvidia.com
- 2CRIUgpu: Transparent Checkpointing of GPU-Accelerated…arxiv.org
- 3NVIDIA Dynamo Snapshot: Fast Startup for Inference…developer.nvidia.com
- 4Parasail: Faster autoscaling for vLLMparasail.io
- 5Cerebrium: Reducing GPU Cold Starts with Memory Snapshotscerebrium.ai
- 6ai-dynamo/snapshot (GitHub)github.com
Reach: not measured yet
Snapshot is released under Apache 2.0 and adds Kubernetes-native resources. PodSnapshot checkpoints a running pod, SnapshotJob checkpoints a pod once it is ready, and a restore-from annotation starts new pods from a saved snapshot. It now supports multi-GPU checkpoints on a single node with vLLM, SGLang and TensorRT LLM. [1] [6]
NVIDIA's own B200 benchmarks used a single storage backend and covered 22 configurations. Restores took 3.6 to 42.7 seconds, with speedups of 8x to 36x. In every case, creating the checkpoint plus one restore took less time than a single cold start. Compilation, autotuning and CUDA graph capture made up most of the large MoE cold starts. For DeepSeek-V4.1-Flash, that phase took 784 of 1,110 seconds. [1]
in every configuration tested, checkpointing plus one restore beat a single cold start. Implication: inference teams could keep fewer warm replicas for autoscaling, but only on matching GPU and driver fleets.
Read the full assessment
Shared-storage throughput would then become the main limit on restore speed.
Executive brief
NVIDIA has open-sourced Snapshot, a Kubernetes checkpoint/restore system for GPU inference pods. Its headline number: a DeepSeek-V4.1-Flash replica that took about 18.5 minutes to cold-start on vLLM was restored in about 31 seconds, a 36x speedup (NVIDIA blog). Across 22 tested configurations, restores took 3.6–42.7 seconds. In every case, creating the checkpoint plus restoring once took less time than one cold start. All of these figures come from NVIDIA or its design partners. No independent reproduction was found in the reviewed sources.
What changed and event timeline
CRIUgpu paper
Researchers with NVIDIA co-authors combined CRIU and cuda-checkpoint to checkpoint whole GPU containers with no overhead while running. Restores took about 38–98 seconds ().
Dynamo Snapshot preview
NVIDIA described experimental Snapshot support for single-GPU vLLM and SGLang. It reported gpt-oss-120b going from about 119 seconds cold to 15 seconds restored, and said multi-GPU support was planned ().
Parasail's own snapshotting
Parasail used cuda-checkpoint with a patched CRIU on H200s and reported vLLM startups 1.15x–4.18x faster ().
Cerebrium memory snapshots
Cerebrium reported up to 88% faster vLLM cold starts and published comparisons against Modal and Baseten ().
NVIDIA Snapshot released as standalone open source
Snapshot is now part of the DSX AI Factory Platform. It adds multi-GPU single-node checkpoints and benchmarks for vLLM, SGLang and TensorRT LLM (;).
Capabilities and access
- License and code: Apache 2.0, in the
ai-dynamo/snapshotrepository (GitHub). - Kubernetes interface:
PodSnapshotcheckpoints a pod that is already running.
Read the full section
- License and code: Apache 2.0, in the
ai-dynamo/snapshotrepository (GitHub). - Kubernetes interface:
PodSnapshotcheckpoints a pod that is already running.SnapshotJoblaunches a pod and checkpoints it once it is ready.- The
nvidia.com/restore-fromannotation restores a new pod from a snapshot (NVIDIA blog). - Requirements: Linux kernel 5.13+, containerd or CRI-O, GPU Operator 26.3+ with MIG disabled, driver 580+, and shared RWX or S3 storage (GitHub).
- Not supported: multi-node workloads and vGPU. A restore needs the same GPU model and driver as the checkpoint.
Technical analysis for researchers and developers
- Mechanism: CRIU captures host process state and cuda-checkpoint captures GPU memory.
- Smaller artifacts: The KV cache is released before each checkpoint.
- Where cold-start time goes: Compilation, autotuning and CUDA graph capture dominate the large mixture-of-experts (MoE) models.
Read the full section
- Mechanism: CRIU captures host process state and cuda-checkpoint captures GPU memory. A per-node privileged agent coordinates the two, and the workload needs a small pause/resume handshake (GitHub).
- Smaller artifacts: The KV cache is released before each checkpoint. The earlier preview did this with CUDA virtual memory management, which cut a Qwen3-0.6B checkpoint from about 190 GiB to about 6 GiB (Dynamo blog).
- Where cold-start time goes: Compilation, autotuning and CUDA graph capture dominate the large mixture-of-experts (MoE) models. For DeepSeek-V4.1-Flash on vLLM, that phase took 784 of 1,110 seconds. Restoring GPU memory took 18–29 seconds of each 8-GPU restore.
- Test setup: NVIDIA B200 GPUs, VAST NFS storage, vLLM 0.31.0, SGLang 0.5.21, TRT-LLM 1.3.0. Image pull was excluded from the timings. The benchmarking tools are in the repository (NVIDIA blog).
Claims and evidence
- 3.6–42.7 second restores, 8x–36x speedups: NVIDIA's own B200 tables, on a single storage backend (NVIDIA blog).
- Crusoe: GLM-5.3 pod readiness went from about 8 minutes to 31 seconds on HGX B300.
- Peer results: The approach itself is backed by other teams. None of them tested NVIDIA Snapshot.
Read the full section
- 3.6–42.7 second restores, 8x–36x speedups: NVIDIA's own B200 tables, on a single storage backend (NVIDIA blog).
- Crusoe: GLM-5.3 pod readiness went from about 8 minutes to 31 seconds on HGX B300. Saturn Cloud: vLLM startup went from 2–8.5 minutes to 6–34 seconds. Both claims appear only in NVIDIA's post; no partner-published data was found.
- Peer results: The approach itself is backed by other teams. CRIUgpu was peer-research work, and Parasail and Cerebrium built their own systems (arXiv; Parasail; Cerebrium). None of them tested NVIDIA Snapshot.
Context and prior work
- Building blocks: NVIDIA's cuda-checkpoint utility exposes lock, checkpoint, restore and unlock actions (GitHub). CRIU documents GPU checkpointing (CRIU wiki).
- Earlier NVIDIA work: Kubernetes checkpointing first appeared in Dynamo under the name ChReK (Dynamo docs).
- What's new: NVIDIA's main additions are Kubernetes-native resources, a standalone release, and multi-GPU support on a single node.
Limitations, safety and contested findings
- Different numbers for the same model: gpt-oss-120b cold-started in about 119 seconds in the May post but 156–269 seconds in November.
- Checkpoint size outlier: gpt-oss-120b on SGLang TP4 produced a 694 GB checkpoint from 65 GB of weights. It also had the weakest speedup, 8.1x.
- Portability: CRIUgpu found that restores require matching GPU topology, and that cuda-checkpoint did not support NCCL in multi-node settings (arXiv).
Read the full section
- Different numbers for the same model: gpt-oss-120b cold-started in about 119 seconds in the May post but 156–269 seconds in November. The setups differ, so speedups depend on the stack (Dynamo blog).
- Checkpoint size outlier: gpt-oss-120b on SGLang TP4 produced a 694 GB checkpoint from 65 GB of weights. It also had the weakest speedup, 8.1x.
- Portability: CRIUgpu found that restores require matching GPU topology, and that cuda-checkpoint did not support NCCL in multi-node settings (arXiv).
- Security: The node agent runs with privileged access (GitHub).
Business and practitioner implications
- Autoscaling: Faster startup lets teams keep fewer warm replicas, provided they run identical GPU and driver fleets.
- Storage and copying GPU memory are the new bottleneck. Checkpoints reach 1 TB per model configuration, so shared-storage throughput directly limits restore time.
- Competition: Inference clouds such as Parasail and Cerebrium already sell snapshot-based cold starts. A free NVIDIA implementation weakens that as a selling point.
Read the full section
- Autoscaling: Faster startup lets teams keep fewer warm replicas, provided they run identical GPU and driver fleets.
- Storage and copying GPU memory are the new bottleneck. Checkpoints reach 1 TB per model configuration, so shared-storage throughput directly limits restore time.
- Competition: Inference clouds such as Parasail and Cerebrium already sell snapshot-based cold starts. A free NVIDIA implementation weakens that as a selling point.
- Before relying on the speedups: Teams should run the repository's benchmarking tools on their own storage and engine versions.