Nemotron 3.5 Lightning: NVIDIA's 30B MoE built for high-volume agent steps

Nemotron 3.5 Lightning is a sparse mixture-of-experts model with about 30 billion parameters in total and roughly 3 billion active per token. Active parameters drive per-token compute, so the model behaves like a mid-sized model for cost and latency while its capability ceiling comes from the full parameter pool. NVIDIA's own framing is that this makes it suitable for high-volume execution rather than for the hardest reasoning steps.

The architecture is hybrid: interleaved Mamba-2 and mixture-of-experts layers with selective attention layers. For a team, the architecture matters mainly for one reason — it constrains which runtimes and serving paths are available. A hybrid Mamba model needs a runtime that supports it, which is a practical filter when you are choosing between self-hosting stacks.

NVIDIA documents up to a 1-million-token context at the model level. Third-party indexes record that providers commonly serve a smaller window — one gateway recorded 262,144 tokens on its standard endpoints and completion limits ranging from roughly 32,768 to 235,929 tokens depending on provider. A context figure without the provider's completion limit is not actionable.

NVIDIA's own local guidance makes the same point in a different form: the default context in its Ollama guidance scales with available VRAM, with lower defaults on smaller cards and larger ones with more memory. Running a model locally does not automatically mean you can use its maximum documented context.

NVIDIA releases Nemotron 3.5 Lightning under OpenMDW-1.1, and states the model is ready for commercial use. NVIDIA publishes five checkpoints: a BF16 reference build, an NVFP4-quantised build tuned for inference, a base BF16 build for further training, and two draft-model variants for speculative decoding. Weights, training data descriptions, and recipes are published alongside them.

OpenMDW-1.1 is permissive but it is not the MIT licence, and the distinction matters if you plan to redistribute a modified model. Read the licence before making a deployment or redistribution decision; a model card's summary of its own licence is not the licence.

NVIDIA states that Lightning reaches up to four times the throughput of similarly sized open models, and third-party analysis recorded performance comparable to a much larger model at roughly a quarter of the parameters. Those are the developers' and analysts' claims, produced on their own configurations. They are useful for deciding whether the model class is worth evaluating; they are not a prediction for your hardware.

There is one claim that is unusually checkable. The model ships with a native multi-token-prediction head providing speculative decoding enabled by default, and an independent engineering write-up reported that a purpose-trained EAGLE-3 draft head could only match the built-in head. If speculative decoding matters to your latency, that is a specific thing to verify on your own serving stack rather than a claim to accept.