DGX Spark Model Benchmark: Qwen 3.6 AEON vs Gemma-4 vs DiffusionGemma

I ran three different models on the same DGX Spark hardware to find out which one delivers for agent workloads. The winner surprised me.

The hardware

Same machine for all three tests. No variables beyond the model and the serving configuration.

• Machine: NVIDIA DGX Spark (ARM, 128GB unified memory, Blackwell GB10)
• Memory bandwidth: 273 GB/s
• OS: Headless Linux, NVMe SSD
• Serving runtime: vLLM (versions noted per model)
• Quantisation: NVFP4 across all three models

The three setups

Qwen 3.6 27B AEON Ultimate

This is the model I started with. Dense 27B architecture, served via vLLM 0.20.2 with speculative decoding via DFlash. The drafter model adds roughly 3.2GB of VRAM and achieves a 64.8% acceptance rate at 7 speculative tokens. The model itself sits at around 21GB in NVFP4.

I ran this setup for about a month. It was stable. Not fast, but consistent.

• Architecture: Dense 27B
• Drafter: DFlash, 3.2GB, 64.8% acceptance
• Image: ghcr.io/aeon-7/vllm-aeon-ultimate-dflash:qwen36-v4
• Context: 256K max
• KV cache: ~322K tokens

Aeon Gemma-4 26B-A4B

MoE architecture with 4B active parameters per token. I tested this on June 10 using the corrected v2 image. The first image I tried was broken. The latest tag on the unified image dropped DFlash acceptance to around 10% and crashed above 16 concurrent requests. The v2 image is the one that works. At the time of testing, latest was broken and v2 was the stable tag. That may be resolved now.

This model is lighter. 15.3GB on GPU plus 1GB for the DFlash drafter. Context window goes to 262K. The drafter acceptance is lower than Qwen (55-62%) but the MoE architecture more than makes up for it in raw throughput.

• Architecture: MoE 26B, 4B active per token
• Drafter: DFlash, 1GB, 55-62% acceptance
• Image: ghcr.io/aeon-7/aeon-gemma-4-26b-a4b-dflash:v2
• Context: 262K max
• KV cache: ~340K tokens

DiffusionGemma 26B-A4B NVFP4

This is the newest one. Nvidia's block-diffusion architecture, deployed on June 11 from stevibe/diffusiongemma-26b-a4b-nvfp4-dgx-spark. Single docker compose up, one model, no drafter to download or configure. Uses torch.compile with a 31.5-second compile time that gets cached for subsequent starts.

The KV cache allocation is what sets this apart: 78.4 GiB reserved, holding 3.59 million tokens. That means you can run 13.7 concurrent requests at full 262K context.

• Architecture: MoE 26B, 4B active per token, block-diffusion decoder
• Drafter: None
• Image: vllm/vllm-openai:gemma (v0.22.1rc1)
• Context: 262K max
• KV cache: 3.59M tokens
• torch.compile: Yes, 31.5s cold, cached thereafter

Single-stream throughput

This is the benchmark that matters for agent workloads. When Hermes is running a session, it makes one API call at a time. The model responds, the agent processes, then it makes the next call. Single-stream speed determines how fast the agent feels.

DiffusionGemma is 2.6x faster single-stream than the next best option. That is not a small gap. For an agent that makes 30 tool calls in a session, the difference between 81 tok/s and 210 tok/s is the difference between waiting 8 seconds and waiting 20 seconds for each call. It adds up.

The torch.compile pass is doing heavy lifting here. Same MoE premise (26B total, 4B active) as Aeon Gemma-4, same NVFP4 quantisation, same memory bandwidth ceiling of roughly 97 tok/s theoretical. But the compiled graph plus the block-diffusion architecture pushes real-world throughput well past that ceiling.

Aggregate throughput

If you are serving multiple users or running batch inference, single-stream numbers are not the full story.

The stock DiffusionGemma config ships with max_num_seqs=4. That is deliberate. At full 262K context, each request consumes a large chunk of KV cache. With 3.59M tokens available, 13.7 concurrent requests is the ceiling before memory pressure kicks in. Setting max_num_seqs higher on a lower context window would match or beat the Gemma-4 aggregate numbers. But the default is tuned for reliability, not benchmarking.

Gemma-4 wins aggregate throughput in the current config because it was tested with max_num_seqs=64 at 32K context. Different settings, different goal.

KV cache and concurrency headroom

The KV cache difference is architectural, not config. DiffusionGemma reserves 78.4 GiB for KV cache at fp8_e4m3 precision. That gives it headroom for multiple concurrent full-context requests without fragmentation. Aeon Gemma-4 and Qwen both cap out around one or two requests at full context before hitting memory limits.

For agent workloads that keep a session alive for hours with growing context, this headroom is not theoretical. A Hermes session can hit 150K tokens of context midway through a long task. If the cache fragments under concurrent load, the next request queues or fails. DiffusionGemma avoids that entirely at the stock config.

Speculative decoding comparison

Qwen has the best drafter. Higher acceptance, larger model, proven stability at high concurrency. But the drafter costs VRAM and adds deployment complexity. You are downloading and configuring a second model. If the drafter image is wrong (like my first attempt with Aeon Gemma-4), you are chasing a phantom bug for an hour before realising the latest tag is broken.

DiffusionGemma skips speculative decoding entirely and still wins single-stream by 2.6x. That is the more interesting number. torch.compile plus block-diffusion beats drafter-based speedups without the extra moving part.

Deployment complexity

This was not a variable I planned to measure. It turned out to be the most practical difference.

Qwen 3.6: Custom AEON image, DFlash drafter download, model download, env configuration, vLLM flags for speculative decoding. Takes about 15 minutes to set up from scratch if you know the flags. Took me longer the first time.

Aeon Gemma-4: Same AEON ecosystem, different image, different drafter. The image tag matters. At the time of testing, v2 worked and latest did not. That may have changed by the time you read this. Took a wasted hour to learn that lesson.

DiffusionGemma: git clone, set HF_TOKEN in .env, docker compose up -d --build. That is three commands. No drafter, no speculative decoding flags, no image tag roulette. Total cold start including model download: 3.5 minutes. Subsequent starts with cached compile: 2 minutes.

Startup times

VRAM comparison

Aeon Gemma-4 is the lightest, but the gap to DiffusionGemma is only 2 GiB. Qwen is the heaviest by a margin, and the drafter pushes it further.

Which model for which job

The bottom line

I moved off Qwen because the single-stream throughput was too slow for interactive agent use. 35-57 tok/s with speculative decoding is fine for one-shot prompts but frustrating when you are waiting for the agent to decide which tool to call next, then waiting again for the tool output, then waiting again for the next reasoning step.

Aeon Gemma-4 fixed that. 81 tok/s no-spec, 54 tok/s with thinking, and the MoE architecture means the model is reading 4B parameters per token instead of 27B. The memory bandwidth ceiling stops being the bottleneck.

DiffusionGemma takes the same MoE idea and adds torch.compile plus block-diffusion, then strips out the drafter entirely. The result is 210 tok/s single-stream on the same hardware, with 10x the KV cache headroom. That is not an incremental improvement. That is a different tier of performance for the same memory bandwidth constraint.

I am running DiffusionGemma on the DGX Spark now as the primary model for Hermes Agent. The Aeon Gemma-4 v2 image is stopped but not deleted, in case I want to spin it up on a second port for batch workloads. Qwen is archived.

If you are running a DGX Spark or any ARM machine with 128GB unified memory, this setup works. Three commands, one model, no drafter, and 210 tok/s for agent workloads.

Source: stevibe/diffusiongemma-26b-a4b-nvfp4-dgx-spark on GitHub. Model: nvidia/diffusiongemma-26B-A4B-it-NVFP4 on Hugging Face.

Notes

• All benchmarks run on a single DGX Spark with 128GB unified memory. Results may differ on other hardware.
• The stock DiffusionGemma config ships with max_num_seqs=4. Aggregate throughput numbers reflect this default, not the tuned ceiling. Raising max_num_seqs at a lower context window would produce higher batch numbers.
• The Aeon Gemma-4 image tag issue (latest broken, v2 working) was observed on June 10, 2026. The AEON maintainers may have resolved this since. Always check the repo README for the current recommended tag.
• Speculative decoding numbers for Qwen 3.6 and Aeon Gemma-4 are with DFlash. Other drafters would produce different acceptance rates.
• DiffusionGemma's thinking token mode was enabled via chat_template_kwargs: {"enable_thinking": true}. Thinking tokens are included in the tok/s measurements and consume output budget, so set max_tokens higher than you would without thinking.

Connect with me on LinkedIn.