DGX Spark Qwen 3.8 Benchmark: 23.1% Faster With AEON DFlash2

AI · Benchmarks · Hardware
Same-harness DGX Spark benchmark table: Qwen 3.8 with DFlash2 reaches 43.7 tok/s median, up 23.1% over the July Qwen 3.6 two-run mean

Qwen 3.8 27B AEON Ultimate with DFlash2 hit 43.7 tok/s median in both repeats of the exact harness I used for my July Qwen 3.6 tests. That is 23.1% above the old two-run mean, with 30.8% higher end-to-end throughput across the sequential 11-prompt suite. It is now the only model deployed on my DGX Spark and serves as Hermes Agent's secondary local provider.

This is a follow-up to my June three-way Qwen, Gemma and DiffusionGemma benchmark. The short version is that Qwen 3.8 plus AEON's custom DFlash2 drafter is a solid generation-over-generation improvement. It is not a new overall speed record across every model I have tested, and I am not going to pretend that mismatched harnesses make it one.

The headline result

Metric July Qwen 3.6
two-run mean
Qwen 3.8
run 1
Qwen 3.8
run 2
Change vs
old mean
Median decode 35.5 tok/s 43.7 tok/s 43.7 tok/s +23.1%
Min / max decode 15.2 / 74.5 tok/s
across runs
23.7 / 79.4 tok/s 23.7 / 79.0 tok/s —
Median TTFT 278.5 ms 272 ms 273 ms 2.2% lower
Sequential suite throughput 25.0 tok/s 32.7 tok/s 32.7 tok/s +30.8%

Here, sequential suite throughput means total output tokens across the 11 sequentially measured prompts divided by their summed end-to-end request times, including TTFT. It is not multi-user aggregate throughput; the separate concurrency results appear below.

The new median is also 4.8% above the best July run, which reached 41.7 tok/s, and 49.1% above the slower July repeat at 29.3 tok/s. Using the old two-run mean as the primary baseline keeps one unusually fast or slow run from driving the story.

Both Qwen 3.8 repeats were back-to-back on an otherwise idle server. The identical 43.7 tok/s medians demonstrate repeatability under those conditions, not day-to-day variance.

What made this a like-for-like comparison

I recovered the same bench_aeon.py harness used for the July 3 Qwen 3.6 runs and changed only its served-model alias from aeon-ultimate to aeon. The model and serving stack are the deployment changes being measured; the benchmark script itself stayed fixed.

This is not AEON's full 537-prompt served benchmark, and it did not include an accuracy or quality evaluation. It answers a narrower question: how much faster is the new deployment than my July Qwen 3.6 deployment on the older workload?

Every category mean improved

Category July Qwen 3.6 mean Qwen 3.8 mean Change
Code62.9 tok/s70.1 tok/s+11.4%
Decode18.65 tok/s24.9 tok/s+33.5%
Long-form22.65 tok/s31.3 tok/s+38.2%
Math45.7 tok/s60.9 tok/s+33.3%
Reasoning43.9 tok/s51.55 tok/s+17.4%
Security22.15 tok/s30.6 tok/s+38.1%

Long-form and security saw the largest gains against the old means; code improved by a more modest 11.4%. Compared only with the single best July run, reasoning is the one category slightly below its old best, down 3.3%. The others remain ahead.

The exact deployment

ComponentTested configuration
HardwareOne NVIDIA DGX Spark, Blackwell GB10, aarch64, 121 GiB usable unified memory
SoftwareDGX OS 7.6.0, NVIDIA driver 580.178.04
EnginevLLM 0.29.0+aeon.sm121a.dspark
Imageghcr.io/aeon-7/aeon-vllm-ultimate:2026-09-18-v0.29.0-omni
TargetQwen3.8-27B AEON Ultimate NVFP4-MIXED
DrafterAEON DFlash2 Qwen3.8-27B, NVFP4 W4A16 with Marlin kernel
Context and cache262,144 tokens configured (not full-context tested), FP8 KV cache, TRITON attention
Batching16 sequences, 16,384 batched tokens, chunked prefill and prefix caching enabled
DFlash2Up to 9 speculative tokens, with a batch-size-dependent lattice and probabilistic drafting
GPU memory targetgpu_memory_utilization=0.80

I pinned and verified the target revision 02af3dcf… and drafter revision 964d336d…, including the target shards and drafter weights. The live model cards and DFlash2 runbook still matched the reviewed recipe when I audited the deployment.

The recipe matters. This uses the AEON-trained DFlash2 drafter runbook, not the generic stock z-lab drafter configuration. It does not add a separate --quantization flag or enable MTP alongside DFlash2.

Thinking-on speed and concurrency

The 43.7 tok/s headline comes from the greedy, thinking-off, same-harness comparison. I also ran AEON's shorter thinking-on smoke and concurrency scripts. These are different workloads, so the numbers should not be averaged together.

A single streamed request asking why the sky is blue produced 115 output tokens with:

The quick concurrency benchmark used thinking on, server-default sampling, 384-token output caps, two requests per user and a cycle of 15 generic prompts.

Concurrent users Requests Output tokens Total tok/s Per-user tok/s Acceptance τ
1268334.534.54.17
481,99874.918.73.89
16328,600163.510.23.28

After that run, the Spark still had 19 GiB of unified memory available and was not using swap. Cold start to a healthy endpoint took 561 seconds, or 9 minutes 21 seconds.

How it compares with AEON's reference

Users My total tok/s AEON reference Throughput difference My τ / reference τ
134.532.7+5.5%4.17 / 4.12
474.998.4−23.9%3.89 / 3.95
16163.5217.1−24.7%3.28 / 3.38

Single-user throughput is on the published reference level. The quick run's aggregate results are lower at 4 and 16 users, while acceptance length remains close. That is consistent with a working drafter, but it does not isolate the cause of the throughput difference.

This is not enough evidence to claim a concurrency regression. AEON's reference uses larger controlled prompt sets, 512-token caps and a different target revision. Its published served measurements also describe the previous lk3 drafter stage, while the deployed release identifies itself as lk4. A controlled comparison needs the full prompt set, cache-salting protocol, matching output caps and repeated runs.

What about Gemma and DiffusionGemma?

The June article's historical numbers still stand as records of those deployments: Gemma-4 reached 81 tok/s without speculation in that test, and DiffusionGemma reached 210 tok/s single-stream in its stock configuration. Qwen 3.8's 43.7 tok/s same-harness median does not displace those raw-speed results.

But those numbers are not a fresh three-way shootout. The models, harnesses, sampling settings, prompt mixes, context settings and output caps differ. I did not rerun Gemma or DiffusionGemma against Qwen 3.8 for this update, and their model directories were removed during the November 13 cleanup.

The current production choice is therefore about more than a leaderboard. Qwen 3.8 gives me the current Qwen model, a configured 262K context window, the multimodal AEON target, validated endpoint tool calling and basic Hermes routing/chat. I did not test a full 262K prompt or an image/multimodal request. The DFlash2 drafter uses text-only draft inputs in this deployment, so this run does not validate the target's multimodal capability.

Gemma, Ornith and DiffusionGemma retain faster historical results in my notes; none of them is deployed now.

Endpoint and Hermes validation

Throughput is only useful if the endpoint works as an agent backend. This deployment passed all of the checks that mattered for my setup:

The structured tool-call check was a direct LAN API test. The separate Hermes check verified secondary-provider routing and a basic chat response, not an end-to-end Hermes tool-call loop.

Qwen 3.8 is configured as Hermes' secondary local provider. It did not replace my primary provider.

The bottom line

The defensible headline is 23.1% higher median decode throughput than my July Qwen 3.6 two-run mean on the same harness, with a 30.8% gain in the sequential whole-suite end-to-end measure and improvements across all six category means. The fact that both new repeats landed at 43.7 tok/s is encouraging.

The equally important caveat is that this is not proof of better model quality, a controlled reproduction of AEON's concurrency reference, or a new cross-model speed record. Those claims need different tests.

For my DGX Spark, Qwen 3.8 plus DFlash2 is a healthy, verified deployment with the context length and Hermes integration I wanted. It earns its place as the current local model without needing an exaggerated benchmark claim.

Sources: AEON Qwen 3.8 NVFP4-MIXED target and AEON DFlash2 drafter and benchmark notes on Hugging Face.

Notes

Connect with me on LinkedIn.