DGX Spark Qwen 3.8 Benchmark: 23.1% Faster With AEON DFlash2
Qwen 3.8 27B AEON Ultimate with DFlash2 hit 43.7 tok/s median in both repeats of the exact harness I used for my July Qwen 3.6 tests. That is 23.1% above the old two-run mean, with 30.8% higher end-to-end throughput across the sequential 11-prompt suite. It is now the only model deployed on my DGX Spark and serves as Hermes Agent's secondary local provider.
This is a follow-up to my June three-way Qwen, Gemma and DiffusionGemma benchmark. The short version is that Qwen 3.8 plus AEON's custom DFlash2 drafter is a solid generation-over-generation improvement. It is not a new overall speed record across every model I have tested, and I am not going to pretend that mismatched harnesses make it one.
The headline result
| Metric | July Qwen 3.6 two-run mean |
Qwen 3.8 run 1 |
Qwen 3.8 run 2 |
Change vs old mean |
|---|---|---|---|---|
| Median decode | 35.5 tok/s | 43.7 tok/s | 43.7 tok/s | +23.1% |
| Min / max decode | 15.2 / 74.5 tok/s across runs |
23.7 / 79.4 tok/s | 23.7 / 79.0 tok/s | — |
| Median TTFT | 278.5 ms | 272 ms | 273 ms | 2.2% lower |
| Sequential suite throughput | 25.0 tok/s | 32.7 tok/s | 32.7 tok/s | +30.8% |
Here, sequential suite throughput means total output tokens across the 11 sequentially measured prompts divided by their summed end-to-end request times, including TTFT. It is not multi-user aggregate throughput; the separate concurrency results appear below.
The new median is also 4.8% above the best July run, which reached 41.7 tok/s, and 49.1% above the slower July repeat at 29.3 tok/s. Using the old two-run mean as the primary baseline keeps one unusually fast or slow run from driving the story.
Both Qwen 3.8 repeats were back-to-back on an otherwise idle server. The identical 43.7 tok/s medians demonstrate repeatability under those conditions, not day-to-day variance.
What made this a like-for-like comparison
I recovered the same bench_aeon.py harness used for the July 3 Qwen 3.6 runs and changed only its served-model alias from aeon-ultimate to aeon. The model and serving stack are the deployment changes being measured; the benchmark script itself stayed fixed.
- The same 11 measured prompts, excluding warmup
- Greedy sampling at
temperature=0.0 - Thinking disabled
- The same token caps and streaming TTFT method
- The same localhost endpoint
This is not AEON's full 537-prompt served benchmark, and it did not include an accuracy or quality evaluation. It answers a narrower question: how much faster is the new deployment than my July Qwen 3.6 deployment on the older workload?
Every category mean improved
| Category | July Qwen 3.6 mean | Qwen 3.8 mean | Change |
|---|---|---|---|
| Code | 62.9 tok/s | 70.1 tok/s | +11.4% |
| Decode | 18.65 tok/s | 24.9 tok/s | +33.5% |
| Long-form | 22.65 tok/s | 31.3 tok/s | +38.2% |
| Math | 45.7 tok/s | 60.9 tok/s | +33.3% |
| Reasoning | 43.9 tok/s | 51.55 tok/s | +17.4% |
| Security | 22.15 tok/s | 30.6 tok/s | +38.1% |
Long-form and security saw the largest gains against the old means; code improved by a more modest 11.4%. Compared only with the single best July run, reasoning is the one category slightly below its old best, down 3.3%. The others remain ahead.
The exact deployment
| Component | Tested configuration |
|---|---|
| Hardware | One NVIDIA DGX Spark, Blackwell GB10, aarch64, 121 GiB usable unified memory |
| Software | DGX OS 7.6.0, NVIDIA driver 580.178.04 |
| Engine | vLLM 0.29.0+aeon.sm121a.dspark |
| Image | ghcr.io/aeon-7/aeon-vllm-ultimate:2026-09-18-v0.29.0-omni |
| Target | Qwen3.8-27B AEON Ultimate NVFP4-MIXED |
| Drafter | AEON DFlash2 Qwen3.8-27B, NVFP4 W4A16 with Marlin kernel |
| Context and cache | 262,144 tokens configured (not full-context tested), FP8 KV cache, TRITON attention |
| Batching | 16 sequences, 16,384 batched tokens, chunked prefill and prefix caching enabled |
| DFlash2 | Up to 9 speculative tokens, with a batch-size-dependent lattice and probabilistic drafting |
| GPU memory target | gpu_memory_utilization=0.80 |
I pinned and verified the target revision 02af3dcf… and drafter revision 964d336d…, including the target shards and drafter weights. The live model cards and DFlash2 runbook still matched the reviewed recipe when I audited the deployment.
The recipe matters. This uses the AEON-trained DFlash2 drafter runbook, not the generic stock z-lab drafter configuration. It does not add a separate --quantization flag or enable MTP alongside DFlash2.
Thinking-on speed and concurrency
The 43.7 tok/s headline comes from the greedy, thinking-off, same-harness comparison. I also ran AEON's shorter thinking-on smoke and concurrency scripts. These are different workloads, so the numbers should not be averaged together.
A single streamed request asking why the sky is blue produced 115 output tokens with:
- 32.3 tok/s decode and 30.5 tok/s end-to-end
- 0.24-second TTFT
- Acceptance length τ = 3.83 across 30 verification steps
The quick concurrency benchmark used thinking on, server-default sampling, 384-token output caps, two requests per user and a cycle of 15 generic prompts.
| Concurrent users | Requests | Output tokens | Total tok/s | Per-user tok/s | Acceptance τ |
|---|---|---|---|---|---|
| 1 | 2 | 683 | 34.5 | 34.5 | 4.17 |
| 4 | 8 | 1,998 | 74.9 | 18.7 | 3.89 |
| 16 | 32 | 8,600 | 163.5 | 10.2 | 3.28 |
After that run, the Spark still had 19 GiB of unified memory available and was not using swap. Cold start to a healthy endpoint took 561 seconds, or 9 minutes 21 seconds.
How it compares with AEON's reference
| Users | My total tok/s | AEON reference | Throughput difference | My τ / reference τ |
|---|---|---|---|---|
| 1 | 34.5 | 32.7 | +5.5% | 4.17 / 4.12 |
| 4 | 74.9 | 98.4 | −23.9% | 3.89 / 3.95 |
| 16 | 163.5 | 217.1 | −24.7% | 3.28 / 3.38 |
Single-user throughput is on the published reference level. The quick run's aggregate results are lower at 4 and 16 users, while acceptance length remains close. That is consistent with a working drafter, but it does not isolate the cause of the throughput difference.
This is not enough evidence to claim a concurrency regression. AEON's reference uses larger controlled prompt sets, 512-token caps and a different target revision. Its published served measurements also describe the previous lk3 drafter stage, while the deployed release identifies itself as lk4. A controlled comparison needs the full prompt set, cache-salting protocol, matching output caps and repeated runs.
What about Gemma and DiffusionGemma?
The June article's historical numbers still stand as records of those deployments: Gemma-4 reached 81 tok/s without speculation in that test, and DiffusionGemma reached 210 tok/s single-stream in its stock configuration. Qwen 3.8's 43.7 tok/s same-harness median does not displace those raw-speed results.
But those numbers are not a fresh three-way shootout. The models, harnesses, sampling settings, prompt mixes, context settings and output caps differ. I did not rerun Gemma or DiffusionGemma against Qwen 3.8 for this update, and their model directories were removed during the November 13 cleanup.
The current production choice is therefore about more than a leaderboard. Qwen 3.8 gives me the current Qwen model, a configured 262K context window, the multimodal AEON target, validated endpoint tool calling and basic Hermes routing/chat. I did not test a full 262K prompt or an image/multimodal request. The DFlash2 drafter uses text-only draft inputs in this deployment, so this run does not validate the target's multimodal capability.
Gemma, Ornith and DiffusionGemma retain faster historical results in my notes; none of them is deployed now.
Endpoint and Hermes validation
Throughput is only useful if the endpoint works as an agent backend. This deployment passed all of the checks that mattered for my setup:
- The LAN
/v1/modelsendpoint returnedaeon. - A non-thinking LAN chat request returned the expected sentinel exactly.
- A structured tool call returned
get_current_weatherwith the requested Sydney location. - An explicit Hermes request through provider
sparkand modelaeoncompleted successfully. - DFlash2, the Marlin NVFP4 kernel and growing speculative-acceptance counters were all verified in logs and metrics.
The structured tool-call check was a direct LAN API test. The separate Hermes check verified secondary-provider routing and a basic chat response, not an end-to-end Hermes tool-call loop.
Qwen 3.8 is configured as Hermes' secondary local provider. It did not replace my primary provider.
The bottom line
The defensible headline is 23.1% higher median decode throughput than my July Qwen 3.6 two-run mean on the same harness, with a 30.8% gain in the sequential whole-suite end-to-end measure and improvements across all six category means. The fact that both new repeats landed at 43.7 tok/s is encouraging.
The equally important caveat is that this is not proof of better model quality, a controlled reproduction of AEON's concurrency reference, or a new cross-model speed record. Those claims need different tests.
For my DGX Spark, Qwen 3.8 plus DFlash2 is a healthy, verified deployment with the context length and Hermes integration I wanted. It earns its place as the current local model without needing an exaggerated benchmark claim.
Sources: AEON Qwen 3.8 NVFP4-MIXED target and AEON DFlash2 drafter and benchmark notes on Hugging Face.
Notes
- All new benchmarks ran on one DGX Spark on November 13, 2026. Results may differ on other hardware or serving configurations.
- The two same-harness Qwen 3.8 runs were deterministic and back-to-back on an idle server. They establish repeatability under those conditions, not longer-term stability.
- No accuracy or quality benchmark was performed. These results measure serving health, latency, throughput and speculative acceptance.
- Historical Gemma, Ornith and DiffusionGemma figures use different harnesses and configurations. They are planning context, not like-for-like performance or quality proof.
- The target and drafter may require access approval on Hugging Face. Follow the current DFlash2 model-card runbook rather than mixing settings from the generic stock-drafter recipe.
Connect with me on LinkedIn.