Infron Open-Sources a Two-Node NVIDIA DGX Spark GB10 Benchmark Suite

GitHub: https://github.com/InfronAI/NVIDIA-DGX-Spark-GB10-TestingSuits

Hardware specifications are necessary inputs to an infrastructure decision. They are not predictions of the experience an application will deliver.

A peak arithmetic rating describes a device under defined numeric-format and sparsity conventions. It does not tell us how quickly a serving engine processes a prompt, how two nodes coordinate, how scheduling behaves under concurrent traffic or how long a user waits before meaningful content appears.

That gap is why we built and open-sourced the NVIDIA DGX Spark GB10 Testing Suite.

Measure the system, not one number

The study follows a layered path:

hardware → operators and communication → inference engine → user-visible service

The measured system uses two NVIDIA DGX Spark / GB10 nodes, each with 128 GB of LPDDR5x unified memory, with tensor parallelism across both nodes. The public model identifier is qwen/qwen3.8-flash-next-abliterated; the pinned weights are dealignai/Qwen3.8-Flash-Next-ABLITERATED-NVFP4 at revision be794b990578ef3031eccf9f28e675a289a09ee9. The serving environment uses SGLang 0.5.21, modelopt_fp4, TP=2 and MTP, with a 262,144-token total context ceiling. NVFP4 W4A4 is applied to the routed experts; not every model module is four-bit.

The answer requires separate measurement boundaries.

Hardware tests measure arithmetic, memory access and sustained behavior. Communication tests measure two-node collectives and overlap. Native-engine probes isolate prefill, decode and fixed-length generation. Public-serving tests measure complete streamed requests, concurrency, arrivals, latency, throughput and failures through the actual service path.

We do not pool those layers into one headline score. Each supports a different conclusion.

Capacity and waiting belong on the same chart

For the matched public short-prompt workload, client concurrency increased from 1 to 8 while aggregate output throughput rose from 45.1 to 158.9 output tokens per second.

That is a 3.52× increase in aggregate output throughput. It is also incomplete without the latency curve.

Across the same operating points, visible-content TTFT p95—measured from sending a request to receiving the first nonempty visible content fragment—moved from 1.65 to 2.39 seconds; reasoning-only deltas are excluded. Median visible-content TTFT remained between 1.01 and 1.17 seconds, while the tail moved more visibly at concurrency eight.

Operators purchase capacity, but users experience waiting. A useful deployment decision therefore needs at least both dimensions:

• aggregate work completed over time;

• the distribution of user-visible response latency.

The right operating point depends on the workload and service objective. It is not automatically the point with the highest aggregate throughput.

Long output is a capacity result, not a quality claim

The native fixed-output experiment completed one 260,096-token output from a 1,024-token input, within the configured 262,144-token total context ceiling.

That result demonstrates a capacity and scheduling path under the recorded configuration. It does not establish answer quality, useful long-form coherence or a universal public-API output limit. Requested budgets, realized lengths and natural completions are kept separate in the report.

The same discipline applies to the prompt bank. The study uses prompts sampled from six public datasets as repeatable serving inputs. It does not score the answers or rank model capability.

Reproducibility means more than publishing a chart

The repository includes protocols, configuration snapshots, workload plans, measurement tables, provenance records and English and Chinese reports in Markdown, HTML and PDF.

The sparkbench CLI supports two distinct forms of reproduction.

First, anyone can validate the repository and recompute published statistics offline from saved evidence. That checks whether the disclosed data produce the reported result.

Second, an operator can configure sparkbench serve to run a new bounded experiment against an OpenAI-compatible endpoint:

sparkbench serve \
  --config configs/serving.local.json \
  --workloads configs/smoke-workloads.jsonl \
  --output outputs/my-serving-run \
  --execute

sparkbench serve \
  --config configs/serving.local.json \
  --workloads configs/smoke-workloads.jsonl \
  --output outputs/my-serving-run \
  --execute

A new run produces new requests, costs and environment-specific evidence. It is not the same as recreating the historical environment, and the documentation states that boundary explicitly.

This is why the project is an open-source benchmark harness or testing suite, not an open benchmark API. There is no hosted benchmarking endpoint or public leaderboard behind the claim.

Economics are scenarios, not promises

The report includes ownership and payback scenarios for a US$10,000 two-node acquisition.

Under the report’s modeled assumptions—US$10,000 hardware cost, concurrency-eight short-prompt aggregate throughput, 75% productive utilization and estimated electricity—the self-use scenario yields a 5.48-year payback and −8.7% five-year ROI. A separate sales scenario assumes 75% sold utilization and pricing at twice the blended API reference; it yields a modeled 2.65-year payback and +88.5% five-year ROI.

These are scenario outputs under stated assumptions. They exclude other operating costs and should not be read as universal returns. Their value is that the assumptions are visible and adjustable.

What the project is, and what it is not

The NVIDIA DGX Spark GB10 Testing Suite is a custom engineering study. It is not an MLPerf-certified submission, a model-quality ranking or a matched H200 performance comparison.

It is designed to let engineers ask:

• What exactly was measured?

• Where did the timing boundary begin and end?

• Which requests completed, failed or were rejected?

• Can the published statistics be recomputed?

• Can the harness be pointed at another endpoint or configuration?

• Which conclusions are measurements, and which are economic or architectural assumptions?

GitHub: https://github.com/InfronAI/NVIDIA-DGX-Spark-GB10-TestingSuits

Q & A

What did Infron test on the two-node NVIDIA DGX Spark GB10 system?

The study covers two linked layers: native SGLang and hardware tests, plus controlled public-service tests through Infron, Cloudflare and a registered-payload forwarding proxy. The 45.1-to-158.9 output-token/s and visible-content TTFT results come from the public short-prompt windows; the 260,096-output-token result comes from the native SGLang path. These results describe this documented stack and workload, not every GB10 deployment.

Why can’t peak hardware specifications predict user latency?

Peak specifications describe component-level ceilings. A real request also passes through the model runtime, inter-node communication, scheduling, queueing and streaming path. Those layers affect when a user sees the first response content, so capacity and waiting need to be measured together.

How should aggregate throughput and first-content p95 be read together?

From concurrency 1 to 8, aggregate output throughput increased from 45.1 to 158.9 output tokens per second. Over the same public short-prompt operating points, visible-content TTFT p95—the 95th-percentile time from sending a request to receiving the first nonempty visible content fragment—increased from 1.65 to 2.39 seconds; reasoning-only deltas are excluded. More aggregate capacity did not keep tail waiting time flat.

Does the 3.52× result mean each request became 3.52× faster?

No. The figure describes aggregate output throughput for the tested short-prompt workload as concurrency increased from 1 to 8. It is not a single-request speedup and should not be interpreted without the accompanying first-content latency result.

What does the 260,096-token run demonstrate?

One native fixed-output run completed 260,096 tokens within the configured 262,144-token total context ceiling. That demonstrates a capacity and scheduling path under the recorded configuration; it does not establish answer quality, a universal public-API output limit or equivalent behavior across other workloads.

Is this an open benchmark API or an open-source testing suite?

It is an open-source benchmark harness and testing suite, not a hosted benchmarking endpoint or public leaderboard. The repository includes protocols, configuration snapshots, workload plans, measurement tables, provenance records and bilingual reports, and the harness can run new bounded experiments against an OpenAI-compatible endpoint.