9-Year-Old V100 GPUs vs DGX Spark and RTX Pro 6000: Local AI Benchmark Build
Can 9-year-old V100 GPUs beat DGX Spark and RTX Pro 6000 for local AI? We build, benchmark Qwen 3.8 Flash, and compare costs and trade-offs.
This article compares a four-GPU NVIDIA V100 system with two NVIDIA DGX Spark systems for local large language model (LLM) inference. It covers the hardware configuration, benchmark results against other published setups, and the trade-offs between the two platforms.
Hardware Configuration
V100 System
The V100 system uses a carrier board that holds up to four SXM2 V100 GPUs. The board provides direct NVLink connectivity to every GPU. Each GPU has six NVLink 2.0 links, with a total bandwidth of 300 GB/s.
Assembly procedure:
- Align each GPU with its carrier board slot and secure it with screws.
- Apply thermal putty and thermal paste.
- Install the heat sinks and fasten them.
The builder milled down three posts on the heat sinks to fit an A100-style heat sink. The custom heat sinks provide better thermal performance than the stock heat sinks. Each heat sink connects to a pass-through adapter. Slim SAS ports link the adapters to the carrier board, where each GPU runs on a PCIe x8 connection.
DGX Spark Systems
The builder used two DGX Spark units: a Founders Edition and a Gigabyte Atom.
The builder upgraded the NVMe drive in each unit from 1 TB to 2 TB. The new drive uses PCIe 5, replacing PCIe 4.
Upgrade procedure:
- Remove the screws under the rubber feet.
- Carefully pry off the cover.
- Remove the old thermal pads.
- Unscrew and remove the old NVMe drive.
- Install the new NVMe drive and reassemble the unit.
The DGX Spark has no NVLink. It connects through ConnectX-7 networking at 200 Gb/s. The high-speed link allows LLM model shards to span the cluster.
Benchmark Methodology
The benchmark used Qwen 3.8 Flash as the test model. Both the V100 system and the DGX Spark systems ran NVFP1.
Benchmark Results
Single-System Comparison
| System | Prompt Processing (tokens/s) | Token Generation (tokens/s) |
|---|---|---|
| V100 system (4 GPUs) | ~4,000 | 155 |
| DGX Spark | ~2,900 | Less than 62 (V100 is more than 2.5x faster) |
The V100 system leads in both metrics. In token generation, it is more than two and a half times faster than the DGX Spark.
Comparison With Published Results
The builder used published data from Alex because the builder did not own an RTX Pro 6000 or an M5 Ultra. Alex did not publish long-prompt results for his DGX Spark cluster or his M5 Ultra, so the builder estimated those values.
| System | Prompt Processing (tokens/s) | Token Generation (tokens/s) |
|---|---|---|
| V100 system (4 GPUs) | ~4,000 | 155 |
| RTX Pro 6000 | ~12,000 (short context); ~9,000 (long context) | 122 |
| DGX Spark cluster (Alex) | ~2,500 (short context) | Slightly below the builder's results |
| M5 Ultra (llama.cpp) | Below 1,400 | 44 |
| M5 Ultra (MLX) | 3,400 | 125 |
Key Findings
- DGX Spark cluster: Prompt processing results closely match the builder's results. The builder considers this a sign that the two test methods are similar.
- M5 Ultra: The llama.cpp result is below 1,400 tokens/s for prompt processing. The builder attributes this to software that has not yet been optimized. This result used llama.cpp, while the other systems in the comparison did not. The MLX result shows 3,400 tokens/s for prompt processing and 125 tokens/s for token generation.
- RTX Pro 6000: Prompt processing drops from about 12,000 tokens/s to about 9,000 tokens/s at long context. The builder attributes this drop to PCIe bandwidth limits.
- Alex's cluster: The token generation result is slightly lower. The builder suspects that Alex did not enable both ConnectX-7 interfaces. If only one interface ran, the link would operate at 100 Gb/s.
- M5 Ultra token generation: The M5 Ultra reaches 44 tokens/s. The builder attributes this result to its higher memory bandwidth than the DGX Spark.
- V100 compared with RTX Pro 6000: The V100 system generates 155 tokens/s, compared with 122 tokens/s for the RTX Pro 6000. Alex attributes the RTX Pro 6000 result to PCIe 5 bandwidth limits that prevent multi-GPU scaling.
Interconnect Bandwidth
NVLink 2.0 on the V100 provides 300 GB/s per GPU. PCIe 5 provides 128 GB/s. The NVLink bandwidth is more than twice the PCIe 5 bandwidth. The builder concludes that inter-GPU communication is the main bottleneck in multi-GPU setups. The V100 system's NVLink interconnect explains its advantage over the RTX Pro 6000 and the DGX Spark, despite the V100 being nine years old.
Platform Background
DGX Spark Pricing and Scaling
The DGX Spark launched in 2025. Next to a pair of RTX 3090 GPUs, it appeared to be one of the least attractive options for local AI. Prices rose in 2026.
In June, the builder found a Gigabyte Atom at its 2025 retail price. It was the least expensive system with 125 GB of memory and 1 TB of storage. The memory and storage alone cost more than half the price of the Atom. The builder purchased the unit.
The DGX Spark scales well when nodes are added. Community reports cover clusters of two, four, and eight nodes. In July, the builder bought a second unit, the Founders Edition. Prices for the DGX Spark rose after that purchase.
Recently, Nvidia released a DGX Spark with 64 GB of memory at the same price as the original 128 GB model. This change makes the 128 GB model effectively more expensive. Each added DGX Spark node contributes 128 GB of VRAM.
Workloads on the DGX Spark Pair
The two DGX Spark units run larger models, including DeepSeek, Qwen 3.8 Flash, and GLM 5.2 Flash. They also run image models. The units operate 24 hours a day, seven days a week. They are quiet, small, and energy efficient. They are the slowest systems in this comparison, but they can run nearly any model.
V100 Development
The builder first used two V100 GPUs with PCIe-to-SXM2 adapters. The builder later moved to the four-GPU carrier board with an NVLink mesh.
Performance for the V100 has improved every few weeks through updates from One Cat AI, which publishes code in a GitHub repository. The company's account on X, formerly One Cat AI, now operates under the name BCat.
According to the builder, the V100 system performs better than the DGX Spark on Qwen 3.8 Flash. The model is a preview of the upcoming Qwen 4. The V100 system matches or slightly exceeds the M5 Ultra. It also generates tokens faster than the RTX Pro 6000.
The V100 price has not increased.
Risks
The V100 is older hardware. Software support may end. The builder notes that community developers are writing new software for the V100, which reduces this risk.
Power Considerations
The V100 system consumes more power than the DGX Spark units. The builder states that the system suits locations with solar power or low electricity costs. The builder has used solar power for the past five years.
Conclusion
The builder does not recommend one platform over the other. Each platform suits different workloads.
- V100 system: Delivers higher speed. It suits users who are willing to assemble the hardware and accept higher power consumption.
- DGX Spark systems: Operate quietly, consume less power, and suit continuous 24/7 operation.
The builder uses both systems for production work. The builder used local AI for a significant portion of a talk at Cybercon 2026.
Hardware Build
The creator builds a four-V100 NVLink system and upgrades two DGX Spark units, outlining the hardware work for each.
- Four SMX2 V100 GPUs fit on a 4-GPU NVLink carrier board with direct NVLink connectivity to every GPU.
- NVLink 2.0 provides about 300 GB/s bandwidth (six links).
- Heat sinks were custom-fit: three posts were milled down to fit an A100-style heat sink, done with a friend's milling machine.
- Thermal putty and thermal paste were applied before mounting the heat sinks.
- GPUs connect via pass-through adapters and slim SAS ports to the carrier board's PCIe 8x connection.
- A DGX Spark Founders Edition and a Gigabyte Atom were used.
- The NVMe was upgraded from 1 TB to 2 TB and moved from PCIe 4 to PCIe 5.
- The DGX Spark has no NVLink but has ConnectX-7 networking at about 200 GB/s, used to shard LLM models across the cluster.
Benchmark Comparison
Qwen 3.8 Flash (NVFP4) is used to compare the V100 and DGX Spark, and then the results are set against published data from other systems.
- Benchmark model: Qwen 3.8 Flash using NVFP4 on both the V100 and DGX Spark.
- V100 prompt processing: about 4,000 tokens/s; DGX Spark: about 2,900 tokens/s.
- V100 token generation: 155 tokens/s, more than two and a half times the DGX Spark.
- DGX Spark cluster (from Alex's data): about 2,500 tokens/s prompt processing on short context.
- M5 Ultra: under 1,400 tokens/s prompt processing, using llama.cpp rather than MLX; 44 tokens/s generation.
- RTX Pro 6000: about 12,000 tokens/s prompt processing, dropping to about 9,000 at long context; 122 tokens/s generation.
- Alex's cluster may have had only one ConnectX-7 interface enabled (about 100 Gb/s), which would explain slower generation.
- Long-prompt figures for Alex's DGX Spark and M5 Ultra were estimated because they were not published.
- With MLX, the M5 Ultra reaches 3,400 tokens/s prompt processing and 125 tokens/s generation.
DGX Spark Purchase, V100 Software, and Recommendations
The DGX Spark purchases and their 24/7 use are covered, along with price changes and a new 64 GB model, followed by V100 software gains and final recommendations.
- The Gigabyte Atom was bought locally in June at 2025 retail prices; it was the cheapest PC with 125 GB memory and 1 TB storage.
- The second DGX Spark (Founders Edition) was bought in July.
- The DGX Spark pair runs DeepSeek, Qwen 3.8 Flash, and GLM 5.2 Flash, plus image models, as 24/7 compute.
- Nvidia released a 64 GB DGX Spark at the original price, making the 128 GB model effectively more expensive.
- The V100 software improves every few weeks, according to the One Cat AI GitHub repo.
- The V100 is about as fast as or slightly faster than an M5 Ultra on Qwen 3.8 Flash and is faster than the DGX Spark.
- The V100 is described as a risky choice because it is old hardware and software support might not last.
- The creator's solar power covers the last five years, making the V100's higher power draw easier to accept.