The Nvidia Tesla V100 is a data center GPU that readers have requested for some time. This article covers the test system, a BIOS fan configuration, the software setup, and inference benchmarks against consumer and professional GPUs. Benchmarks use the llama.cpp backends Vulkan and CUDA.

Background

One reader, James in Oregon, emailed in November of the previous year. He described the V100 as the highest-performing value GPU available. The author states that this claim may still hold almost a year later.

Test System

The tests ran on an HP Z4 G4 workstation from the author's budget build.

BIOS Fan Configuration

The V100 cards run hot and require their blower fans to operate at high speed. The author changed the following settings to control cooling:

1. Enter the BIOS and go to Advanced > Built-in Devices.
2. Unplug the rear fan on the HP Z4 G4. The rear fan is loud at full operation.
3. Connect the top GPU blower fan to the rear fan header.
4. Leave the front fan header empty.
5. Connect the lower GPU blower fan to the header on the left side at the bottom of the motherboard.
6. Set Increased idle fan speed to 40.

According to the author, this configuration keeps the blower fans running effectively and keeps GPU temperatures low. The system displays an error on boot. Pressing Enter bypasses it.

Software Setup and Benchmark Method

The author used llama.cpp with llama-bench for all tests. Early tests ran on a different system, an ASUS X870E Taichi Lite board, where both x16 PCIe slots connected directly to the CPU. The author had previously charted multiple GPUs on Qwen 3 14B and Qwen 3 32B, both dense models, about a year earlier. The author retained those results for comparison.

The author initially tested the V100 with the Vulkan backend, then built a CUDA backend.

Vulkan Results

• Qwen 3 14B ran on a single GPU because it fit in one card's memory.
• Qwen 3 32B split across both V100 cards, using about 14 GB on each.
• The author stated that the 32B result is comparable to RTX 3090 performance.
• The author believed the test used llama.cpp build 012.3 from about a year earlier. The author noted uncertainty about this detail.

CUDA Backend Setup

The llama.cpp build page requires the CUDA Toolkit. The latest version listed was CUDA Toolkit 13.4.2, as of October 2, 2026. The V100 does not support CUDA 13 in the author's testing. The author installed CUDA Toolkit 12.8 instead.

The installation required the following steps:

1. The system runs Ubuntu 26. CUDA Toolkit 12.8 is not available in the default repository, so the author downloaded the installer file manually.
2. The author edited the math_functions.h header file.
3. The author added noexcept(true) to the end of multiple function lines in that file, using the nano editor.
4. The author built llama.cpp with the CUDA backend for the V100 cards.

CUDA Results

Qwen 3 32B

CUDA generated tokens slightly more slowly than Vulkan. Its prompt processing was about five times faster.

Comparison with the V620 Pro

The AMD Radeon Pro V620 Pro scored 18 tokens/s on Qwen 3 32B.

GPT-OSS 20B

The author stated that the GPT-OSS 20B result was weaker than expected. The author compared the result to RTX 5070 Ti-class hardware. The recording names several cards in this comparison, so the exact reference is unclear.

Qwen 3.8 27B Benchmark

The author ran Qwen 3.8 27B without speculative decoding (MTP) or other parameter changes. This matched the method used in earlier comparisons.

Earlier tests of the V620 Pro and MI50 cards on this model produced scores in the 20s tokens/s. The Radeon VII reached 24 tokens/s, but its prompt processing was slow.

The author stated that the V100 results were strong for a card in this price range.

Pricing

• Tesla V100 16 GB: just over $200 per card
• Tesla V100 32 GB: about $700 per card
• RTX 3090 (24 GB): $1,000 or more. The author estimated$1,200 in current pricing, possibly higher. The author had not checked the price recently.

Year-Over-Year Comparison: Qwen 3 32B

The author compared results from a year earlier with new tests.

The author stated that llama.cpp is likely faster than the earlier tool. Backends and drivers have improved since the previous test.

The author also noted a change in methodology. Early benchmarks did not measure prompt processing. The author has since updated llama-bench and now reports prompt processing results.

Conclusions

• The V100 16 GB delivered strong inference performance for its price, according to the author's tests.
• Compared with the V620 Pro, MI50, and Radeon VII, the V100 showed higher performance on the tested models.
• Backend and driver updates have improved inference speed over the past year.

Community Budget AI Build Challenge

The author reported 11 entries in a budget AI build challenge. The entries came from Poland, Spain, France, Germany, Australia, and Croatia. The author stated that the list may be incomplete.