~13m27:09
Applied AI - GenAI, ChatGPT et LLM-powered apps

J’évalue le DGX Spark sur un workflow AI Engineer complet: Mistral 3, Unsloth, vLLM [ASUS Ascent GX]

Jan 7, 2026

Read: ~13m · You save: 14 min

Evaluating the DGX Spark on a Complete AI Engineer Workflow: Mistral 3, Unsloth, vLLM (ASUS Ascent GX10)

Can the DGX Spark really help AI engineers ship reliable systems faster? A full test with Ministral 3, Unsloth GRPO fine-tuning, quantization, and inference on the ASUS

This evaluation tests whether the NVIDIA DGX Spark, in the ASUS Ascent GX10 implementation, allows an AI engineer to deliver reliable AI systems faster. The test covers a full project cycle: fine-tuning, quantization, inference, and integration into a conversational interface. The author received the unit on loan from ASUS and NVIDIA and states that he received no payment for the evaluation.

Context and Positioning of the DGX Spark

The DGX Spark is a compact desktop system built on NVIDIA's Grace Blackwell architecture. At launch, it drew strong interest, but its reception was lukewarm. The author attributes this to evaluations that focused on token speed alone. Compared with consumer RTX GPUs, the DGX Spark offers a weaker price-performance ratio.

The author argues that the DGX Spark was not designed for enthusiasts. It targets builders who:

  • Optimize models
  • Develop agents
  • Work on complete pipelines in environments close to production

Target Profiles

The author identifies three profiles:

  • Lilia, an applied engineer at a large company. She builds LLM applications mainly on cloud services and proprietary models, using RAG, long chains, and business assistants. She is evaluating open-source models and fine-tuning.
  • Théo, a researcher at a small AI lab. He works with open-weight models, has limited compute, and has no time or interest in DevOps. He needs a stable platform with large memory to train, adapt, and evaluate models end to end.
  • Guillaume, an enthusiast who runs models locally on an RTX-equipped PC. He questions why NVIDIA launched the DGX Spark when RTX cards offer better price-performance.

How the Engineering Role Has Changed

Earlier AI engineering centered on three steps: choosing a model, refining the prompt, and adding context through RAG, sometimes with chain-of-thought prompting. Optimization worked at two levels: what the model knows and how it behaves. These methods were sufficient for simple use cases. Multi-step, domain-specific tasks and tool calls exposed their limits.

By 2025, the focus had moved from the model to the system around it. Engineers now design systems that:

  • Plan
  • Reason
  • Self-evaluate
  • Remain controllable

Three years earlier, the author recommended proprietary models such as GPT or Claude, mainly for strategic reasons: fast delivery, lower risk, and mature platforms. Open-weight alternatives such as Mistral and, more recently, DeepSeek now make open source a credible option. Fine-tuning, optimization, and model control have become core skills rather than advanced options.

The author currently uses APIs from OpenAI, Mistral, and hyperscalers. He considers these platforms the best choice for integrating models quickly. Their limitation appears when engineers need deeper control. Within a single lab's API, teams cannot easily combine a reasoning model from one family with a specialized model from another. This also restricts testing of models such as DeepSeek, Gemma, or Qwen, and access to enterprise runtimes such as vLLM or NVIDIA NeMo for fine-grained performance and cost tuning.

The author's stated requirement is reproducible multi-LLM agent systems that behave the same way in development and production.

Performance in Context

The author states that performance depends on the use case, and he lists four dimensions: latency, load, power consumption, and stability. He cites one example: the DGX Spark can train nanochat, a full GPT-2-class model, in 9 days for less than $10.

The DGX Spark includes:

  • A 4-bit floating-point format
  • Optimized processing of sparse matrices
  • 128 GB of unified memory

The author states that these design choices directly affect the generation phase.

The author summarizes an AI engineer's needs as:

  • Dev-to-production reproducibility
  • Freedom to choose models
  • Access to open-source innovation
  • The ability to test before industrialization
  • Understanding of trade-offs

In his view, the DGX Spark is a realistic development environment. It is neither a high-performance desktop nor a pre-production server. It exposes the real constraints of the Grace Blackwell architecture and provides access to NVIDIA's enterprise ecosystem.

The Model: Ministral 3

Mistral released version 3 of its Ministral family. The family is designed for edge deployment, with a focus on performance-to-cost ratio and robust function calling for agentic systems. Key characteristics:

  • Licence: Apache 2.0 (open weight)
  • Sizes: 3 billion, 8 billion, and 14 billion parameters
  • Variants: Three per size, including instruct and reasoning versions, for nine models in total
  • Capabilities: Multimodal and multilingual
  • Availability: Hugging Face

Mistral and Ministral 3 were trained on NVIDIA Hopper GPUs, the previous generation. NVIDIA works with Mistral, Ollama, and llama.cpp so that their software benefits from accelerations already available on the DGX Spark.

Environment Validation

The first step validated the hardware and software environment. NVIDIA's playbooks ran without errors. The author then ran the following:

  1. Mistral Large 2 with Ollama and Open WebUI. The 128 GB of unified memory handled the model without issues.
  2. Benchmarks with llama.cpp on GPT-OSS. The author compared the results with figures published online.
  3. Ministral 3 in vLLM containers available from NGC, NVIDIA's official container catalog. This also ran without issues.

Inference Benchmark Metrics

The benchmarks report two values:

  • TTFT (Time to First Token): The prefill phase. The prompt passes through the transformer layers, and the KV cache is populated. The KV cache stores previously computed values so the model does not recompute them for each generated token.
  • TPOT (Time Per Output Token): The decoding phase. The model generates tokens one at a time, passing through its layers for each token.

Compared with RTX results, the DGX Spark posts a relatively strong TTFT and a weaker TPOT. The author attributes this to NVIDIA's design choice. NVIDIA prioritized loading very large models over generation speed. Memory bandwidth is limited to 273 GB/s, a direct consequence of unified memory shared between CPU and GPU.

The author reports that the DGX Spark delivers four to five times lower performance than the latest-generation RTX GPUs. He describes this as a deliberate trade-off that enables experimentation with very large models that cannot be loaded elsewhere.

The DGX Spark includes a NVIDIA ConnectX network interface. Two Spark units can be linked to load models up to 405 billion parameters. The author plans to test Mistral Large 3, a mixture-of-experts model with 51 billion active parameters, on two units.

The author also reports that Ministral 3 performs considerably below GPT-OSS-20B, which benefits from the 4-bit floating-point format. He concludes that models must be optimized to use hardware acceleration fully.

Fine-Tuning with GRPO and Unsloth

The author's goal is not to beat benchmarks. It is to adapt a model's behavior to a specific case, as on a client project. He states that a recent model can be adapted locally in under one hour under realistic conditions.

Fine-tuning does not mean retraining a model. It constrains behavior using targeted examples to improve robustness, response consistency, or adherence to specific logic.

Method: GRPO

The author uses GRPO (Group Relative Policy Optimization), a reinforcement learning variant popularized by DeepSeek and suited to reasoning and structured-generation tasks. It is lighter than standard RLHF pipelines, which makes it practical on the DGX Spark.

The method works as follows:

  1. The model generates several candidate solutions.
  2. The solutions are compared with each other.
  3. The most effective solutions are reinforced.

The example task trains Ministral 3 to generate Python code that implements an optimized Sudoku-solving strategy. The author chose this task because its reasoning is explicit, its output is measurable, and its strategies can be compared objectively.

Tool: Unsloth

Unsloth is a fine-tuning toolkit built on PyTorch and Transformers. Its main strength is optimized fine-tuning through LoRA (Low-Rank Adaptation).

LoRA does not retrain all model weights. It adds small adaptation matrices to selected key layers. The pretrained weights stay frozen, and only the lightweight adapters are trained. The result is:

  • Lower memory consumption
  • Faster training
  • Fine-tuning on a single GPU

Unsloth builds on PEFT (Parameter-Efficient Fine-Tuning) in the Transformers ecosystem. It automates training and the final merge of adapted weights with the base model.

The author chose Unsloth because it was ready at the right time. Unsloth published quantized versions of Ministral 3 for mainstream inference engines, along with a notebook implementing GRPO on Ministral 3. The author used this notebook as the base for his tests and adapted it for NVIDIA's official images.

Results

The full fine-tuning of Ministral 3 took about 45 minutes on the DGX Spark. The author describes this as a fast, realistic iteration cycle that fits an AI engineer's workflow. He notes that such tests are usually difficult to set up in the cloud, costly to repeat, or hard to stabilize on a custom infrastructure.

Quantization and Inference Engines

Version Constraint

Unsloth's latest release relies on Transformers 5, which was still a release candidate during the test. Running a model with Transformers 5 requires vLLM 0.12. NVIDIA's latest official vLLM container provides version 0.11.

The author did not want to compile vLLM from source. He chose llama.cpp, which he had already compiled and validated on the DGX Spark. Using llama.cpp required converting the model to GGUF format, which also allowed a quantization step.

Quantization

Quantization reduces the numerical precision of model weights, for example from FP16 to 8-bit or 4-bit formats. The goal is lower memory consumption, faster inference, and better energy efficiency.

The author used llama.cpp for this step because it is one of the most robust tools for producing quantized formats such as GGUF. It also makes it easy to test trade-offs between performance and quality.

Inference Engines

An inference engine is the software layer that controls:

  • How weights are loaded
  • How the KV cache is managed
  • How requests are batched
  • How hardware acceleration is used

The author notes that inference engines differ from local tools such as Ollama or LM Studio in their operational demands.

On the DGX Spark, the options include:

  • vLLM
  • TensorRT-LLM
  • NVIDIA NIM

The author would have preferred vLLM, which he describes as the reference for enterprise environments. His fine-tuned Ministral 3 required the newer version, which was not available in NVIDIA's catalog at the time of testing. He therefore used llama.cpp. He notes that the community maintaining llama.cpp is actively optimizing it for the DGX Spark.

vLLM and llama.cpp Compared

  • vLLM serves LLMs under high concurrency. GPUs and memory are shared resources managed dynamically. It includes advanced KV-cache management and integrates with orchestrators such as Kubernetes. It is designed to serve dozens or hundreds of simultaneous requests.
  • llama.cpp targets single-user local inference. It is optimized for one user on one or more fast GPUs. It does not need orchestration or concurrent memory management, so it is less suited to shared, scalable deployments.

The author argues that the DGX Spark, with its unified memory and large RAM capacity, suits engines such as vLLM that optimize concurrent memory access and GPU sharing.

Interface Validation

The author connected the fine-tuned Ministral 3 3B Instruct model to Open WebUI, an open-source conversational interface. Open WebUI offers many ChatGPT-like features and connects to inference engines such as Ollama, llama.cpp, and vLLM.

The validation was functional, not a full evaluation. The author compared the fine-tuned model with official Mistral models of the same size in instruct and reasoning versions. The test task matched the training task: generating a Sudoku-solving strategy in Python. The author then compared the generated algorithm with one produced by ChatGPT.

The results:

  • The fine-tuned model produced an algorithm quickly. The algorithm was reasonable but contained a few bugs.
  • The standard models did not produce a comparable result.

The author concludes that the cycle from fine-tuning to conversational use is complete.

Workflow Summary

The author reports that the workflow encountered several issues. He attributes them to the immaturity of some of the software solutions he deliberately used. He considers this an expected outcome, since the test aimed to verify the ability to run a full project lifecycle on an industrial-grade stack while retaining freedom to test open-source optimizations.

Evaluation Against the Test Objectives

The author evaluates three questions.

1. Can the full project cycle be completed without shortcuts? Yes. The author completed a full loop: model adaptation, quantization, inference, and functional testing in a conversational interface. He was able to experiment, break components, fix them, and repeat the process.

2. Does the DGX Spark increase engineering freedom? Yes. The author states that he is not restricted to one model provider, one API, or one imposed abstraction. On the DGX Spark, he can use, compare, and optimize open-weight models in one place, choose inference engines, and work at a low level when needed. He also has access to NVIDIA's enterprise stack. He states that he could not achieve this combination in the cloud, or could do so only with significant effort.

3. Does the DGX Spark speed up delivery? Yes, according to the author. The reason is not raw speed. The author says he is not blocked, because he understands the workflow and can change tools when needed.

The author concludes that the DGX Spark does not replace the cloud but complements it. It allows engineers to prepare solutions locally and validate them before scaling them.

Assessment

The author argues that the DGX Spark provides the following:

  • Access to architecture designed for future data centers, available to individual engineers at the same time as it reaches major AI labs. The author notes that Ministral 3 was trained on Hopper, while he works on Blackwell.
  • Hardware and software value at a price of about €3,000. The ASUS Ascent GX10 costs about €1,000 less than the DGX Spark. The author states that it is difficult to offer more for the price, given the hardware and software stack.

Recommendations

  • For Lilia, an applied engineer dependent on cloud services, the DGX Spark is presented as an entry point to a more advanced engineering practice.
  • For Théo, a researcher working on open-weight models without infrastructure teams or data centers, the author considers the DGX Spark a strong option.
  • For Guillaume, the author's evaluation implies that RTX GPUs remain the better choice when raw inference speed and price-performance are the priority.

The author states that he had not included the DGX Spark in his budget but expects to keep it for further testing, unless ASUS agrees to extend the loan.

Introduction and Context

The DGX Spark is presented as NVIDIA's compact desktop supercomputer built on Grace Blackwell. The question is whether it truly lets AI engineers deliver reliable systems faster. The creator says the device was lent by ASUS and NVIDIA, that he was not paid, and that he is free to express his opinion.

  • The ASUS Ascent GX10 is the ASUS integration of the NVIDIA DGX Spark, described as a desktop supercomputer based on the Grace Blackwell architecture.
  • The central question is whether the machine lets an AI engineer deliver reliable AI systems more quickly.
  • The test uses the latest Mistral model, Ministral 3, and follows a full project from fine-tuning to integration in a conversational interface.
  • The creator states that ASUS and NVIDIA lent the machine, that he is not paid for the evaluation, and that he is free in his remarks.
  • The DGX Spark received a lukewarm reception at launch because it was often judged on inference speed alone, which misses its purpose.
  • Compared with consumer RTX GPUs, its price-performance ratio is clearly not in its favour.

Purpose of the DGX Spark and Video Structure

The DGX Spark targets AI builders who optimize models, develop agents, and run pipelines close to production. The video is structured in three parts: the evolution of the AI engineer's work, a hands-on workflow test, and a final verdict.

  • The DGX Spark is designed for builders who optimize models, develop agents, and work on complete pipelines in production-like environments.
  • Part one covers how the AI engineer's work has evolved and the needs emerging for 2026.
  • Part two is a complete workflow test covering fine-tuning, quantization, and inference with a newly released Mistral model.
  • Part three is the verdict: whether NVIDIA met its goals, what the creator thinks of the ASUS Ascent, and who the machine is for.
  • The creator says he has never shown some of the tools and practices used in this video on his channel.

Target Audience Profiles

Three audience profiles are introduced: Lilia, an applied engineer in a large company using cloud and proprietary LLMs; Théo, a researcher in a small AI lab working with open-weight models and limited compute; and Guillaume, an enthusiast who uses a consumer RTX GPU and wonders why the Spark exists.

  • Lilia is an applied engineer in a large company who develops LLM applications mainly with cloud services and proprietary models, using RAG, long chains, and business assistants.
  • Lilia is starting to explore open source and asks how to develop her practice and career as internal models, agents, and fine-tuning become accessible.
  • Théo is a researcher in a small AI lab who works with open-weight models, has limited compute, and has no time or desire for DevOps.
  • Théo needs a stable platform with a lot of memory to train, adapt, and evaluate models end to end.
  • Guillaume is an enthusiast who already runs models locally on an RTX card and sees that RTX GPUs offer a better price-performance ratio than the Spark.
  • The creator's core question is whether the DGX Spark can be ignored without regret or whether it is the kind of machine one regrets not buying earlier.

Evolution of the AI Engineer's Work

The creator explains that an AI engineer's needs differ from an enthusiast's, and that the work has moved from prompt and RAG tuning to system design. Optimization has shifted from the model to the system around it, with planning, reasoning, and self-evaluation.

  • For an AI engineer, raw speed is not the first problem to solve; access to the right working environment matters more.
  • The creator says he has never considered buying an RTX GPU or upgrading his MacBook to run local inference.
  • Earlier applied engineering work mainly meant choosing a model, tweaking prompts, adding RAG, sometimes using chain-of-thought, and hoping it held in production.
  • Optimization happened at two levels: what the model needed to know and how it should behave.
  • Simple tasks worked, but multi-step, business, and tool-calling tasks quickly exposed limits.
  • In 2025 the problem is no longer just the model but the system around it. Engineers must design systems that plan, reason, self-evaluate, and remain controllable.
  • The centre of gravity moved from optimizing the model to optimizing the system.

Limits of Closed APIs and the Shift to Open Source

Three years ago the creator recommended proprietary models for strategic reasons. Now serious open-weight alternatives such as Mistral and DeepSeek make open source a credible option, and fine-tuning and model mastery have become central skills. Closed APIs remain efficient for quick integration but limit deeper control.

  • Three years ago the creator advised proprietary models such as GPT or Claude, chosen for speed of delivery, lower risk, and mature platforms.
  • Serious open-weight alternatives such as Mistral and more recently DeepSeek make open source a credible option rather than a gamble.
  • Fine-tuning, optimization, and model mastery are no longer advanced options but central skills.
  • The creator currently works mainly with APIs from OpenAI, Mistral, and hyperscalers, which he considers the best choice for quickly integrating a model into a product.
  • Closed-lab APIs lock users into one model family, making it hard to combine, for example, one lab's reasoning model with another's specialized model.
  • This lock-in limits testing new solutions such as DeepSeek, Gemma, or Qwen, and access to enterprise runtimes such as vLLM or NVIDIA NeMo for fine-grained performance and cost tuning.
  • The creator wants to build complex multi-LLM agentic systems under reproducible conditions between development and production.

Performance Context and Architecture

NVIDIA positions the DGX Spark as a way for many developers to access its enterprise ecosystem in a compact, quiet form. Performance depends on context such as latency, load, consumption, and stability, and comparisons are misleading without a stated goal. The architecture includes 4-bit floating point, sparse matrix processing, and 128 GB of unified memory.

  • The DGX Spark gives many people access to NVIDIA's enterprise ecosystem in a compact, silent, and efficient format.
  • The creator wants to identify the best models for clients, adapt them when needed, and quickly validate their integration.
  • The DGX Spark also makes it easy to explore NVIDIA's architectural innovations and those of the open-source community.
  • Performance depends on usage context: latency, load, consumption, and stability.
  • The DGX Spark can train nanochat, a complete open-source GPT-2-class model, in 9 days for less than $10.
  • The architecture introduces a new 4-bit floating-point format, optimized sparse matrix processing, and 128 GB of unified memory.
  • These choices directly affect the generation phase.
  • The creator argues that LLM performance should be judged on systems built for load and efficiency, as in data centers, not on a gaming machine.

The Ministral 3 Model Family

Mistral's Ministral 3 family is presented: open-weight models built for price-performance and robust function calling for agentic systems. The family has three sizes, each in three variants, for nine models in total. They are multimodal, multilingual, and available on Hugging Face. Mistral and Ministral 3 were trained on NVIDIA Hopper GPUs.

  • Ministral 3 is a family of open-weight models released by Mistral under the Apache 2.0 license.
  • Ministral 3 comes in three sizes: 3 billion, 8 billion, and 14 billion parameters.
  • Each size comes in three variants: base, instruct, and reasoning, for nine models in total.
  • The models are multimodal, multilingual, and available on Hugging Face.
  • Ministral 3 was designed with a strong focus on price-performance and robust function calling for agentic systems.
  • Mistral and Ministral 3 were trained on NVIDIA Hopper GPUs, the previous generation of NVIDIA GPUs.
  • NVIDIA works closely with Mistral, Ollama, and llama.cpp so that their solutions benefit from accelerations already available on the Spark.

Summary of Needs and Test Scope

The creator summarizes what an AI engineer needs: dev-to-prod reproducibility, freedom of model choice, access to open-source innovation, and the ability to test before industrializing. The DGX Spark is a realistic development environment, neither a high-end desktop nor a pre-production server.

  • An AI engineer needs dev-to-production reproducibility, freedom of model choice, access to open-source innovation, the ability to test before industrializing, and an understanding of trade-offs.
  • The DGX Spark is a realistic development environment, not a high-performance desktop or a pre-production server.
  • It exposes real constraints introduced by the Grace Blackwell architecture while providing access to NVIDIA's full enterprise ecosystem.
  • NVIDIA's value proposition for the DGX Spark is linking experimentation to production.
  • The test workflow is fine-tuning, quantization, and inference on a very recent model, not a simple tour of the owner's features.

Environment Validation and Initial Benchmarks

The creator checks that the environment works. NVIDIA playbooks run smoothly, and Mistral Large 2 runs with Ollama and Open WebUI thanks to 128 GB of unified memory. Benchmarks were run with llama.cpp on gpt-oss, and Ministral 3 was tested in vLLM containers from NVIDIA NGC.

  • NVIDIA provides playbooks that run perfectly, used as the first validation step.
  • The creator ran Mistral Large 2 with Ollama and Open WebUI, and it ran without trouble thanks to 128 GB of unified memory.
  • Benchmarks were run with llama.cpp on gpt-oss, and the results were compared with figures found online.
  • Ministral 3 was tested in vLLM containers available on NGC, NVIDIA's official container catalogue.
  • No problems were reported in these tests.

Understanding the Benchmarks

Benchmarks report two figures: time to first token (TTFT) for prefill and time per output token (TPOT) for decoding. The Spark has a strong TTFT but slower generation, a deliberate architectural choice caused by memory bandwidth of 273 GB/s. It can run very large models and, with two Sparks, up to 405 billion parameters.

  • The first benchmark figure is TTFT (Time to First Token), covering prefill: passing the prompt through the transformer layers and loading the KV cache.
  • The KV cache lets the model remember what it has already computed, avoiding repeated work for every generated token.
  • The second figure is TPOT (Time Per Output Token), covering the decoding or generation phase.
  • The Spark's TTFT is proportionally much better than its TPOT compared with RTX results.
  • NVIDIA prioritized loading very large models over generation speed, and memory bandwidth is limited to 273 GB/s.
  • Unified memory shared with the CPU is the direct cause of this bandwidth constraint.
  • The Spark has an NVIDIA ConnectX network interface and NVLink, which lets two Sparks be connected to load models up to 405 billion parameters.
  • The creator wants to test Mistral Large 3, a mixture-of-experts model with 51 billion active parameters, on two Sparks.

Performance Trade-offs

The DGX Spark trades raw performance for memory capacity. It delivers roughly four to five times lower performance than the latest-generation RTX, which is deliberate. It is designed for experimentation, prototyping, and optimization research rather than maximum inference performance. Ministral 3 also performs well below gpt-oss-20B on the Spark, showing the value of model-side optimization.

  • The DGX Spark is weaker in raw performance, and this is a deliberate design choice.
  • Unified memory allows experimentation with very large models that could not be loaded elsewhere.
  • Performance is four to five times lower than the latest-generation RTX GPUs.
  • The Spark is designed for experimentation, prototyping, and optimization research, not maximum inference performance.
  • Ministral 3 performance on the Spark is far below gpt-oss-20B, which already benefits from a 4-bit floating-point format.
  • The results confirm that models must be optimized to benefit from hardware accelerations.

Fine-Tuning Goals and Method

The creator does not aim to beat benchmarks but to adapt a model's behaviour to a specific case, as in a client project. He claims a recent model can be adapted locally in under an hour. Fine-tuning is defined as constraining behaviour with targeted examples, not retraining, and GRPO with Unsloth is chosen.

  • The goal is to adapt the model's behaviour to a specific use case, as on a real client project, rather than to beat benchmarks.
  • The creator claims a recent model can be adapted locally in under one hour under realistic conditions.
  • Fine-tuning means constraining model behaviour with targeted examples to improve robustness, response consistency, and ability to follow a specific logic, rather than retraining.
  • The algorithm used is GRPO with Unsloth.
  • GRPO was popularized by DeepSeek and suits reasoning and structured-generation tasks.
  • GRPO is lighter than classic RLHF pipelines, making it practical on a DGX Spark.

Unsloth and LoRA

Unsloth is presented as a toolbox for fine-tuning built on PyTorch and Transformers. Its main strength is optimized LoRA fine-tuning, which trains small adapter matrices instead of all weights. It automates training and merging the adapted weights into the base model, and it was chosen because it was ready at the right time.

  • Unsloth is a fine-tuning toolbox built on PyTorch and Transformers.
  • Its main strength is optimized fine-tuning through LoRA (Low-Rank Adaptation).
  • LoRA adds a small adaptation matrix to key layers and trains only those lightweight adapters, keeping the pre-trained weights intact.
  • LoRA reduces memory use, speeds up training, and makes single-GPU fine-tuning possible.
  • Unsloth uses PEFT (Parameter-Efficient Fine-Tuning) mechanisms from the Transformers ecosystem and automates training and merging adapted weights with the base model.
  • The Ministral 3 release was done in collaboration with Unsloth, which published quantized versions for mainstream inference engines and a complete GRPO notebook for Ministral 3.
  • The creator used that notebook as the base for his tests and adapted it for NVIDIA's official images.

GRPO Fine-Tuning Procedure

GRPO (Group Relative Policy Optimization) is a recent RLHF-style approach. The model generates several candidate solutions, the solutions are compared, and the most effective ones are reinforced. The demo trains Ministral 3 to write Python code implementing a Sudoku solving strategy. A full fine-tune took about 45 minutes on the Spark.

  • GRPO stands for Group Relative Policy Optimization and is a recent variant of RLHF-style approaches.
  • Instead of correcting one answer, GRPO teaches the model to prefer one approach over another.
  • The model generates several solutions, these are compared with each other, and the most effective ones are reinforced.
  • The demonstration trains Ministral 3 to play Sudoku, but in a non-conversational setting.
  • The goal is to generate Python code implementing an optimized resolution strategy.
  • The task suits teaching because the reasoning is explicit, the output is measurable, and strategies can be compared objectively.
  • A full fine-tune of Ministral 3 took about 45 minutes on the Spark.
  • The creator says this iteration cycle is fast, realistic, and compatible with an AI engineer's workflow, and is hard to run reliably in the cloud.

Inference Compatibility and Quantization

A compatibility problem appears at inference. Transformers 5, used by the latest Unsloth, was still a release candidate, and it requires vLLM 0.12. NVIDIA's official vLLM container was at 0.11, so the creator switched to llama.cpp, which he had already compiled and validated on the Spark, converting the model to GGUF and quantizing it.

  • The latest Unsloth release relies on Transformers 5, which was still in release candidate status during the test.
  • Running a model with Transformers 5 requires vLLM 0.12.
  • The latest official NVIDIA vLLM container was version 0.11.
  • The creator did not want to compile a full vLLM build and chose llama.cpp instead, which he had already compiled and validated on the Spark.
  • Using llama.cpp required converting the model to GGUF format, during which a quantization was applied.
  • Quantization reduces the numerical precision of model weights, for example from FP16 to 8-bit or 4-bit, to reduce memory use, speed up inference, and improve energy efficiency.
  • llama.cpp was used because it is currently one of the most robust tools for producing quantized formats such as GGUF and for quickly testing trade-offs between performance and quality.

Inference Engines: llama.cpp vs vLLM

An inference engine is the software layer that loads weights, manages the KV cache, groups requests through batching, and exploits hardware acceleration. The DGX Spark supports options such as vLLM, TensorRT-LLM, llama.cpp, and NVIDIA NIM. vLLM is designed for high concurrency and orchestration, while llama.cpp suits single-user local inference.

  • An inference engine is not the model but the software layer that loads weights, manages the KV cache, groups requests through batching, and exploits hardware acceleration.
  • Local tools such as Ollama and LM Studio face different challenges from enterprise inference engines.
  • Options on the DGX Spark include vLLM, TensorRT-LLM, llama.cpp, and NVIDIA NIM.
  • vLLM is the reference for enterprise environments but did not offer the required version in NVIDIA's catalogue at the time of the test.
  • llama.cpp is designed for single-user local inference with one or more fast GPUs and needs no orchestration or concurrent memory management.
  • vLLM is designed to serve dozens or hundreds of simultaneous requests, with advanced KV cache management, PagedAttention, and integration with orchestrators such as Kubernetes.
  • The DGX Spark is well suited to engines like vLLM because of its unified memory and capacity for sharing and scaling.

Functional Test in Open WebUI

The fine-tuned, quantized Ministral 3 model was served with llama.cpp and connected to Open WebUI, an open-source conversational interface similar to ChatGPT that can connect to Ollama, llama.cpp, or vLLM. The model was compared with official Mistral models of the same size on a Sudoku-strategy task. The fine-tuned model produced a decent algorithm with a few bugs, which the standard models did not achieve.

  • Open WebUI is an open-source conversational interface that offers many ChatGPT-like features and connects to engines such as Ollama, llama.cpp, or vLLM.
  • The test was a functional validation rather than a complete evaluation.
  • The fine-tuned model was compared with official Mistral models of the same size in instruct and reasoning versions.
  • The task was generating a Sudoku resolution strategy in Python, and the algorithm was compared with one produced by ChatGPT.
  • The fine-tuned model quickly produced an algorithm that was not bad but had a few small bugs.
  • The standard models did not produce usable results on this task.

Workflow Results

The creator presents the full cycle as a success despite rough edges that came from young software components. The workflow covered adaptation, quantization, inference, and conversational testing, all locally on the Spark with a production-like stack.

  • The full cycle was completed locally: fine-tuning, quantization, inference, and conversational interface testing.
  • The creator encountered pitfalls during the workflow, which came from the immaturity of some solutions he deliberately used.
  • The main value is the ability to traverse the full lifecycle of an AI project with an industrial stack while keeping freedom to try open-source optimizations.
  • The creator says the scenario was deliberately demanding: a model just released, a software stack still in motion, latest-generation hardware, and tools not fully aligned.

Answering the Three Key Questions

The creator answers three questions. Can the full project lifecycle be completed without cheating? Yes. Does the Spark give more freedom? Yes, because the engineer can use, compare, and optimize open-weight models in one place. Does it make work faster? Yes, because the engineer is rarely blocked and can switch tools, not because the machine is faster.

  • Question 1: Could the full project lifecycle be completed without cheating? Yes, with adaptation, quantization, inference, and functional testing in a conversational interface.
  • Question 2: Does the engineer gain freedom? Yes, the Spark removes dependence on a model provider, an API, or an imposed abstraction.
  • On the Spark, users can use, compare, and optimize all open-weight models in one place, choose inference engines, and go down to low-level code when needed.
  • The creator says this combination of freedom and access to the full NVIDIA enterprise stack is something he could not have had elsewhere, and that the same work would be very costly in the cloud.
  • Question 3: Does the Spark make the engineer faster? Yes, because the engineer is never blocked and can change tools when needed, not because the machine is faster.
  • The creator argues the DGX Spark does not remove complexity but makes it manageable.

Who the Machine Is For

The creator says the DGX Spark is a practical access point for engineers like Lilia who feel cloud limits, and a strong choice for researchers like Théo working with open-weight models without infrastructure. He admits he had not budgeted for the machine but may keep it if ASUS allows a longer loan.

  • For engineers like Lilia who feel that cloud-only work limits progress, the DGX Spark is described as an access ramp to another level of practice.
  • For researchers like Théo, working on open-weight models without an infrastructure team or data center, the creator says it is probably one of the best choices.
  • The creator had not included the machine in his budget but thinks he may keep it if ASUS lends it longer.

Context, Democratization, and Price

The creator presents the DGX Spark as a rare early access to Grace Blackwell technology. Ministral 3 was trained on Hopper, while the creator works on Blackwell, so individual AI engineers get the same class of hardware that leading labs use at the same time. The ASUS Ascent is said to cost about 1000 euros less than the DGX Spark, at about 3000 euros.

  • The DGX Spark embeds an architecture designed for future data centers yet is immediately accessible to individual builders.
  • Such technology usually arrives years later, but here it is available at once.
  • Ministral 3 was trained on Hopper, while the creator works on Blackwell.
  • Individual AI engineers now have access to the same hardware technology level as the best labs at the same time.
  • The creator says this is a real democratization of technology rather than marketing.
  • The ASUS Ascent is said to be about 1000 euros cheaper than the DGX Spark, at about 3000 euros.
  • Given the hardware and the software stack, the creator says it is hard to offer better.

Final Verdict

The verdict is yes: the DGX Spark enables more reliable AI systems to be delivered faster, not because it is fast but because it makes serious work possible. It complements the cloud rather than replacing it, and it lets engineers prepare solid, understood solutions before large-scale deployment.

  • Verdict: the DGX Spark does enable faster delivery of more reliable AI systems.
  • The reason is not speed but that it makes serious work possible.
  • The DGX Spark does not replace the cloud; it complements it.
  • It lets engineers prepare solutions that are robust, understood, and controlled before large-scale deployment.
  • The creator says it is the machine he would have wanted to have earlier.
  • He thanks NVIDIA and ASUS for the loan.