Strata is a specialized inference engine built for one model, Qwen 3.8 Flash Next. The author reports strong performance but advises against installing it on local inference machines. The author's main concerns are supply-chain and installation security, along with the design lessons Strata offers for general-purpose inference frameworks such as llama.cpp.

Summary of Recommendations

• Do not install Strata on a home or work inference machine.
• If it is used anyway, run it inside a container, or wipe the machine after installation.
• Do not install it by downloading random binaries, scripts, or markdown files and passing them to an AI agent.
• Inference servers should not have broad permissions or persistent access to the internet.
• Preferred deployment model: local, offline inference.

Security Assessment

The author considers Strata to violate three basic rules of software installation on inference machines.

1. Running binaries from an unvetted source. Strata's binaries come from a GitHub repository that is about two weeks old. The author states that this practice is inadvisable for any tool, not only Strata.
2. Installing through scripts and markdown files fed to an AI agent. The author considers this approach equally unsafe.
3. Automatic downloads during setup. The setup.py file downloads components from GitHub without asking the user. The author states that the code may not be malicious but that the behavior is unacceptable.

The author also describes the following risk factors:

• Strata has about 17,000 GitHub stars. The author estimates that the number of installations could be ten times higher. This scale makes the project an attractive attack target.
• The author describes 2026 as a low-trust environment. Even if a project is not compromised at present, the author recommends treating it with suspicion.
• The author applies the same concern to OpenClaw.

The author contacted the individual who appears to control the GitHub repository. The author reports that this person agreed that the security issues exist and said he would address some of them.

Performance Measurements

The author tested Strata on a home machine with an NVIDIA RTX 4090 and in a RunPod container.

Decode speed decreases as the context window fills.

Thread Configuration Issue

The home machine has 128 physical cores and 256 threads. The RunPod container had about 32 threads allocated. Strata created too many loader threads, which reduced its performance. The author recommends checking the num_workers flag. An AI agent identified the issue quickly.

Design Principle

The author states that Strata's design follows from one principle: keep the GPU busy at all times during decode and prompt processing. The author identifies two main techniques that support this principle: parallelism and caching.

Profiling Analysis

The author captured performance data with NVIDIA Nsight and produced a visualization. An AI agent assembled the frames, added metadata, and produced the sequence.

Visualization Method

• Each frame covers a 1/30-second window.
• The function shown for each frame is the kernel with the largest total overlap with that window. The kernel is not necessarily the longest or the most recent one.
• The example workload is a prefill of 4096 bytes in llama.cpp.
• mul_mat_q is the matrix-by-matrix kernel in llama.cpp.

Lane Definitions

• Teal lane: GPU busy.
• Orange-yellow lane: Memory transfer over PCIe, triggered by CUDA's cudaMemcpy.
• Pink: GPU idle.
• White line: CPU usage. A line at the top indicates one core in use. Gray indicates more than one core in use.

Results

• llama.cpp: The teal lane is not full. The author gives llama.cpp a grade of D on this metric.
• Strata: The teal mass is complete and finishes in 4.8 seconds.

The author notes that the pink idle periods in the llama.cpp trace were reconstructed after the fact.

Technical Mechanisms

The author cites a video by Codacus that explains Strata's mechanisms in detail. The author spent five to six hours reading the code, reviewing it with AI agents, and running Strata alongside llama.cpp and llama.cpp forks that aim for similar results. The author states that the Codacus video is accurate according to the author's understanding.

Strata uses the following methods:

1. Keeping hot experts in VRAM. The engine keeps the most frequently used experts in video memory.
2. Recalculating hot experts during execution. The engine updates the set of hot experts as inference proceeds.
3. Not using a simple LRU cache. The engine maintains a distribution of expert usage frequency.
4. Anticipating cache misses. The engine predicts upcoming misses and streams the required experts into VRAM. The GPU continues to work while the next layer's experts load.

Separate Pack File

Strata uses a separate binary pack file for the model. The GGUF format already contains all data needed to run a model. However, in a GGUF file, a single expert is spread across three different tensors. The pack file stores each expert in a single format on disk, which allows experts to be cached efficiently.

Implications for llama.cpp

The author argues that specialized frameworks will become more common and more important. The author names ninfer, Strata, FreeToken, and DeepSeek or Dwarf Star as examples. The author states that the GGML developers are moving in the wrong direction on this issue.

The author makes the following predictions and observations:

• The author predicts that these frameworks will accumulate more GitHub stars than llama.cpp.
• The author states that llama.cpp does not handle community contributions well.
• The author reviewed 15 pull requests to llama.cpp that add MoE expert caching in VRAM. The author could not determine whether the maintainers declined to merge them or did not review them.
• The author states that llama.cpp lacks efficient MoE execution.

Update

After the initial publication of the author's analysis, Georgi merged a GPU cache for MoE experts into llama.cpp on a Wednesday earlier that week.

The author states that a loss of relevance for llama.cpp would be undesirable. The author values the ability to clone the repository, copy it to a server, and run make. The author also states that blind trust in highly internet-connected new frameworks, such as Strata, would be a serious problem if llama.cpp lost relevance.

The Author's Fork

The author created a fork of Strata for offline use. The author instructed AI agents to:

• Remove all Python code.
• Remove all downloads performed at build time.
• Rebuild the tokenizer in C++.
• Replace the templating and front-end server code, previously in Python, with C++.

The fork works in a manner similar to the llama.cpp model. It requires no network access to build and does not download binaries. The author describes it as a simplified version of Strata that does not include additional features.

The author has not stress-tested the fork to the extent that others have tested Strata.

Build Procedure

1. Install the CUDA drivers on the inference machine.
2. Install the software from the CD-ROM included with the NVIDIA 5090 GPU.
3. Download the source archive and copy it to an offline machine.
4. Run make.

The author states that, when the fork is used as intended, the author cannot observe its deployment.

Commentary on Content Creators

The author advises caution toward influencers and content creators who promote products they use. The author notes that some creators may be vibe coding projects and publishing them without adequate review.

Analysis of Strata's Development

The author describes Strata as the work of a person with deep technical expertise who spent weeks planning and designing it. The author states that the planning traces and conversations were not committed to the repository, which begins with the first commit of source code.

An AI agent, identified as Astra, assessed Strata's techniques as follows: the project cites several papers, uses some ideas from llama.cpp, and incorporates several kernels.

Base Model Outputs

The author tested the base model's general capabilities. The model produced:

• A pagoda.
• A platformer game in which a character named Dario collects money from lakes. The author plans a full video on this output.
• A graphic calculator.

The author describes the model as strong. The author notes that a 177-billion-parameter model from Qwen is expected to perform well.

Conclusion

The author compares two ways of experiencing technological breakthroughs:

• Reading the PageRank paper and the Chord distributed hash table paper produced a sense of insight. The author notes that both ideas underpin real systems, including the large databases that index the web for Google.
• Garry Kasparov's reaction to IBM's Deep Blue illustrates the second way. Kasparov accused the team of secretly using grandmasters after losing to the program. He saw genius in the computer's play for the first time.

The author reports a similar reaction on understanding Strata. The reaction intensified after comparing Strata's traces with llama.cpp, where the author observed poor pre-fetching and GPU idle time.

Final Assessment

• Strata shows strong performance on the tested workloads.
• Strata has significant security risks in its installation and update behavior.
• The author recommends against installing Strata on an inference machine with internet access.
• The author recommends the fork for offline use, with the limitations noted above.