A recent experiment tested eight programming languages on a $12 server with a single CPU core and 2GB of RAM to determine their maximum throughput under a standard configuration. Popular frameworks for each language, the PostgreSQL DBMS, and the Nginx web server were used as the base. The goal was to build an identical API in each language and gradually increase the number of virtual users accessing it until the server could no longer meet the specified criteria. Rust with SQLite showed the best result in the initial test, handling approximately 14,000 concurrent users.

In response, a GitHub repository was created with the task conditions, inviting expert programmers to create the most performant implementations in various languages. Implementations in 13 languages were received, and the best among them managed to support 64,000 concurrent users on the same $12 server, more than quadrupling the initial result.

This article examines the resulting implementations, analyzes the optimizations applied that enabled such high performance, evaluates the trade-offs developers had to make, and discusses the applicability of these methods in real-world production environments.

Testing Methodology

The experiment was conducted on a $12 server with a single shared CPU core, 2GB of RAM, and 50GB of disk space. Developers were tasked with implementing a simple social media API, similar to Twitter, comprising five endpoints: fetching a feed, viewing a post, liking a post, creating a post, and a health check.

Each implementation had to pass a suite of 42 tests, with API responses strictly adhering to the specification. SQLite was used as the database, running within the same process as the application. The use of separate database servers like Redis or PostgreSQL, or dedicated caching servers, was prohibited. Every authenticated request required a signature check of the login token. When creating a post or liking a post, the transaction had to be committed before sending a confirmation to the user. While multiple writes could be batched into a single transaction, the response to the user could not be sent until the transaction was committed, which was crucial for latency criteria.

Otherwise, developers had the freedom to choose their optimizations; they could use frameworks, build their own HTTP servers, or forgo Nginx.

For performance testing, the K6 tool was employed to simulate thousands of virtual users. Each user mimicked real human behavior: fetching their feed, viewing posts, and occasionally liking or creating posts. The number of users was ramped up until the server failed to meet the requirements. The criteria for passing a level included sustaining the load for 5 minutes with 95% of requests completed in under half a second, 99% in under 1 second, and an error rate below 1%. The objective was to measure sustained performance, not short-lived peak loads.

Load Testing Specifics

Unlike the previous video, where a single virtual machine was used to generate traffic, in this experiment, some implementations turned out to be so fast that the load generator became a bottleneck before the $12 server itself. The machine generating the load ran out of RAM due to attempting to simulate too many users. To solve this problem, four additional virtual machines were rented to generate traffic against the target server.

Results and Analysis of Implementations

Implementations were obtained in the following languages: C, C++, Rust, Zig, Go, TypeScript, Python, Bun, PHP, Ruby, Erlang, Fortran, and R (TypeScript on Bun). Of the 28 implementations received, 26 were tested. One C++ implementation used an in-memory database, which violated the rules, and another R implementation failed to start due to a missing system library. Submissions for Java and Node.js were not received.

Ruby

Two Rails implementations achieved 3000 and 3500 users respectively. The fastest Ruby implementation, utilizing the Iodine web server (written in C) and the YJIT compiler, reached 14,000 users. Optimization involved generating JSON responses directly within SQLite, bypassing application-level serialization. At 14,000 users, P95 latency was 50ms, P99 was 154ms, error rate was 0%, and CPU load was 62%. At 15,000 users, P99 latency exceeded 1 second.

Erlang

The Erlang implementation using the Cowboy web server reached 15,000 users. Optimization included tuning the runtime to a single scheduler and dedicating one process to manage the SQLite connection. JSON generation was also performed within SQLite. At 15,000 users, P95 latency was 171ms, P99 was 320ms, CPU load was 83%, and RAM usage was 600MB. At 16,000 users, P95 latency exceeded 4 seconds.

Python

The fastest Python implementation achieved 22,000 users, a tenfold increase from the initial test with FastAPI and PostgreSQL (2150 users). Optimization involved removing most layers of the FastAPI/Starlette framework, allowing requests to pass directly from Uvicorn to the handler. At 22,000 users, CPU load was 91%, P95 was 353ms, P99 was 653ms, and the error rate was 0%. At 23,000 users, P95 latency exceeded 1 second.

Go

The Go implementation reached 22,000 users. A single Go routine was used to handle database operations, batching up to 256 requests at a time, performing operations, committing transactions, and sending responses. This method is known as "group commit." At 22,000 users, P95 latency was 143ms, P99 was 318ms, and the error rate was 0.5%. At 23,000 users, the error rate exceeded 1%. CPU load was 78%, with the bottleneck being the number of connections.

C#

The C# implementation achieved 32,000 users. The Kestrel web server was used with minimal middleware. Memory optimization included using buffer pools for constructing JSON responses and allocating memory on the stack for token validation. At 32,000 users, P95 latency was 346ms, P99 was 731ms, and the error rate was 0%. CPU load was 93%. At 33,000 users, P95 latency exceeded 1 second.

Fortran

The Fortran implementation reached 38,000 users. The C library libmicrohttpd was used for handling HTTP connections. JSON responses were generated using SQLite's JSON function, and token validation was performed via SQL queries. "Group commit" was used for batching database writes. At 38,000 users, P99 latency was 944ms, CPU load was 85%, and RAM usage was 700MB. At 39,000 users, P99 latency exceeded 1.7 seconds.

PHP

The PHP implementation using Swoole (a C extension for PHP that allows PHP to run as a long-lived server) achieved 40,000 users, over 50 times the performance of Laravel (750 users) from the previous video. With the PHP JIT compiler enabled, the application started once, opened a database connection, and prepared statements. At 40,000 users, P95 latency was 189ms, P99 was 853ms, and the error rate was 0%. At 41,000 users, P95 latency was 182ms, but P99 exceeded 2.4 seconds.

C

The fastest C implementation reached 46,000 users. The Linux io_uring interface was used for asynchronous system call handling. The SQLite library itself was optimized by compiling with profiling for the specific workload. The feed retrieval query was rewritten to avoid recounting likes. The implementation showed low RAM usage (39MB). At 46,000 users, P95 latency was 54ms, P99 was 178ms, and the error rate was 0%. At 47,000 users, P99 exceeded 1 second. The bottleneck was likely connection management or internal queues.

TypeScript on Bun

The TypeScript implementation using Bun achieved 47,000 users. Instead of Bun's standard HTTP server, low-level TCP sockets were used with minimal HTTP parsing. JSON responses were generated directly within SQLite and returned as raw bytes. A reusable output buffer and a C library were used to increase connection backlog. At 47,000 users, P95 latency was 96ms, P99 was 356ms, error rate was 0%, CPU load was 72%, and RAM usage was 150MB. At 48,000 users, P95 latency exceeded 12 seconds.

Rust

Two Rust implementations were received. The first, using the Axum framework, reached 43,000 users. It utilized a single thread with a dedicated task for database handling and batch writes. At 43,000 users, errors were observed, and at 52,000 users, the error rate was 6%. It consumed 1.1GB of RAM.

The second Rust implementation used IO Uring, allowing direct kernel-level management of network connections. Database writes were handled in batches. This implementation achieved 64,000 users with a P99 latency of 224ms, while utilizing half the CPU load and less than 100MB of RAM.

Zig

The Zig implementation reached 61,000 users. The C library uSockets was used, along with "group commit" for batching writes. The limitation was a hardcoded value of 60,000 persistent connections. At 62,000 users, P99 latency was 268ms, and the error rate was 1.02%. CPU load was 69%.

C++

Two C++ implementations reached 61,000 users. One of them, which took first place, achieved 64,000 users. It used a single thread, a single event loop, and direct access to SQLite. SQLite's page cache handling was optimized by reducing the internal cache and using memory mapping. Checkpoint operations were moved out of the request handling path. This implementation used write-ahead logging mode with synchronous=normal, implying a trade-off in data durability during sudden power loss. At 64,000 users, P99 latency was 910ms, and CPU load was 68%.

Key Optimizations

Most of the fastest implementations used the following approaches:

1. Direct HTTP Request Handling: Forgoing Nginx and using language-native web servers to minimize overhead.
2. Prepared and Reused SQLite Statements: Database queries were parsed once and reused with a single connection.
3. JSON Response Generation in SQLite: SQLite directly generated JSON responses, bypassing application-level serialization steps.
4. "Group Commit" for Batch Writes: Multiple write operations were batched into a single transaction to reduce commit overhead.
5. Framework Swapping: Developers built their own servers, memory managers, and event loops, eschewing standard frameworks.

Компромиссы и выводы

Несмотря на впечатляющие результаты, многие из этих оптимизаций сопряжены с компромиссами, которые могут быть неприемлемы в производственных средах. Например, реализация на C++ объемом более 1300 строк кода включала собственный парсер HTTP и JSON. Быстрая реализация на Rust использовала много unsafe кода и зависела от современных функций Linux. Реализация на Bun была оптимизирована под конкретный API и менее расширяема.

"Group commit" может привести к тому, что при сбое одной транзакции могут завершиться неудачей все ожидающие запросы. Использование одного соединения с базой данных требует тщательного планирования расписания запросов, чтобы избежать блокировок.

В производственных условиях, где важны поддерживаемость, безопасность, переносимость и простота понимания кода, более реалистичными могут быть результаты, полученные в предыдущем видео с использованием стандартных фреймворков.

Репозиторий с кодом всех реализаций доступен для ознакомления и дальнейших улучшений. Особый интерес представляют дальнейшие разработки на Java, Node.js и Go.

Для лучшего понимания системного дизайна рекомендуется использовать приложение DevMax, предлагающее ежедневные задачи по системному дизайну и краткие уроки. Также полезно участие в Discord-сервере для обсуждения с коллегами и получения ответов на вопросы. Понимание системного дизайна становится все более ценным навыком в эпоху развития ИИ.