Redis Creator Personally Wrote an Inference Engine for DeepSeek V4 Flash, Runs Only on Mac

On May 8, Salvatore Sanfilippo—better known as antirez, the creator of Redis—dropped a new project called ds4 on GitHub. In one sentence: it is a local inference engine built specifically for DeepSeek V4 Flash, written in C and Objective‑C, running only on Apple Silicon via the Metal backend. No CUDA, no ROCm, no CPU.

It sounds completely counter‑trend. While everyone is racing to optimize general inference frameworks such as vLLM, SGLang, and llama.cpp, one veteran programmer—who spent 11 years building Redis—set out to write a single‑purpose, hardware‑bound engine for one specific model. But when you look at its design, it actually makes quite a bit of sense.

The Motivation: Fitting a 284B MoE Model into a Mac

Let’s recap. On April 22, DeepSeek released the V4 series under an MIT license, along with full weights:

• V4‑Pro: 1.6 T parameters total, ≈49 B active, ≈1 M‑token context, flagship edition
• V4‑Flash: 284 B total parameters, 13 B active, same architecture, efficiency‑focused

The appeal of the Flash variant is clear—it only has 13 B active parameters, meaning it can theoretically fit onto consumer‑grade hardware once quantized. Community projects like Unsloth have already produced GGUF versions runnable on 2 × 48 GB VRAM setups. But antirez wanted something different: make V4 Flash not just “runnable” on a Mac, but actually “usable”—able to power coding agents, handle long prompts, and reliably invoke tools.

Hence ds4.c was born.

Project Layout: A Deliberately Narrow Path

Open the repo and you’ll notice how unusually small the project is. Language breakdown:

• C     55.4 %
• Objective‑C     30.2 %
• Metal Shading Language     13.8 %

No Python glue layer, no PyTorch dependency, no runtime framework, no abstractions for multiple backends. Metal‑only. Which means it runs solely on Apple Silicon GPUs—your 4090 or MI300 won’t help.

antirez explains in the README quite plainly: general frameworks must abstract to support all models, but abstraction brings compromise. He wanted a narrow road—bet on one model at a time, validate numerics with official logits, test long contexts, run full agent integrations, and confirm it’s truly usable.

It’s the opposite philosophy of llama.cpp: that project aims to support hundreds of architectures, dozens of backends, and many quantization formats; ds4.c has one Metal‑based graph executor for V4 Flash plus a tailor‑made GGUF loader, prompt renderer, KV state manager, and server API for it.

Incidentally, antirez candidly notes that ds4.c was forked from llama.cpp/ggml—but the main path is now fully DeepSeek V4 Flash–specific code. Very antirez.

Performance: Real‑World Numbers on M3 Max

Benchmarks straight from the source:

MacBook Pro M3 Max (128 GB Unified Memory), 2‑bit quantization, 32 K context:

• Short‑prompt prefill: 58.52 tokens/s
• Generation: 26.68 tokens/s

Mac Studio M3 Ultra (512 GB Unified Memory), long prompt (11 709 tokens):

• Prefill: 468.03 tokens/s
• Generation: 27.39 tokens/s

Think about that: a 284 B‑parameter MoE model stably emitting 26 tokens/s on a laptop—faster than a human reading speed, fully adequate for real‑time coding‑agent interaction. The Mac Studio’s 468 tokens/s prefill means inserting a 25 K‑token Claude Code‑style system prompt takes only tens of seconds before normal conversation begins.

Key Technique: Carefully Tuned Hybrid Quantization

This speed doesn’t come from brute‑force 2‑bit quantization. ds4 uses a specific hybrid scheme:

• MoE Expert Routing Layers (majority of model size): up/gate use IQ2_XXS, down use Q2_K
• Shared experts, projection, routing layers: retain Q8 precision

Smart choice: sparse experts inherently tolerate 2‑bit loss; shared and routing layers are critical paths—if precision drops, routing collapses, and coding‑agents lose tool‑call accuracy.

In the README, antirez writes confidently:

These 2‑bit quantizations are no joke—they perform well in coding‑agent scenarios and reliably invoke tools.

And that’s the metric local inference actually cares about today—not perplexity or MMLU, but tool‑call reliability.

Full‑Stack Local Inference: From Engine to Agent Protocol

ds4 is more than an inference kernel—it ships with an HTTP server that supports two protocols:

• /v1/chat/completions – OpenAI API
• /v1/messages – Anthropic API

Tool‑calling is adapted accordingly. The README includes three client examples: opencode, Pi, and Claude Code.

A particularly nice touch: disk‑based KV cache. Agents like Claude Code send a ≈25 K‑token system prompt at launch—re‑prefilling each time would be painful. ds4 writes the KV state to disk after the initial prefill, then restores it instantly on next session.

Taken together, antirez isn’t building “just an inference framework” but a vision of full‑stack local inference:

• A GGUF format tuned precisely for this engine
• A Metal‑bound executor
• An HTTP API speaking both agent protocols
• A verified test suite integrated with real coding‑agents

Not standalone modules combined—but a product designed end‑to‑end.

One Model, One Engine—Can This Work?

Here lies the controversy: if the engine serves only one model, what happens when V5 arrives?

antirez acknowledges this plainly: ds4 currently bets on V4 Flash; the model might change. Metal‑only is its current form—CUDA could come later, but he cautions, “perhaps, but that’s all.” The project remains intentionally small, fast, and focused.

Yet it may signal a new trend:

As soon as a vendor releases a new model, someone in the community builds a dedicated engine, quantization, and agent integration for it—each model having its own antirez.

This “one model one engine” approach can fill early‑stage, high‑quality local‑deployment needs before general frameworks finish adapting to new architectures and kernels. It doesn’t need long‑term survival—just to be excellent while V4 Flash is the talk of the town.

In a sense, it challenges the “unified framework” philosophy. llama.cpp’s greatness is in abstracting everything so any model runs; ds4’s stance is the opposite—abstraction has a cost, and customization for a single goal raises the ceiling. Both paths can coexist.

Another Point Worth Watching: AI‑Assisted Coding

A notable line in the README: ds4 was developed with heavy assistance from GPT‑5.5, with human responsible for ideas, testing, and debugging.

Two weeks—from forking llama.cpp for adaptation to writing an almost entirely new engine, plus Metal kernels, dual‑protocol server, disk KV cache, and agent integration tests—an impossible workload for one person pre‑AI assistance.

antirez is no ordinary programmer, yet he isn’t an ML infra engineer either. Producing this project demonstrates how far AI‑assisted coding can go in low‑level C systems work. That may be even more important than ds4 itself.

Incidentally, DeepSeek V4 Flash has joined the OpenAI Hub, so developers can first test business logic via the cloud API, then switch seamlessly to ds4 for local/private or long‑prompt caching—protocol compatibility makes that effortless.

A Bit of Old‑School Obsession

We can’t ignore antirez himself. Born in 1977 in Sicily, founded Redis in 2009, led it for 11 years, left in 2020. After Redis, he built:

• Kilo – a text editor under 1000 lines of C
• dump1090 – ADS‑B air‑traffic signal decoder
• linenoise – a tiny replacement for readline
• RF protocol analyzers and Asteroids ports for Flipper Zero
• WOHPE (2022) – a science‑fiction novel

On his blog he wrote a widely quoted idea: “Code is a crafted artifact, not merely a useful tool.” He’d rather be remembered as a bad artist than a good programmer.

ds4 is quintessentially antirez—it’s not about market share or infrastructure; it’s “I want this model to run well on my Mac,” and he spent two weeks building it. This kind of single‑minded, hardware‑bound project is increasingly rare in open source today.

At the end of the README, he offers a thought to modern programmers:

Modern programming is becoming complicated and boring—just layers of glue. It’s losing much of its beauty. Most programmers face neither the artistic nor the engineering side of programming anymore.

The ds4.c project is, in its own way, a counterexample to that.

References

• antirez/ds4 – GitHub: main repository—including full performance data, quantization details, CLI/server usage, and agent integration examples
• Redis creator writes a C‑only inference engine for DeepSeek V4 Flash – linux.do: Chinese developer community discussion and analysis
• Tested DeepSeek v4 flash with some large code change evaluations – Reddit r/LocalLLaMA: community feedback on tool‑call accuracy for V4 Flash in coding‑agent scenarios