Antirez's ds4 Engine Runs DeepSeek V4 Flash Locally at 27 tok/s — The End of Cloud-Only AI Coding?

Antirez Ships a Local Inference Engine for DeepSeek V4 Flash

Salvatore Sanfilippo — better known as antirez, the creator of Redis — has released ds4, an open-source inference engine written in pure C and optimized specifically for running DeepSeek V4 Flash on Apple Silicon. The numbers are striking: on a 128 GB MacBook Pro, ds4 achieves 27 tokens per second with a 1 million token context window. That is fast enough for interactive coding assistance with no cloud dependency, no API key, and no per-token cost.

This is not a wrapper around llama.cpp or another existing runtime. ds4 is a ground-up implementation targeting the specific architecture of DeepSeek's Mixture-of-Experts (MoE) models, with three key technical innovations that make running a model this large on consumer hardware practical: asymmetric 2-bit quantization for the MoE layers, SSD-backed KV cache to break memory limits, and pure Metal GPU optimization for Apple Silicon.

The Three Technical Breakthroughs

Running DeepSeek V4 Flash locally is a hard problem. The model uses a MoE architecture with hundreds of billions of total parameters, even though only a fraction activate per token. Fitting the model weights, KV cache, and runtime state into consumer hardware requires aggressive engineering at every layer. Here is what antirez built:

1. Asymmetric 2-bit MoE quantization. The expert layers in a MoE model account for the vast majority of parameters but are sparsely activated. ds4 quantizes these expert weights to 2-bit precision while keeping the shared attention layers and routing logic at higher precision. This asymmetric approach preserves the model's core reasoning quality (which depends on the attention and routing layers) while dramatically reducing the memory footprint of the dormant experts. The total model size drops to roughly 40-50 GB on disk, fitting comfortably in the 128 GB unified memory of a top-spec MacBook Pro.

2. KV cache on SSD. With 1 million tokens of context, the KV cache alone can consume 20-40 GB of memory depending on quantization. ds4 offloads the KV cache to NVMe SSD storage, using memory-mapped I/O with intelligent prefetching to keep the active portion of the cache in RAM while older context lives on the fast SSD. Apple Silicon's unified memory architecture and the high-speed NVMe controllers in modern MacBooks make this approach viable — sequential SSD read speeds of 6-7 GB/s mean that even cache misses add only single-digit milliseconds of latency per token.

3. Pure Metal optimization. Rather than targeting CUDA (NVIDIA) or generic compute APIs, ds4 is written directly against Apple's Metal framework. This means the GPU kernels are hand-tuned for the specific architecture of M3 Pro, M3 Max, M4 Pro, and M4 Max chips. The result is significantly better GPU utilization than generic frameworks — antirez reports that ds4 achieves near-theoretical memory bandwidth utilization on the M4 Max, which is the primary bottleneck for token generation in autoregressive decoding.

Cost Comparison: Local vs. Cloud API

The economics of local inference versus cloud API access depend heavily on usage volume. Here is a breakdown assuming a developer generates roughly 2 million output tokens per month (approximately 100 coding sessions per day at 700 output tokens each):

The amortized hardware cost assumes a $4,800 MacBook Pro (128 GB M4 Max) with a 3-year lifespan, used for both development work and local inference. Since the laptop is not dedicated AI hardware — you would own it anyway — the real marginal cost of running ds4 is just electricity, which at typical US rates comes to roughly $3-5 per month for heavy usage. That makes local inference with ds4 effectively free at the margin.

Even against DeepSeek V4 Flash's already rock-bottom API pricing of $0.28 per million output tokens, local inference with ds4 is cheaper at moderate to high usage volumes. The crossover point is roughly 500,000 output tokens per month — if you generate more than that, local wins on pure cost.

27 Tokens Per Second: Is It Fast Enough?

The 27 tok/s figure needs context. Cloud-hosted DeepSeek V4 Flash via the API typically delivers 80-150 tok/s depending on server load and request size. So ds4's local inference is roughly 3-5x slower than the API. Is that usable for coding?

At 27 tok/s, generating a 500-token function implementation takes about 18 seconds. A 2,000-token file with multiple functions and comments takes about 74 seconds. For interactive "write this function" workflows, this is adequate — you fire off the request, switch to another tab for a moment, and come back to a completed result. It is slower than the instant-feeling cloud API responses, but it is fast enough to maintain productive flow for most coding tasks.

Where 27 tok/s falls short is in agent-style workflows that chain many sequential generations — the kind of multi-step coding loops that tools like Claude Code and Cursor run. If an agent needs 10 sequential generations of 500 tokens each to complete a task, that is 185 seconds of wall time on ds4 versus 30-60 seconds on the API. For heavy agent users, the cloud API remains the better choice until local inference speeds improve.

Developer Sovereignty: Why Local Inference Matters

The cost and speed comparisons tell part of the story. The deeper significance of ds4 is what it represents for developer sovereignty over AI coding tools. When you run a model locally, several important things change:

• Privacy: Your code never leaves your machine. For developers working on proprietary software, regulated industries, or sensitive personal projects, this eliminates an entire category of risk. No API provider sees your code, no data retention policies apply, and no third-party breach can expose your intellectual property.
• Availability: No rate limits, no API outages, no degraded performance during peak hours. Your inference engine works as long as your laptop has power. This is particularly valuable for developers in regions with unreliable internet or strict network policies.
• Control: You choose the quantization level, the context window size, and the system prompt. You can modify the engine's behavior, contribute to the open-source project, and customize it for your specific workflow. No provider can deprecate your model or change its behavior without your consent.
• Cost predictability: No surprise bills, no variable pricing based on demand, no promotional discounts that expire. Once the hardware is amortized, the cost is fixed and near-zero regardless of how much you use it.

These properties matter most for individual developers and small teams who cannot afford the complexity of enterprise data processing agreements and who value independence from vendor lock-in. Antirez — whose Redis philosophy always emphasized simplicity and developer autonomy — has built a tool that embodies the same values for AI-assisted coding.

The Hybrid Future: Local for Privacy, Cloud for Power

ds4 does not make cloud APIs obsolete. There are clear scenarios where each approach wins:

The most cost-effective setup for a serious developer in 2026 is likely a hybrid: run ds4 locally for routine coding tasks and privacy-sensitive work, and use cloud APIs for frontier-quality reasoning and high-throughput batch processing. This captures the cost advantage of local inference (near-zero marginal cost) while retaining access to models like Claude Opus 4.7 ($5/$25) and GPT-5.5 ($5/$30) when maximum capability is needed.

Compare Your Options

Whether you are considering local inference with ds4, sticking with cloud APIs, or building a hybrid workflow, the starting point is understanding your actual token usage and costs. Different project types — a CLI tool versus a full-stack application versus a data pipeline — generate wildly different token volumes and benefit from different model choices.

Use the AI Cost Estimator to calculate your cloud API costs across 40+ models, then compare against the effective local inference cost to find the right balance for your workflow and budget.