HN Showcase
Back to browse
GitHub Repository

DeepSeek 4 Flash local inference engine for Metal and CUDA

2 starsC

Qwen3.6-35B-A3B on a 16 GB M1 Pro with SSD-streamed MoE

by andreaborio·Jul 17, 2026·2 points·0 comments
Visit ProjectView on HN

AI Analysis

●●SolidBig BrainNiche Gem

SSD-streamed MoE lets 16GB M1s run 35B models, but it's a specialized fork of antirez's ds4.

Strengths
  • •Implements adaptive Metal residency and routed-expert cache policies across memory tiers.
  • •Treats SSD as an active model-memory tier with awareness of page cache and swap.
  • •Focuses on correctness tests and incremental quantization rather than just raw speed hacks.
Weaknesses
  • •Explicitly states it does not aim to replace upstream, limiting its standalone value.
  • •Arbitrary GGUF files are not expected to work, restricting it to specific model architectures.
Category
AI/ML
Target Audience

Developers with Apple Silicon Macs running large local models

Similar To

llama.cpp · MLX · Ollama

Similar Projects

AI/ML●●●Banger

SwiftLM – Qwen Chat on iPhone, 100B+ Moe on M5 Pro 64GB (Native Swift)

Native Swift inference with SSD streaming runs 100B MoE models without kernel panics.

WizardryNiche Gem
aegis_camera
123mo ago
AI/ML●Mid

Qwen Lens Studio – multimodal app on Qwen3.6-35B-A3B, runs on Ollama

Yet another multimodal wrapper when Cursor and Continue already dominate this space.

Ship It
vijgaurav
302mo ago