Edge0's 35B MoE Aims for SSD-Backed Edge Inference

Edge0 has published a preview of Edge0-35B-A3B, a mixture-of-experts language model designed for a specific and often awkward problem: running large models on hardware that doesn't have enough RAM to hold them. According to its Hugging Face repository, the model carries roughly 35 billion total parameters but activates only about 3 billion per token, the sparse pattern that gives MoE models their efficiency edge.

The twist here is deployment strategy. Edge0 says the model is tuned for SSD-offloaded inference, meaning expert weights can live on fast storage rather than being fully resident in memory. To make that practical, the team trained a routing predictor — a mechanism that anticipates which experts a token will need — so the system can prefetch weights from disk before they're required, reducing the stalls that normally make offloading painful.

Why it matters

MoE architectures already reduce compute per token, but their full weight footprint still demands substantial memory. Offloading to SSD is a known workaround, and the bottleneck is latency: fetching the wrong experts on demand can grind throughput to a crawl. Training the model to help predict its own routing is a pragmatic attempt to close that gap.

• Roughly 35B total parameters with about 3B active per token
• Trained routing prediction to enable SSD prefetching
• Released under the permissive Apache-2.0 license

As a preview, this is an early look rather than a finished release, and Edge0 has not published context length or detailed benchmarks. The permissive licensing and the focus on constrained hardware make it worth watching for anyone trying to squeeze capable models onto laptops, workstations, or edge boxes without generous RAM budgets.