# Yandex releases a sparse T5 model with 0.6B active params

> The AliceAI-T5-35B-A0.6B is an encoder-decoder mixture-of-experts model that keeps only a sliver of its 35B parameters active per token.

Published by The Open Weights on Sep 10, 2026. Canonical: https://theopenweights.com/news/aliceai-t5-35b-a0-6b-sfeo

## Key facts

- Company: Yandex
- Model: AliceAI-T5-35B-A0.6B
- Version: T5-35B-A0.6B
- Category: Text / LLM
- Modalities: Text / LLM
- License: Other (Open weights, commercial use allowed)
- Parameters: 35B (mixture of experts)
- Active parameters: 600M
- Context window: 131K tokens
- Architecture: AliceAIT5MoEForConditionalGeneration
- Precision: BF16
- Weights size: 69.1 GB
- Significance: notable
- Published: 2026-09-10
- Last verified: 2026-09-18
- Hugging Face: https://huggingface.co/yandex/AliceAI-T5-35B-A0.6B
- Canonical URL: https://theopenweights.com/news/aliceai-t5-35b-a0-6b-sfeo

Yandex has published AliceAI-T5-35B-A0.6B on Hugging Face, a text model that revisits a design most of the field has drifted away from: the encoder-decoder, or T5-style, architecture. What makes this release notable is the combination — it pairs that architecture with a mixture-of-experts (MoE) layout, totaling 35 billion parameters but activating only about 0.6 billion for any given token ([model card](https://huggingface.co/yandex/AliceAI-T5-35B-A0.6B)).

The model is trained with a UL2 objective, a unified pretraining recipe that mixes different span-corruption and language-modeling tasks so a single encoder-decoder can handle both understanding and generation. That framing sets it apart from the decoder-only transformers that dominate today's open-weight releases.

## Why it matters

The headline figure is the activation ratio. An extremely sparse MoE that fires roughly 0.6B of its 35B parameters per token is a bet on efficiency:

- Total capacity of a 35B model, with the per-token compute closer to a sub-billion-parameter model.
- An encoder-decoder backbone that can be well-suited to structured tasks like summarization, translation, and retrieval-style workloads.
- A UL2 training mix aimed at flexibility across generation and comprehension.

A few practical details are not spelled out in the record, including the context length and the specific license terms, so teams evaluating the model for production should check the repository directly. Still, seeing a major lab lean into a sparse encoder-decoder is a useful reminder that the design space for open models is wider than the decoder-only mainstream suggests.

## Get the model

- [Hugging Face](https://huggingface.co/yandex/AliceAI-T5-35B-A0.6B)

## Sources

- [yandex/AliceAI-T5-35B-A0.6B](https://huggingface.co/yandex/AliceAI-T5-35B-A0.6B) — Hugging Face, Sep 10, 2026

---
Source: The Open Weights (https://theopenweights.com/). Aggregated and written by Claude, curated by humans. Cite as: "Yandex releases a sparse T5 model with 0.6B active params", The Open Weights, Sep 10, 2026, https://theopenweights.com/news/aliceai-t5-35b-a0-6b-sfeo