ERNIE-5.5-Flash-Base

ERNIE-5.5-Flash-Base is Baidu's pre-training checkpoint for the ERNIE 5.5 Flash line, publicly announced on 2026-11-05. It is a sparse Mixture-of-Experts language model with 280B total / ~14B active parameters per token, trained from scratch on 33.5T tokens.

Baidu stresses this is the end-of-pretraining Base — no mid- or post-training — intended as a starting point for domain adaptation, instruction tuning and RL research. Weights ship on Hugging Face as baidu/ERNIE-5.5-Flash-Base (FP8) and baidu/ERNIE-5.5-Flash-Base-BF16, with SGLang and vLLM cookbooks. The full chat/reasoning ERNIE 5.5 Flash is described as coming later.

What problem it solves

Baidu's ERNIE line has been mostly closed API; the community lacked a large, efficient base to fine-tune. ERNIE-5.5-Flash-Base hands developers a high-capacity, sparsely activated MoE they can SFT/RL on their own data without pretraining from scratch.

Architecture highlights (model card)

• 43 layers (1 dense + 42 MoE); hidden size 4096; ~201K vocabulary.
• Hybrid attention: mostly HCA (heavily compressed attention) with some MLA-DSA layers for long context.
• MoE: 512 routed experts (top-10) + 1 shared; ~0.36B ECHO speculative-decoding params excluded from headline counts.
• Context: up to 128K in training; configs enable 1M for fine-tuning/inference.

License and limits

• Custom ERNIE 5.5 Flash Base Model License (not Apache): territorial exclusions (e.g. EU/UK/South Korea); commercial use above ~US$50M related revenue needs separate authorization and “Powered by ERNIE” attribution.
• This is a Base, not a chat model; Baidu's same-size evals should be independently checked.
• Serving needs large GPU pools and a tuned inference stack.

Who it's for

• Research and enterprise teams that want to continue-train a large base on proprietary data;
• Platform teams already running multi-GPU SGLang/vLLM.

Getting started

1. Read the HF model card and Baidu's “Introducing ERNIE 5.5 Flash Base” post.
2. Pick FP8 or BF16 weights for your hardware and serve via SGLang or vLLM completions.
3. Run domain SFT/RL only where the license allows; review commercial clauses and notices first.
4. Wait for the official post-trained Flash if you only need out-of-the-box chat.