Falcon-Emirati-7B adapts a model to a single dialect

The Technology Innovation Institute released Falcon-Emirati-7B, a dialect specialized model aimed at understanding and generating Emirati Arabic the way a native speaker uses it. The framing in the post is that Arabic is really a family of languages under one name: Modern Standard Arabic is what you read in a textbook, but it is rarely how anyone actually talks. Day to day conversation, humour, negotiation and storytelling in the UAE run on Emirati Arabic, with its own vocabulary and rhythm, and meaning carried in nabati poetry and proverbs does not survive a literal word for word reading. A model that only knows MSA can translate every Emirati sentence correctly and still miss what it means.

It did not start from scratch. Falcon-Emirati-7B is built on Falcon-H1-Arabic, which uses the Falcon-H1 hybrid architecture: state space models (Mamba) and Transformer attention running in parallel inside every block, with their outputs fused before each block’s projection. The stated benefit is the linear time efficiency of Mamba on long sequences combined with the precision of attention for long range dependencies, which matters for Arabic as a morphologically rich language. That family spans 3B, 7B and 34B scales with context windows up to 128K and 256K tokens, and was already trained on a broad mix of MSA and dialectal Arabic across Gulf, Levantine, Egyptian and Maghrebi varieties.

The data recipe mixes three sources: authentic Emirati dialect web data, MSA material about Emirati culture and identity, and synthetic data guided by glossaries and style rules. Evaluation is split two ways, manual evaluation by native speakers plus automatic evaluation on Alyah, and then extended beyond multiple choice with LLM as judge scoring and pairwise comparison broken out category by category. The post is direct about responsible AI and limitations alongside the results, and Falcon-Emirati-7B is available to try now.

(Original source: Hugging Face)