TII releases 15 tiny Falcon language models
Specialised Falcon-H1-Tiny models redefine capabilities at small scale
#UAE #LLMs - Technology Innovation Institute (TII), the applied research arm of Abu Dhabi's Advanced Technology Research Council (ATRC) has released Falcon-H1-Tiny, a series of 15 extremely small yet powerful open-source language models. The research redefines capabilities at small scale, covering popular use cases including general chatbot assistance, multilingual applications, coding, function calling and state-of-the-art reasoning models. The release includes a family of 90 million parameter models for English and 100 million parameter models for multilingual applications, each trained separately on specific domains. There is also a state-of-the-art 600 million parameter reasoning model pretrained directly on long reasoning traces.
SO WHAT? - TII’s research paves the way for a future that might rely on a multitude of tiny specialised models, rather than bigger and more generalist models for multiple viable scenarios. The Falcon-H1-Tiny project explores whether “anti-curriculum” strategy (pretraining directly on instruction, chat or reasoning data from scratch) leads to stronger specialised models at extremely small scale. The institute has shared all model artifacts via the Hugging Face AI community to allow researchers and model builders to develop new use cases, enhance existing specialised models or explore research ideas.
Here are some key points about the release of this new research release:
Abu Dhabi-based Technology Innovation Institute (TII) has released Falcon-H1-Tiny, a series of 15 extremely small yet powerful open-source language models covering general chatbot assistance, multilingual applications, coding, function-calling and state-of-the-art reasoning capabilities.
The release includes a family of 90 million parameter models for English and 100 million parameter models for multilingual applications, each trained separately on specific domains using novel optimisation algorithms and training data strategies crucial to obtaining state-of-the-art results.
A state-of-the-art 600 million parameter reasoning model Falcon-H1-Tiny-R-0.6B was pretrained directly on long reasoning traces, outperforming larger reasoning model variants at its size through specialised domain training from scratch.
General usage English models include Falcon-H1-Tiny-90M-Base trained on English-heavy data mixture, Falcon-H1-Tiny-90M-Instruct-Curriculum with supervised fine-tuning and direct preference optimisation, and variants serving both base and instruction-following capabilities.
With Falcon-H1-Tiny-R (0.6B and 0.09B), the research team explored how compact architectures behave when trained exclusively on reasoning data. By prioritising data efficiency over scale, the two models show strong reasoning behavior and achieve competitive results on AIME24, AIME25, LiveCodeBench, and Math500.
Multilingual models include Falcon-H1-Tiny-Multilingual-100M-Base pretrained on a mix of multilingual and high-quality English data, and an instruction-tuned variant applying direct preference optimisation for improved multilingual performance.
Specialised models include Falcon-H1-Tiny-Coder-90M trained on code data for code generation and fill-in-the-middle tasks, and Falcon-H1-Tiny-Tool-Calling trained on calling data for daily function-calling tasks requiring tool-use capabilities.
The release includes concrete application of a novel model optimisation paradigm combining Learnable Multipliers with Muon optimiser, alongside key insights into pretraining data strategies for building more capable language models targeted at specific domains.
The Falcon-H1-Tiny project explores “anti-curriculum” approaches where small models benefit from pretraining directly on instruction, chat or reasoning data to build target capabilities earlier and more deeply, rather than classical pretrain-then-finetune pipelines suited for larger models.
All Falcon-H1-Tiny models are available for download on Hugging Face, released under the TII Falcon License to encourage responsible and ethical AI development whilst enabling community experimentation with extremely small-scale language models. The training approach and data strategy for these models has been made available via a lengthy Falcon-H1-Tiny technical report.
Full list of open-source Falcon-H1-Tiny models
The artifacts that are being open-sourced in this Falcon-H1-Tiny series release are the following:
General usage English models - 90M parameters:
Falcon-H1-Tiny-90M-Base: A base model trained on an English-heavy data mixture, similar to the Falcon-H1 pretraining setup.
Falcon-H1-Tiny-90M-Instruct-Curriculum: A supervised fine-tuned (SFT) model initialised from the English base checkpoint. A lightweight DPO stage is applied on top.
Falcon-H1-Tiny-90M-Instruct: A model pretrained from scratch using SFT data, followed by a lightweight DPO stage. This checkpointe can serve both as a base and an SFT model.
Falcon-H1-Tiny-90M-Instruct-Curriculum-pre-DPO: As per the name, this model corresponds to Falcon-H1-Tiny-90M-Instruct-Curriculum before the DPO stage.
Falcon-H1-Tiny-90M-Instruct-pre-DPO: same as above.
General usage multilingual models - 100M parameters:
Falcon-H1-Tiny-Multilingual-100M-Base: a 100M language model pretrained on a mix of multilingual and high quality English data
Falcon-H1-Tiny-Multilingual-100M-Instruct: Falcon-H1-Tiny-Multilingual obtained after applying a DPO stage on top of the sft checkpoint.
Small Reasoning models - 600M and 90M parameters:
Falcon-H1-Tiny-R-0.6B: state of the art 600M language model pretrained directly on Reasoning data, while doing a GRPO stage on top of it.
Falcon-H1-Tiny-R-0.6B-pre-GRPO: Falcon-H1-Tiny-R checkpoint before the GRPO RL stage.
Falcon-H1-Tiny-R-90M: A 90M language models pretrained on the same data mixture as Falcon-H1-Tiny-R.
Small Specialised models - 90M parameters:
Falcon-H1-Tiny-Coder-90M: a powerful 90M language model trained on code data, which performs code generation and Fill in the Middle (FIM) tasks.
Falcon-H1-Tiny-Tool-Calling: a powerful 90M language model trained on calling data for your daily function-calling tasks.
ZOOM OUT - Technology Innovation Institute first announced the Falcon-H1 family in May 2025. The H1 models were designed to solve one of the biggest challenges in AI by delivering speed and performance without bloated infrastructure through a unique hybrid architecture combining Transformer and Mamba architectures. The initial Falcon-H1 family included six models ranging from 0.5 billion to 34 billion parameters, each outperforming models twice their size, with the small 0.5 billion parameter model delivering performance close to typical 7 billion parameter models on edge devices. A few weeks ago TII announced Falcon-H1R 7B, an open-source AI model with 7 billion parameters that delivers advanced reasoning capabilities, plus a new Falcon-H1-Arabic family.
[Written and edited with the assistance of AI]
LINKS
Falcon-H1-Tiny Series (Hugging Face)
Falcon H1 Tiny Series Technical Report (Hugging Face)
Read more about Falon-H1 family:
TII releases compact reasoning model Falcon-H1R (Middle East AI News)
TII announces new Arabic AI model family (Middle East AI News)
Falcon-H1 LLM joins NVIDIA’s inference microservice (Middle East AI News)
Falcon 3 Arabic LLM + Falcon-H1 model family (Middle East AI News)