SLM (Small Language Model)

"Small" Does Not Mean Weak

In the world of LLMs, "bigger means smarter" has long been conventional wisdom. Compared to GPT-4's estimated 1.8 trillion parameters, SLMs sit at around 1B–10B — a difference of two orders of magnitude. However, since 2025, this conventional wisdom has been rapidly crumbling.

Microsoft's Phi-4 (14B) has achieved scores rivaling GPT-4o on several reasoning benchmarks. Google's Gemma 3, ranging from 1B to 27B, delivers extremely high performance per parameter for its size. Through improvements in model architecture and the curation of high-quality training data, "small but sufficient for specific tasks" has become a reality.

Where Are They Being Used?

SLMs have three primary battlegrounds.

Edge devices: Environments with limited GPU resources, such as smartphones, IoT gateways, and embedded systems. Apple's on-device inference running on iPhones is a prime example of SLMs in action.

Cost optimization: Using GPT-4-class models for routine tasks like classification, summarization, and data extraction is overkill. With SLMs, inference costs can be reduced to less than one-tenth.

Latency requirements: Scenarios demanding responses in tens of milliseconds, such as real-time chat, voice response, and game AI. With fewer parameters, inference speed is faster by orders of magnitude.

How to Use SLMs and LLMs Differently

LLMs still hold the advantage when general-purpose responses are needed — complex reasoning, multilingual support, and long-form generation. On the other hand, when tasks can be narrowed down, fine-tuning an SLM can outperform in terms of accuracy, speed, and cost all at once.

In practice, a standard workflow is emerging: "first prototype with an LLM API, then once the task is well-defined, distill it into an SLM to reduce costs." Distillation refers to the technique of training a smaller model using the outputs of a larger model as teacher data.