Base Model (Foundation Model)
A base model (Foundation Model) is a general-purpose AI model pre-trained on large-scale datasets. Rather than being specialized for a specific task, it functions as a "foundation" that can be adapted to a wide range of applications through fine-tuning or prompt engineering.
Models as a "Foundation"
The term "Foundation Model," coined by Stanford University in 2021, draws an analogy to the foundation of a building. The idea is that a single foundation can support the construction of diverse applications — chatbots, code generation, translation, summarization, and more.
Major LLMs such as GPT, Claude, Llama, and Gemini are all base models, pre-trained on trillions of tokens of text data. This pre-training instills the "groundwork" of language structure, world knowledge, and reasoning capabilities.
Methods of Customization
There are multiple ways to adapt a base model for specific tasks.
The most accessible is prompt engineering, which refines instruction text without modifying the model itself. Next is fine-tuning, which adjusts the model's weights using task-specific data. LoRA and QLoRA are techniques that significantly reduce the cost of this fine-tuning process.
For deeper adaptation, Continued Pre-training can be employed to incorporate domain-specific knowledge into the model. This approach is sometimes used in fields with extensive specialized terminology, such as medicine and law.
Open-Weight vs. Proprietary
Base models fall broadly into two categories. "Open-weight models," such as Meta's Llama and Mistral, make their model weights publicly available, while proprietary models — such as OpenAI's GPT and Anthropic's Claude — are accessible only via API.
When fine-tuning or running a local LLM in-house, open-weight models are a prerequisite. When API access is sufficient, proprietary models may offer lower operational costs.
Articles covering this term
- Introduction to Fine-Tuning — The Fundamentals and Decision Criteria B2B Companies Should Know Before Building Their Own LLMLearn how fine-tuning works from the basics. Covers PEFT/LoRA, when to use RAG, cost estimates, and a checklist for B2B companies evaluating custom model development.
- What is Harness AI? How It Integrates AI into DevOps and Step-by-Step Implementation GuideHarness AI is a suite of AI agents by DevOps platform Harness. Learn agent roles, supported models, pre-deployment checks, PoC steps, and HITL design in one practical guide.
- What is Model Merging? A Technique to Combine Multiple LLMs Without Training to Improve PerformanceExplore Model Merging: mathematically combine fine-tuned models to integrate multiple skills while reducing computational costs.
- Local LLM / SLM Deployment Comparison — AI Utilization Without Cloud API DependencyCompare open-weight models like GPT OSS, Phi-4, and Llama 4 Scout against cloud APIs on GPU requirements, task accuracy, and TCO. A local AI deployment guide for data sovereignty and cost optimization.
Related Terms
Context Window
A context window refers to the maximum number of tokens an LLM can process at one time, indicating t
Fine-tuning
Fine-tuning refers to the process of providing additional training data to a pre-trained machine lea
Generative AI (Generative AI)
Generative AI is a collective term for AI models capable of autonomously generating content such as
LoRA
LoRA (Low-Rank Adaptation) is a technique that inserts low-rank delta matrices into the weight matri