MLOps
MLOps is a practice that automates and standardizes the entire lifecycle of machine learning model development, training, deployment, and monitoring, enabling the continuous operation of models in production environments.
"Building a Model" and "Operating a Model" Are Different Jobs
Even if you can build a highly accurate model in Jupyter Notebook, keeping it running stably in a production environment requires an entirely different skill set. Updating training data, retraining models, version control, A/B testing, detecting performance degradation——managing all of this manually will break down regardless of team size.
MLOps applies the DevOps philosophy to machine learning, but it comes with unique challenges that differ from software deployment. These include the need to simultaneously version-control three things——code, data, and model weights——the fact that model performance degrades over time due to shifts in data distribution (drift), and the requirement to ensure reproducibility of experiments.
Components of an MLOps Pipeline
Data Pipeline: Automates the collection, preprocessing, and validation of training data. Since data quality directly determines model quality, this is the most critical layer.
Experiment Tracking: Tools like MLflow, Weights & Biases, and Comet are used to record hyperparameters, learning curves, and evaluation metrics, ensuring reproducibility of experiments.
Model Registry: Stores trained models with versioning and manages the promotion flow from staging to production.
Serving: Exposes models as APIs. Inference engines such as vLLM, TensorRT-LLM, and Triton Inference Server are commonly used.
Monitoring: Tracks not only inference latency and error rates, but also data drift (shifts in input data distribution) and model drift (gradual degradation of accuracy over time). It is also common to have a mechanism that automatically triggers retraining when a threshold is exceeded.
MLOps in the Age of LLMs
The rise of LLMs has given birth to a derivative concept called "LLMOps." New operational challenges have emerged that did not exist in traditional MLOps, including prompt version control, evaluation of RAG pipelines, configuring guardrails, and optimizing inference costs. The toolchain has also expanded to include LLM-specific offerings such as LangSmith, Braintrust, and Arize AI.
Articles covering this term
- What is AI Observability? A Practical Guide to Monitoring LLMs in ProductionFrom AI observability basics to tracing, evaluation, and cost management for production LLM apps—learn key implementation steps and tool selection tips.
- What is Edge AI? How On-Device LLMs Work and How to Choose One for BusinessLearn the basics of edge AI & on-device LLMs, and how to design workflows requiring low latency, data privacy, or unstable connectivity—where cloud LLMs fall short.
- What is SLM Distillation? How to Create Small Specialized Models from Large LLMsLearn how SLM distillation uses large LLMs as teachers to build cost-effective, task-specific private models—covering the mechanism and implementation steps.
- What Is an AI-Native Management Strategy? How to Fundamentally Redesign Your Business ModelFrom "adding AI" to "redesigning with AI." Learn AI-native management transition steps, ERP/FMS integration strategies, and real-world examples from Thai and Japanese companies.
Related Terms
No-Code/Low-Code Development
No-code/low-code development is a development approach that minimizes the need for programming exper
E2E Test
E2E testing (End-to-End testing) is a testing methodology that simulates user interactions to drive
Acceptance Testing
Acceptance testing is a testing method that verifies whether developed features meet business requir
SSM (AWS Systems Manager)
AWS Systems Manager (SSM) is an AWS managed service for centrally operating and managing EC2 instanc