MLflow is an open-source platform for managing the machine learning lifecycle: it records experiments, packages trained models in a standard format, keeps a versioned registry of those models and, in recent releases, traces and evaluates generative AI applications. Among AI frameworks & tools in artificial intelligence, it does not train models itself; it sits beside training libraries such as PyTorch, scikit-learn or XGBoost and captures what they produce.
Databricks released MLflow in 2018, and the project later moved to the Linux Foundation. It is written mainly in Python, offers REST, Java and R interfaces, and can run on a laptop, on a self-hosted server or as a managed service. Its central promise is reproducibility: any model in production can be traced back to the code version, parameters, data references and metrics that created it.
How MLflow organizes the machine learning lifecycle
MLflow organizes work into loosely coupled components that can be adopted one at a time. A team may start with experiment tracking alone and add the registry or serving later, because each component communicates through the same tracking server and storage layer.
The table below lists the main components and the problem each one solves.
| Component | What it stores or does | Typical user |
|---|---|---|
| MLflow Tracking | Runs with parameters, metrics, tags and artifacts | Data scientist |
| MLflow Models | A standard model directory with an MLmodel descriptor | ML engineer |
| Model Registry | Named models, numbered versions, aliases and tags | ML platform team |
| MLflow Projects | Reproducible runs defined by an MLproject file | Researcher |
| Evaluation | Metrics and judges applied to models or prompts | Quality reviewer |
| Tracing | Spans of LLM calls, tools and retrieval steps | GenAI developer |
Behind every component sit two stores. The backend store, usually a SQL database such as PostgreSQL, MySQL or SQLite, holds metadata like run identifiers and metric values. The artifact store, usually object storage such as Amazon S3, Azure Blob Storage or Google Cloud Storage, holds large files such as model weights, plots and datasets.
Experiment tracking with runs and autologging
Experiment tracking in MLflow records each training attempt as a run inside a named experiment. Code calls functions such as mlflow.log_param, mlflow.log_metric and mlflow.log_artifact, and the tracking UI then lets users sort, filter and compare runs side by side.
Autologging removes most of that manual code. A single call to mlflow.autolog() patches supported libraries, including scikit-learn, XGBoost, LightGBM, Keras and PyTorch Lightning, so that hyperparameters, training curves and the fitted model are logged automatically. Autologging is convenient, but it can log more artifacts than needed, which inflates storage costs on large hyperparameter sweeps.
The following table shows what a single run typically captures and why each item matters.
| Recorded item | Example | Why it matters |
|---|---|---|
| Parameters | learning_rate = 0.001 | Explain why results differ |
| Metrics | validation AUC per epoch | Rank and compare runs |
| Artifacts | Model files, confusion matrix | Reuse outputs without retraining |
| Tags | git commit hash, owner | Search and audit runs |
| Dataset inputs | Dataset name, digest, source | Detect training data changes |
The MLflow model format and the Model Registry
An MLflow model is a directory with an MLmodel file that lists one or more flavors, meaning the ways the model can be loaded. A PyTorch model may carry a native pytorch flavor and the generic python_function (pyfunc) flavor, which exposes a uniform predict() method to any downstream tool.
Each saved model can include a model signature that defines input and output schemas, an input example, and an environment specification such as a requirements.txt or conda file. These details let a serving system reject malformed requests and rebuild the exact dependency set used during training.
The Model Registry adds governance on top of the format. A registered model has numbered versions, and teams point aliases such as champion or challenger at specific versions. Older releases used fixed stages (Staging, Production, Archived); these stages are now deprecated in favor of aliases and tags, which support more flexible promotion workflows.
MLflow for generative AI and LLM applications
Recent MLflow releases, especially the MLflow 3 line, extend the platform from classic models to LLM applications and agents. The emphasis shifts from logging weights to recording prompts, traces and quality judgments, because many generative systems call hosted models rather than train their own.
- Tracing records each step of a request as spans, with inputs, outputs and latency, and is compatible with OpenTelemetry conventions.
- Automatic tracing integrations cover libraries such as LangChain, LlamaIndex, DSPy and several model provider SDKs.
- A prompt registry stores versioned prompt templates that applications can load by name and alias.
- Evaluation supports LLM-as-a-judge scorers for criteria such as relevance, groundedness and safety, alongside human feedback.
These features place MLflow in direct overlap with LangSmith and Langfuse for LLM observability, although the depth of each integration differs and changes quickly between releases.
How MLflow compares with related tools
MLflow is a lifecycle and governance tool, so its closest neighbors are tracking and observability products rather than training frameworks. The comparison below summarizes the main differences.
| Tool | Main focus | Hosting model | Model registry |
|---|---|---|---|
| MLflow | Full lifecycle, classic ML and GenAI | Self-hosted or managed | Yes, built in |
| Weights & Biases | Experiment tracking and visualization | Mainly hosted service | Yes, artifact-based |
| TensorBoard | Training curves and graphs | Local viewer | No |
| LangSmith | LLM tracing and evaluation | Mainly hosted service | Prompt hub only |
| Langfuse | Open-source LLM observability | Self-hosted or hosted | Prompt management only |
The decisive advantages of MLflow are its permissive Apache 2.0 license, its vendor-neutral model format and its breadth. The usual counterarguments are a less polished visualization layer than Weights & Biases and the operational work required to run it securely at scale.
Deployment choices, limitations and common mistakes
Running MLflow well depends mostly on storage and access control decisions. The open-source server historically shipped with minimal authentication, so production installations usually add a reverse proxy, single sign-on or the optional basic authentication module, or they use a managed offering.
The table below lists frequent problems, their causes and practical fixes.
| Symptom | Likely cause | Fix |
|---|---|---|
| Runs vanish after a restart | Local file store on ephemeral disk | Use a SQL backend store |
| Slow tracking UI | Millions of metric rows in SQLite | Move to PostgreSQL or MySQL |
| Model fails in serving | Missing or wrong dependency pins | Log a pinned environment |
| Bad input reaches the model | No model signature logged | Log a signature and input example |
| Exploding storage bills | Autologging every sweep artifact | Disable unneeded artifact logging |
A frequent misconception is that MLflow is an orchestration or training system. It does not schedule jobs or allocate GPUs; teams pair it with tools such as Airflow, Kubeflow or Ray for that work. Built-in serving through mlflow models serve is suitable for testing and moderate traffic, while high-throughput inference usually moves to a dedicated serving platform that loads the MLflow model.
In plain words
MLflow is like a lab notebook and a warehouse for machine learning. Every time a team trains a model, it writes down the settings it used, the scores it achieved and the files it produced, so nobody has to remember them later.
When a model is good enough, MLflow stores it on a labeled shelf with a version number. Anyone can then see which version is in use, how it was made and how to load it again. For chatbots and other AI assistants, it also records each step the assistant took to answer a question.
A real-world example
A data science team at a mid-sized insurer builds a model that predicts which claims need manual review. Over three weeks, four people run about 300 experiments with gradient boosting, varying tree depth, learning rate and feature sets. With autologging enabled, every run lands in one shared MLflow experiment backed by PostgreSQL and object storage.
The team filters runs by validation AUC, picks the best one at 0.87, and registers it as version 12 of the claims-triage model with the alias challenger. After a two-week shadow test against the current champion, an engineer moves the champion alias to version 12. The serving job loads the model by alias, so no code change is needed.
Six months later, an auditor asks how the model was built. The team opens version 12 and shows the exact parameters, data digest and code commit, answering the question in minutes instead of days.