A Bayesian network is a probabilistic graphical model that represents a set of random variables and their conditional dependencies as a directed acyclic graph (DAG), where each node stores a conditional probability distribution given its parents. In machine learning, it is also called a belief network, and it serves as a compact, interpretable way to reason under uncertainty, to compute the probability of hidden causes from observed evidence and to combine expert knowledge with data.
Unlike most discriminative models in artificial intelligence, a Bayesian network models the full joint distribution of its variables. That property allows the same model to answer many different questions, such as diagnosis, prediction and what-if analysis, without retraining.
How a Bayesian network factorizes a joint distribution
A Bayesian network works by applying the chain rule of probability together with conditional independence assumptions encoded in the graph. The joint probability of all variables equals the product of each variable’s probability given its parents, so a model with dozens of variables needs far fewer parameters than an explicit joint table.
Each node holds a conditional probability table (CPT) for discrete variables or a parametric density, such as a linear Gaussian, for continuous variables. A binary node with three binary parents needs eight parameters, while the full joint table over the same four variables would need fifteen. The savings grow exponentially with the number of variables when the graph is sparse.
The following table lists the core components and what each contributes to the model.
| Component | What it represents | Typical form |
|---|---|---|
| Node | A random variable | Discrete state or continuous value |
| Directed edge | Direct probabilistic influence | Arrow from parent to child |
| Parent set | Variables that condition a node | Usually one to five parents |
| Local distribution | Probability of a node given its parents | CPT, linear Gaussian or logistic |
| Acyclicity constraint | No directed loops in the graph | Ensures a valid factorization |
Conditional independence and d-separation
The graph structure tells the analyst which variables are independent once others are known, and the formal criterion for reading these statements is d-separation. Two nodes are d-separated by an evidence set when every path between them is blocked, which guarantees conditional independence in every distribution that factorizes according to the graph.
Three local patterns determine whether a path is blocked. The collider pattern is the most counterintuitive, because observing a common effect makes two otherwise independent causes dependent, a phenomenon known as explaining away.
| Pattern | Structure | Path open when | Path blocked when |
|---|---|---|---|
| Chain | A to B to C | B is unobserved | B is observed |
| Fork | B causes both A and C | B is unobserved | B is observed |
| Collider | A and C both cause B | B or a descendant is observed | B and its descendants are unobserved |
Each node is independent of its non-descendants given its parents, which is called the local Markov property. The smallest set that shields a node from the rest of the network is its Markov blanket: its parents, its children and the other parents of its children.
Inference: answering queries from evidence
Inference in a Bayesian network means computing the posterior probability of query variables after setting observed variables to known values. Exact inference is NP-hard in general, but it is efficient for graphs with low treewidth, so practitioners choose the algorithm according to the network’s shape and size.
| Method | Type | Best suited to | Main limitation |
|---|---|---|---|
| Variable elimination | Exact | Single queries on small networks | Cost depends on elimination order |
| Junction tree algorithm | Exact | Many queries on moderate treewidth | Clique tables grow exponentially |
| Loopy belief propagation | Approximate | Large sparse networks | May not converge |
| Likelihood weighting | Approximate sampling | Evidence on upstream nodes | Weights degrade with unlikely evidence |
| Gibbs sampling | Approximate MCMC | Complex continuous models | Slow mixing with strong dependencies |
Typical queries include diagnostic reasoning from effects to causes, predictive reasoning from causes to effects and most probable explanation queries that return the single likeliest joint assignment of hidden variables.
Learning parameters and structure from data
Learning a Bayesian network has two parts: parameter learning, which estimates the local distributions for a fixed graph, and structure learning, which searches for the graph itself. With complete data, parameter learning reduces to counting, using maximum likelihood or a Bayesian estimate with a Dirichlet prior that prevents zero probabilities for unseen combinations.
When some values are missing or variables are latent, parameters are usually estimated with the expectation-maximization algorithm. Structure learning is harder, because the number of possible DAGs grows super-exponentially with the number of nodes, so exact search is feasible only for roughly 20 to 30 variables.
| Approach | Example algorithms | How it works | Trade-off |
|---|---|---|---|
| Constraint-based | PC, FCI | Runs conditional independence tests | Sensitive to test errors on small samples |
| Score-based | Hill climbing, tabu search | Optimizes BIC or BDeu score | Can stop at local optima |
| Hybrid | MMHC | Restricts candidates, then scores | More tuning parameters |
| Expert-defined | Manual elicitation | Domain specialists draw the graph | Subjective and labor-intensive |
In practice, many deployed networks combine expert-defined structure with parameters learned from data, because data alone often cannot distinguish graphs that encode the same independencies, known as Markov equivalent structures.
How Bayesian networks relate to other machine learning models
A Bayesian network generalizes several familiar models. The Naive Bayes classifier is a Bayesian network in which the class node is the only parent of every feature, and a Hidden Markov model is a dynamic Bayesian network unrolled over time with one hidden and one observed variable per step.
| Model | Graph type | Key difference |
|---|---|---|
| Bayesian network | Directed acyclic | Arbitrary sparse dependencies |
| Naive Bayes classifier | Directed, star-shaped | Assumes features independent given class |
| Hidden Markov model | Directed, chain over time | Fixed temporal structure |
| Markov random field | Undirected | Models symmetric associations, no direction |
| Logistic regression | No explicit graph | Discriminative, models only the label |
A Bayesian network is not automatically a causal model. Edges express statistical dependence; a causal Bayesian network additionally assumes that each edge reflects a real mechanism, which permits reasoning about interventions with the do-operator.
Limitations and common mistakes
Bayesian networks offer transparency, but they demand careful modeling. The most frequent failures come from unrealistic independence assumptions, oversized parent sets and the confusion of correlation with causation.
- Too many parents make CPTs explode in size and leave most cells estimated from few or no observations.
- Discretizing continuous variables coarsely discards information and makes results depend on arbitrary bin boundaries.
- Reading edge direction as causation without domain justification produces misleading what-if conclusions.
- Ignoring feedback loops is unavoidable in a static DAG; cyclic systems require a dynamic Bayesian network over time slices.
- Overconfident expert probabilities should be checked through sensitivity analysis before the model supports decisions.
Stripped to the essentials
A Bayesian network is a map of cause and effect with probabilities attached. Each circle on the map is something that can be true or false, or take a value, and each arrow shows what directly influences what. Every circle carries a small table that says how likely each outcome is, depending on the circles that point to it.
When new facts arrive, the network updates every other probability in a consistent way. If a patient has a fever and a cough, the map can say how likely flu is compared with a cold. The key benefit is that people can see and question the reasoning, rather than trusting a hidden formula.
Where it is applied
A regional power utility uses a Bayesian network to diagnose transformer faults. Engineers define about 40 nodes, including oil temperature, dissolved gas levels, load history, age and insulation condition, and draw the edges from their maintenance experience. The conditional probability tables are then estimated from twelve years of inspection records, with Dirichlet priors filling gaps for rare fault types.
When a sensor reports a rise in acetylene and hydrogen, a technician enters those readings as evidence. The network returns a posterior of roughly 70 percent for internal arcing, compared with a low prior, and shows that a recent overload partly explains away an overheating hypothesis. The utility schedules a targeted inspection instead of a full shutdown, which cuts unplanned outage time and lets engineers audit exactly why the model reached its conclusion.