Bayesian network

Also known as: belief network

On this page 8

A Bayesian network is a probabilistic graphical model that represents a set of random variables and their conditional dependencies as a directed acyclic graph (DAG), where each node stores a conditional probability distribution given its parents. In machine learning, it is also called a belief network, and it serves as a compact, interpretable way to reason under uncertainty, to compute the probability of hidden causes from observed evidence and to combine expert knowledge with data.

Unlike most discriminative models in artificial intelligence, a Bayesian network models the full joint distribution of its variables. That property allows the same model to answer many different questions, such as diagnosis, prediction and what-if analysis, without retraining.

How a Bayesian network factorizes a joint distribution

A Bayesian network works by applying the chain rule of probability together with conditional independence assumptions encoded in the graph. The joint probability of all variables equals the product of each variable’s probability given its parents, so a model with dozens of variables needs far fewer parameters than an explicit joint table.

Each node holds a conditional probability table (CPT) for discrete variables or a parametric density, such as a linear Gaussian, for continuous variables. A binary node with three binary parents needs eight parameters, while the full joint table over the same four variables would need fifteen. The savings grow exponentially with the number of variables when the graph is sparse.

The following table lists the core components and what each contributes to the model.

ComponentWhat it representsTypical form
NodeA random variableDiscrete state or continuous value
Directed edgeDirect probabilistic influenceArrow from parent to child
Parent setVariables that condition a nodeUsually one to five parents
Local distributionProbability of a node given its parentsCPT, linear Gaussian or logistic
Acyclicity constraintNo directed loops in the graphEnsures a valid factorization
Core components of a Bayesian network

Conditional independence and d-separation

The graph structure tells the analyst which variables are independent once others are known, and the formal criterion for reading these statements is d-separation. Two nodes are d-separated by an evidence set when every path between them is blocked, which guarantees conditional independence in every distribution that factorizes according to the graph.

Three local patterns determine whether a path is blocked. The collider pattern is the most counterintuitive, because observing a common effect makes two otherwise independent causes dependent, a phenomenon known as explaining away.

PatternStructurePath open whenPath blocked when
ChainA to B to CB is unobservedB is observed
ForkB causes both A and CB is unobservedB is observed
ColliderA and C both cause BB or a descendant is observedB and its descendants are unobserved
Path patterns and their effect on information flow

Each node is independent of its non-descendants given its parents, which is called the local Markov property. The smallest set that shields a node from the rest of the network is its Markov blanket: its parents, its children and the other parents of its children.

Inference: answering queries from evidence

Inference in a Bayesian network means computing the posterior probability of query variables after setting observed variables to known values. Exact inference is NP-hard in general, but it is efficient for graphs with low treewidth, so practitioners choose the algorithm according to the network’s shape and size.

MethodTypeBest suited toMain limitation
Variable eliminationExactSingle queries on small networksCost depends on elimination order
Junction tree algorithmExactMany queries on moderate treewidthClique tables grow exponentially
Loopy belief propagationApproximateLarge sparse networksMay not converge
Likelihood weightingApproximate samplingEvidence on upstream nodesWeights degrade with unlikely evidence
Gibbs samplingApproximate MCMCComplex continuous modelsSlow mixing with strong dependencies
Main inference methods compared

Typical queries include diagnostic reasoning from effects to causes, predictive reasoning from causes to effects and most probable explanation queries that return the single likeliest joint assignment of hidden variables.

Learning parameters and structure from data

Learning a Bayesian network has two parts: parameter learning, which estimates the local distributions for a fixed graph, and structure learning, which searches for the graph itself. With complete data, parameter learning reduces to counting, using maximum likelihood or a Bayesian estimate with a Dirichlet prior that prevents zero probabilities for unseen combinations.

When some values are missing or variables are latent, parameters are usually estimated with the expectation-maximization algorithm. Structure learning is harder, because the number of possible DAGs grows super-exponentially with the number of nodes, so exact search is feasible only for roughly 20 to 30 variables.

ApproachExample algorithmsHow it worksTrade-off
Constraint-basedPC, FCIRuns conditional independence testsSensitive to test errors on small samples
Score-basedHill climbing, tabu searchOptimizes BIC or BDeu scoreCan stop at local optima
HybridMMHCRestricts candidates, then scoresMore tuning parameters
Expert-definedManual elicitationDomain specialists draw the graphSubjective and labor-intensive
Families of structure learning algorithms

In practice, many deployed networks combine expert-defined structure with parameters learned from data, because data alone often cannot distinguish graphs that encode the same independencies, known as Markov equivalent structures.

How Bayesian networks relate to other machine learning models

A Bayesian network generalizes several familiar models. The Naive Bayes classifier is a Bayesian network in which the class node is the only parent of every feature, and a Hidden Markov model is a dynamic Bayesian network unrolled over time with one hidden and one observed variable per step.

ModelGraph typeKey difference
Bayesian networkDirected acyclicArbitrary sparse dependencies
Naive Bayes classifierDirected, star-shapedAssumes features independent given class
Hidden Markov modelDirected, chain over timeFixed temporal structure
Markov random fieldUndirectedModels symmetric associations, no direction
Logistic regressionNo explicit graphDiscriminative, models only the label
Bayesian networks and related models

A Bayesian network is not automatically a causal model. Edges express statistical dependence; a causal Bayesian network additionally assumes that each edge reflects a real mechanism, which permits reasoning about interventions with the do-operator.

Limitations and common mistakes

Bayesian networks offer transparency, but they demand careful modeling. The most frequent failures come from unrealistic independence assumptions, oversized parent sets and the confusion of correlation with causation.

  • Too many parents make CPTs explode in size and leave most cells estimated from few or no observations.
  • Discretizing continuous variables coarsely discards information and makes results depend on arbitrary bin boundaries.
  • Reading edge direction as causation without domain justification produces misleading what-if conclusions.
  • Ignoring feedback loops is unavoidable in a static DAG; cyclic systems require a dynamic Bayesian network over time slices.
  • Overconfident expert probabilities should be checked through sensitivity analysis before the model supports decisions.

Stripped to the essentials

A Bayesian network is a map of cause and effect with probabilities attached. Each circle on the map is something that can be true or false, or take a value, and each arrow shows what directly influences what. Every circle carries a small table that says how likely each outcome is, depending on the circles that point to it.

When new facts arrive, the network updates every other probability in a consistent way. If a patient has a fever and a cough, the map can say how likely flu is compared with a cold. The key benefit is that people can see and question the reasoning, rather than trusting a hidden formula.

Where it is applied

A regional power utility uses a Bayesian network to diagnose transformer faults. Engineers define about 40 nodes, including oil temperature, dissolved gas levels, load history, age and insulation condition, and draw the edges from their maintenance experience. The conditional probability tables are then estimated from twelve years of inspection records, with Dirichlet priors filling gaps for rare fault types.

When a sensor reports a rise in acetylene and hydrogen, a technician enters those readings as evidence. The network returns a posterior of roughly 70 percent for internal arcing, compared with a low prior, and shows that a recent overload partly explains away an overheating hypothesis. The utility schedules a targeted inspection instead of a full shutdown, which cuts unplanned outage time and lets engineers audit exactly why the model reached its conclusion.