1- Notation and Definitions Section v1

Source : Probabilistic Graphical Models: Principles and Techniques

Energy-based model

Def. 1

For a set of variables , we define the energy of a system according to these variables by a function , such that we can define a Boltzmann/Gibbs-like probability distribution over these variables that satisfies :

up to a normalization constant

where could denote a discrete or continuous set of possible states of x. We obtain :

When algorithms of sampling (MCMC etc..) are used to generate such a distribution, it is expected that the ground states (meaning the states with a low value of energy) are sampled more often than the others, given the shape of the distribution.

Def. 1bis — Multilinear expansion, QUBO and HUBO

Let be a real function of binary variables , . Also, for , write for the configuration with if and otherwise; in particular is the all-zero state.

Then admits a unique multilinear expansion — a linear combination of monomials indexed by the subsets :

(the monomials are linearly independent and thus a basis of the functions on ).

Grouping the terms by order and writing the coefficients in the hardware sign convention gives, for maximal order , the HUBO formulation (Higher-order Unconstrained Binary Optimization):

Its truncation at order is the QUBO formulation (Quadratic Unconstrained Binary Optimization):

with biases on the diagonal and couplings off it.

The two conventions are linked by , and in general ; the constant offsets the energy without changing the distribution.

---

Bayesian Networks

Notations

is a random variable that represents a node within a graph for both Bayesian and Markov Networks. Its parents are noted as . A network depicts a set of these random variables that are connected with each other to give a graph structure.

Throughout, abbreviates : capital is the parent variable set, lowercase is a realised configuration of it.

For a variable , denotes the index set of the node together with its parents .

Def. 2 — Factorization of a distribution

Let be a DAG over the variables . We say that a distribution over the same factorization space factorizes according to if can be expressed as a product

This equation is called the chain rule for Bayesian networks. The individual factors Bayesian networks are called conditional probability distributions (CPDs) or local probabilistic models.

It means that we can fully learn the distribution of the system (considered as the set of variables here), just by the knowing the marginals of each node given all its parents).

Def. 3 — Bayesian Network

A Bayesian network is a pair where factorizes over , and where is specified as Bayesian network a set of CPDs associated with ’s nodes (parents of each node). The distribution is often annotated .

Def. 4 — Moral graph

Let be a DAG. The moral graph of is the undirected graph obtained by:

• keeping every edge of and dropping its direction;
• adding an edge between any two parents of a common child (“marrying the parents”).

Every family of is by construction a clique of .

Def. 5 — DIRAC mode

Let be a Bayesian Network. A DIRAC mode is a rule mapping a causal query on to a job, specified by

1. the target distribution it is meant to sample;
2. a transform of — a penalty added on top of it, or a factor swapped inside it — yielding the compiled energy;
3. its realisation on the backend.

A mode is a reference if its target is fixed by causal theory, and a test if its target is defined only as “the physical clamp’s output”. The identity or divergence of a test and its references is an experimental result, never an assumption.

---

Markov Random Fields

Def. 6 — MRFs

Let an undirected graph. Each node carries a random variable . The edges encode direct probabilistic dependence. Critically, absence of an edge means something precise: those two variables are conditionally independent given everything else.

Properties

This is the defining idea, and it comes in three equivalent strengths:

• Local Markov property: a variable is independent of all other variables given its immediate neighbors. That neighbor set is called its Markov blanket — once you condition on it, nothing else in the graph tells you anything more.
• Pairwise Markov property: any two non-adjacent variables are conditionally independent given all the rest.
• Global Markov property: if a set of nodes separates the graph into two pieces and (every path from to passes through ), then and are conditionally independent given

Hammersley-Clifford theorem

For any strictly positive distribution satisfying the Markov property on , the joint distribution must factorize over the graph’s cliques (fully-connected subsets of nodes):

Here is a nonnegative potential function defined on clique , and is the partition function — a normalizing constant summing over every possible configuration:

Because , it’s conventional to rewrite each potential as , giving the Gibbs/Boltzmann form:

This is exactly the energy-based form used in the Ising model — an Ising model is a pairwise MRF, where cliques are just edges (pairwise potentials) and single nodes (local fields).

Technical detail

The distributions mentionned are usually implemented through tables (CPTs), which purpose in BNs and MRFs is the speedup of the calculations through factorization of probability distribution using independency properties between variables.

Conversely, any distribution of this Gibbs form — a product of clique potentials on — satisfies the Markov properties with respect to ; for strictly positive distributions the two characterisations are equivalent.

Def. 7 — Job

A job submitted for sampling is the pair :

• the coefficient set of the energy’s multilinear expansion (Def. 1bis) — diagonal , pairwise links , higher orders (if supported) —
• the inverse-temperature schedule at which it is sampled.

Implementing or updating any query means writing exactly these two objects.

---

Clamping

Let index all variables in the model, and be a full system assignment. That is, contains one value for every variable in the model.

Let be the set of clamped variable indices, and be the target assignment for the clamped variables only. In general, is a partial assignment, not a full state vector.

For example, if:

x = (x1, x2, x3, x4, x5)
S = {2, 5}
a = {a2 = 1, a5 = 0}

then only and are being clamped.

We denote by the clamped block, i.e the one we try to set to , and by the free/non-clamped variables.

The clamp penalty is This means: if every clamped variable matches its target assignment, and if one or more clamped variables disagree with their target values.

The penalty depends on through the entries for . The clamped energy is:

where is the clamp strength. This yields a different distribution given , for which the constant of normalisation is no longer the same. The clamped model is a joint over the full state space:

The object of interest is the free-variable marginal, obtained by summing over the clamped block :

Important clarification:

is not the metric used to validate the conditional distribution. It is an energy penalty function used to construct the clamped model. Validation should compare the resulting clamped distribution against the exact target distribution using KL divergence, total variation distance, query-level error, invalid-state mass, and partition-function checks.

---

Intervention vs Observation

Conceptual understanding of the difference

Take the confounding structure :

graph LR 
	C((C)) --> X((X))
	C((C)) --> Y((Y))
	X((X)) --> Y((Y))

Observing X = x: You’re just looking at a subpopulation where X happens to equal x. But C still varies as it normally would among that subpopulation, and since C also drives Y, some of the association you see between X and Y is really just C leaking through — confounding.

Intervening do(X = x):

graph LR 
	C((C)) --> Y((Y))
	X((X)) --> Y((Y))

You’ve erased the C → X arrow. X no longer depends on C at all — you set it directly. So C can no longer create a spurious channel to Y through X. Any leftover association between X and Y in the post-surgery graph is now genuinely caused by X.

This is the entire reason the field exists: P(Y|X=x) mixes real causal effect with confounding bias. P(Y|do(X=x)) isolates the real causal effect, by construction, because the graph surgery removes the confounding pathway.

Example :

• Observational: “Among people who happen to smoke, what fraction get sick?” → P(Y | X = x)
• Interventional: “If I forced everyone to smoke, what fraction would get sick?” → P(Y | do(X = x))

Impact on the distribution

the operator acts directly on a graph (performing graph mutilation). A hard intervention goes as follow :

• Delete every arrow pointing into X — X’s own structural equation is discarded entirely
• Set X = x as a constant, unconditionally
• Leave every other distribution in the system untouched

if the normal joint distribution was the one defined in the Bayesian Network Section, after , X’s own factor is deleted and the truncated distribution becomes :
with set to x wherever it appears as a parent elsewhere

---

Intervention

Def. 8

Let be a Bayesian Network over , with specified by the CPTs , and . An intervention is a pair where is the set of intervened nodes and, for each , the replacement kernel is a conditional distribution on satisfying local normalisation:
acts on the model, not on a distribution: it returns the Bayesian Network The graph is unchanged and every CPT of is locally normalised, so is a well-defined Bayesian Network. Its chain-rule joint is the post-intervention joint
Special intervention cases:

• Identity: gives and .
• Hard intervention: . That is exactly what the do(X=x) represents. All incoming edges of are deactived and the mechanism is replaced with a parent-freepoint mass . On other cases, interventions are called soft.
• -smoothing: , replacing every mechanism by its -mixture with the uniform distribution on — see Treating zero-probability entries and corollary for detailed explanation.

Notational convention

• A subscript names the operation that produced a derived global object: for the soft-intervened model, in strict parallel with (clamping at strength ) and (smoothing at level , the special case ; formally ).