3. The Constraint Matrix¶
Chapter 1 separated a law from the operations we may perform on it: sampling, density evaluation, scores, and conditional expectations. Chapter 2 showed that different divergences ask for different combinations of those operations. We can now turn both chapters into a method for choosing generative modeling objectives.
Different stages of generative modeling are not the same optimization problem. They share the same outer form, while the model, target, comparison, and available operations change. The useful question is therefore not merely "which divergence is best?" It is:
Which comparison can be computed from the operations that the model and target actually expose?
1. Three moving pieces inside one mathematical form¶
Let $P_Z$ be an easy source law, let $G_\theta$ be a parameterized generator, and let
$$ Q_\theta=(G_\theta)_\#P_Z \tag{3.1} $$
be the law produced by that generator. Training has the common outer form
$$ \min_\theta D(Q_\theta,P). \tag{3.2} $$
The notation is compact, but it hides three separate choices:
| Piece | Question it answers |
|---|---|
| Model law $Q_\theta$ | What parameterized distribution are we changing? |
| Target law $P$ | Which distribution should the model match? |
| Comparison $D$ | Which disagreements should be expensive, and what computable form is legal under the constraints? |
The first two pieces specify the object being matched. The third specifies both the mathematical comparison and the form through which we can actually optimize it. A density ratio, a variational bound, a learned witness, a kernel average, and a transport plan are all possible computable forms of $D$, each legal under a different constraint pattern.
2. An example: pretrain with forward KL¶
Take ordinary pretraining as an example. The target $P$ is the data law and $Q_\theta$ is the model law. Forward KL is
$$ \mathrm{KL}(P\,\|\,Q_\theta) = \mathbb{E}_{y\sim P}[\log p(y)] - \mathbb{E}_{y\sim P}[\log q_\theta(y)]. \tag{3.3} $$
Evaluating the numerical KL requires both densities. Optimizing it over $\theta$ needs less: the first term is constant in $\theta$, so maximum likelihood needs target samples $y\sim P$ and an evaluable model log-density $\log q_\theta(y)$.
That distinction identifies the actual obstruction:
| Required operation | Data target $P$ | Implicit generator $Q_\theta$ |
|---|---|---|
| Draw samples | available | available |
| Differentiate through generated samples | not needed | available |
| Evaluate normalized log-density | unavailable | generally unavailable |
A dataset supplies the target samples. A neural generator supplies generated samples and gradients through them. But a general pushforward does not supply $\log q_\theta(y)$: reading its density from $G_\theta$ would require a suitably invertible map and a tractable Jacobian determinant.
So the forward-KL goal is not wrong. Its direct maximum-likelihood form is blocked by one missing model-side operation. The methods in the next chapters are different ways to route around that missing cell.
3. The matrix records operations, not names¶
Five operations recur across the generative-model lifecycle:
- Sample -- draw a point from the law.
- Differentiable sampler -- backpropagate through the map that produced that point.
- Normalized density -- evaluate $p(x)$ or $\log p(x)$, including its normalizing constant.
- Unnormalized density -- evaluate $\tilde p(x)\propto p(x)$, sufficient when constants cancel.
- Score or path field -- obtain a local instruction at a supplied point. A score is $\nabla_x\log p(x)$; later path chapters introduce velocity fields. They are different objects, grouped here only because both expose local information without evaluating a normalized density.
The table below marks an operation as available (✅), unavailable (❌), or conditional, approximate, or expensive (⚠️).
| Object in its current role | Sample | Diff. sampler | Norm. density | Unnorm. density | Score / path field |
|---|---|---|---|---|---|
| Gaussian source $P_Z$ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Dataset target $P_{\mathrm{data}}$ | ✅ | ❌ | ❌ | ❌ | ⚠️ after smoothing |
| Tractable density model $Q_\theta$ | ✅ | ⚠️ model-dependent | ✅ | ✅ | ✅ |
| Implicit generator $Q_\theta$ | ✅ | ✅ | ❌ | ❌ | ⚠️ auxiliary field |
| Trained diffusion / flow model | ⚠️ solver required | ⚠️ expensive | ⚠️ expensive | ⚠️ indirect | ✅ at intermediate states |
| Reward-tilted target | ❌ directly | ❌ | ❌ | ⚠️ relative weight | ⚠️ if derivatives exist |
The art of generative modeling is finding a proper solution while walking around those red crosses.
Gaussian noise is the unusually generous row. It can be sampled, reparameterized, scored, and evaluated. That is why it is the universal source. But once it passes through a general generator, most of those operations disappear: sampling and sample differentiability survive, while density evaluation usually does not.
The rows describe roles, not permanent identities. The same trained network may be an implicit generator while it produces samples, a path model while it exposes scores or velocities, a teacher during distillation, and the reference law underneath a reward tilt. Each job exposes a different subset of operations.
4. A mindmap for reading generative modeling methods¶
The constraint matrix provides a mindmap for understanding many generative modeling algorithms. Whenever a method appears, we will place it on the same map by asking three questions.
Question 1: What is being matched?¶
State the model law, the target law, and what must agree: an endpoint distribution, an intermediate marginal, a field, a coupling, or a trajectory. Then choose the divergence whose weighting and geometry match that goal.
Question 2: How is $D$ made computable?¶
List the operations available on both sides. Then derive a computable training signal using only those operations. A discriminator may recover a density ratio; a kernel may replace density evaluation by sample averages; a conditional regression target may expose a score or velocity without evaluating the underlying density.
Question 3: What does that choice cost?¶
Record what the computable form introduced: finite-minibatch error, a learned witness, a variational bound, a transport solver, an auxiliary network, a numerical sampler, or a long trajectory. The workaround's bias, variance, computation, and failure modes enter here.
Each later method can therefore be summarized in four columns:
| Object | $D$ in its computable form | Constraints used | The bill |
|---|---|---|---|
| what must match | what is optimized | why that form is legal | where it can leak |
Keeping these columns separate prevents a common confusion: the loss written in code is not necessarily the population divergence in the mathematical statement. It may be a rewrite, bound, or minibatch approximation chosen because the constraint matrix makes that form available.
There is no universally best choice of $D$. Its computable form must first be legal under the constraints. Among the legal routes, we can then compare statistical accuracy, geometry, optimization, computation, and the artifact each route leaves behind.
Using the same questions at every stage makes unfamiliar methods easier to understand, compare, and remember.
The next chapter opens the pretraining sequence by applying this matrix to a samples-only data target.