SDMatch
Sketched Distribution Matching as a small, direct loss for making one distribution look like another.
SDMatch is a new loss I have been trying to make useful. The immediate motivation is to replace generative objectives that only implicitly try to match distributions with an explicit batch-level distribution matching objective.
Can a loss compare two distributions without a discriminator? GO TO
What changes when the target is arbitrary? GO TO
Where does it work, and where does it break? GO TO
If it works, the use cases are broader than just generation. You could use it to fix arbitrary model layers to a chosen distribution. In my self-teaching autoencoder work I used crop-resize transformations to keep image reconstructions honest; in hindsight, SDMatch is exactly the sort of tool I would have liked to try there. In the long run, if the rough edges are solved, it might even become a simple alternative training signal for generative models that currently need diffusion or adversarial training.
Starting from SIGReg
The starting point is SIGReg, which I described in the previous research note on practical leJEPA failures. SIGReg samples random directions, projects embeddings onto those 1D axes, and checks whether each projected marginal looks like a standard Gaussian.
SIGReg asks one fixed question: does this batch look Gaussian after I sketch it down to many 1D views?
For each projection, SIGReg is comparing the empirical characteristic function of the batch to the characteristic function of a Gaussian. That gives a compact, differentiable statistic for whether the projected distribution has the right shape.
Visual intuition
Sample directions, then push a collapsed cloud toward Gaussian marginals
The demo shows the geometry rather than the exact optimiser. Each step samples a new projection and nudges the collapsed cloud so that the values seen along that direction look more Gaussian. A single projection only fixes one shadow; many shadows reshape the whole cloud.
Sketched Distribution Matching
SDMatch changes one thing: instead of comparing a batch to a fixed Gaussian, compare one empirical distribution to another. The target can be real images, activations from a reference model, latents from a frozen encoder, or any other sample distribution you can put into a tensor.
Visual intuition
Match many 1D sketches, recover the 2D distribution
The demo shows the core idea rather than the exact optimiser. Each step samples a projection, compares the 1D sketches of the real and generated points, then nudges the generated cloud toward the real sketch. One projection is weak; many projections start to pin down the full distribution.
The random-projection version has no learnable parameters. Draw projections, compare the distributions of fake and real samples across a small grid of frequencies, average the discrepancy, and backpropagate. There is no discriminator, no classifier, and no learned critic.
Learned projections
Random projections are clean, but they get weak as the dimensionality of the object gets harder. A random view of a high-dimensional image often misses the structure you actually care about. You can brute-force that with more projections, but the compute gets ugly quickly.
The obvious next move is to learn the projections. That gives the loss a way to spend its limited sketch budget on directions where fake and real samples still differ.
This starts to look more adversarial, but it is still a small constrained object. The projection matrix is the only learned part, and the comparison statistic stays fixed. My current expectation is that random SDMatch is useful for simple low-dimensional structure, while learned SDMatch is the more plausible path for images or internal model layers.
Experiments so far
These are early experiments, but they are useful because the failure modes are visible. The loss can match simple distributions cleanly, starts to struggle as structure becomes more image-like, and then needs learned projections to keep up.
2D double spiral
Simple 2D images with random projections
Simple 2D images with learned projections
CIFAR directly in pixel space
For the CIFAR run, I matched distributions at three scales: whole images, 16x16 patches, and 4x4 patches. The aim was to give the loss access to both global structure and local texture. That helped the images look more natural, but it did not solve the semantic problem. Pixel-space SDMatch sees statistics. It does not automatically discover the object-level abstractions that make CIFAR CIFAR.
That may be the wrong space to ask the question in. SDMatch might work better in a learned semantic space than directly in pixel space: match distributions over features that already know about objects, while still keeping the loss explicit and batch-level. Even if that is not enough to train a full generator by itself, it may still be useful as an auxiliary pressure that helps stabilise GAN training.
Where this leaves things
SDMatch is attractive because it is direct. It does not ask a discriminator to learn what realism means or diffuse over many steps from noise to distribution. It asks whether fake and real samples have the same sketched distribution under many projections. That directness is also the limitation: if the sketches do not expose the important structure, the loss can be satisfied while the thing you wanted remains unresolved.
My current read is that random SDMatch is a clean tool for low-dimensional matching and layer-level regularisation, while learned SDMatch is the version worth testing for serious image work. The open question is whether the learned projections can be made strong enough to find semantic mismatches without becoming just another unstable adversary.
Acknowledgements
Thanks to Ludo (@ludocomito) for discussions and for trying to make this work in higher dimensions.