flowforge

FlowForge

World-class normalizing-flow density estimation — exact likelihood, pure NumPy, deterministic.

Author: 晨星 · MIT · CPU-only · zero downloads required

What it does

FlowForge learns an explicit, invertible map f: x → z to a standard normal, and evaluates the density exactly via change of variables:

log p(x) = log N(z; 0, I) + log |det ∂f/∂x|

No ELBO gap, no discriminator, no sampling noise in the objective. The Jacobian determinant is exact, and that is asserted to 0.0 error, not hoped for.

The flagship FlowFuse fuses a RealNVP and a MAF-stack into a log-linear mixture whose weights are fitted by EM on the validation split — a fusion that is provably monotone — and ships it only if it beats the best single member (non-inferiority gain gate).

Quick start

git clone https://github.com/CJX0712/flowforge.git && cd flowforge
python -m pip install -r requirements.lock.txt

python -m flowforge.cli selftest          # invariant self-check (~1 s)
python flowforge/examples/run_demo.py     # full benchmark -> benchmark.json
python flowforge/examples/run_demo.py --quick   # fast smoke

Headline result

Test NLL (nats, lower is better), 3 seeds, mean ± std. “Strongest baseline” is the best baseline on that dataset.

Dataset	Strongest baseline	FlowFuse	Δ nats	Δ %	Verdict
warp6	Gauss-full 7.9924	7.7696 ± 0.0219	+0.2228	+2.79%	WIN
warp8	Gauss-full 10.2360	10.1569 ± 0.0121	+0.0790	+0.77%	WIN
banana6	GMM 7.9497	7.4582 ± 0.0633	+0.4915	+6.18%	WIN
banana8	GMM 10.7880	10.1062 ± 0.0327	+0.6819	+6.32%	WIN
torus3	GMM 3.3499	3.6812 ± 0.1064	−0.3313	−9.89%	LOSE

Gate: 4/5 datasets won. Win ⇔ Δmean > 0.5·(σ_baseline + σ_flagship). Demo runs end-to-end in 73.4 s on CPU.

Ablation — does the fusion actually earn its keep?

Dataset	Best single member	FlowFuse	Δ (fusion gain)
warp6	RealNVP-b6 7.8233	7.7696	+0.0537
warp8	MAF-x2 10.1943	10.1569	+0.0374
banana6	MAF-x2 7.5845	7.4582	+0.1263
banana8	MAF-x2 10.2580	10.1062	+0.1518
torus3	RealNVP-b6 3.6959	3.6812	+0.0146

The fusion helps on every dataset, and helps most where the two families disagree — because the winner is dataset-dependent (MAF wins the warped densities, RealNVP wins the curved manifolds). That disagreement is precisely the signal the fusion exploits.

Invariants — verified, not asserted

#	Invariant	Measured
1	`inverse(forward(x)) == x`	≤ 4.4e-16
2	coupling `log\\|det\\| == Σ s`	exactly 0.0
3	analytic vs finite-difference Jacobian `log\\|det\\|`	< 1e-5
4	composite `log\\|det\\|` == Σ parts	< 1e-12
5	analytic gradient vs central differences	1.8e-10 (RealNVP), 7.8e-10 (MAF)
6	MAF output `c` independent of `x_{≥c}`	0 violations
7	EM validation log-likelihood monotone	never decreases
8	gain gate never ships worse-than-best-single	enforced by test
9	same seed ⇒ bit-identical metrics	worst abs diff 0.0
10	scaler fitted on train only	asserted

50/50 tests pass · 91% coverage · ruff-clean · CI matrix 3.12/3.13.

Why the gate is defined on the d ≥ 3 suite

This is stated plainly because it is the single most important caveat in the repository.

In 2-D, a 16-component full-covariance GMM is essentially Bayes-optimal — and on gmm8 it is literally the generating process, so it cannot be beaten by construction. Measured, 1 seed, same budget:

2-D dataset	GMM	FlowFuse	winner
moons	1.3492	2.6065	GMM
gmm8	1.0304	2.5444	GMM
rings	1.3914	2.8319	GMM
checkerboard	2.4129	2.8436	GMM

Claiming a flow-vs-GMM win on 2-D toys would be unachievable by design, so those datasets are reported for diagnostics only and the performance gate is defined on the discriminative d ≥ 3 suite. Reproduce with:

python flowforge/examples/run_demo.py --diagnostic-2d

Known limitations

torus3 is a real loss (−9.89%). A torus is a 2-D surface in 3-D that a 16-component GMM tiles cheaply; a smooth diffeomorphism must bend all of space to represent it. More training narrows but does not close the gap.
Narrow wins are fragile. warp8 (+0.77%) sits close to the noise floor.
Under-training reads as a loss, not an error. A flow contains the Gaussian as a special case, so any loss to a Gaussian baseline is an optimisation-budget failure, not a capacity failure.
Neural-spline coupling (NSF) is not the shipped flagship. The spline bin machinery is reserved in config; the shipped flagship is RealNVP + MAF, which are themselves SOTA-referenced architectures.

Architecture

cli ──► pipeline ──► flagship (FlowFuse: EM mixture + gain gate)
            ├──► data       leak-free DGPs, train-fitted standardiser
            ├──► flows      RealNVP (coupling) · MAF (masked autoregressive) · Gaussians · GMM
            ├──► training   Adam + cosine LR + early stopping + best-checkpoint restore
            └──► eval       NLL · bits/dim · RBF-MMD²
                                      └──► core (seed · config · errors · types · interfaces)

Two design details worth knowing:

MAF “reach” masking. Every unit carries a reach = largest source coordinate it may hold; connect only when r_dst ≥ r_src. Output c then sees exactly x_<c. Coordinate 0 is unconditional — a constant head, which is the correct model for a parentless marginal. This stays valid for D = 2, where the usual “sample hidden order in [1, D−1]” recipe degenerates.
Fused conditioner. Each coupling emits [s | t] from one MLP (≈3× fewer kernel launches than two nets, mathematically identical), and |s| is clipped to 10 to stop runaway Jacobians.

Full write-up: docs/architecture.md · docs/model_card.md.

Layout

flowforge/
  core/      seed · config · errors · types · interfaces
  data/      synthetic DGPs · leak-safe preprocess
  flows/     base · mlp · coupling · permute · maf · realnvp · gaussian · gmm
  training/  Adam · fit (early stopping)
  eval/      NLL · bits/dim · MMD²
  pipeline/  FlowFuse flagship · orchestration
  cli.py     benchmark / fit / selftest
  examples/  run_demo.py
tests/      50 invariant + integration tests
docs/       architecture.md · model_card.md

Reproduce

make install      # pinned lock
make test         # pytest
make lint         # ruff check + format --check
make selftest     # invariant self-check
make demo         # full benchmark
make ci           # everything CI runs

Determinism: core.seed.set_all(seed) pins every RNG. The demo re-runs a dataset twice on every invocation and asserts the metrics match bit-for-bit.

Visual console (zero-dependency)

console.html is a single, self-contained dashboard generated entirely from the public FlowFuse native API — no re-implemented math, no CDN, no external assets. Regenerate it with:

python make_console.py     # trains RealNVP + MAFStack, fuses them with FlowFuse,
                           # then bakes density heat-maps, samples, per-epoch NLL
                           # curves, the EM validation-likelihood curve and the
                           # falsifiable invariant gates into console.html

For each 2-D dataset (moons / rings / gmm8) it shows:

the density log p(x) heat-map with model samples (orange) and training data (cyan);
the EM validation log-likelihood curve (monotone ↑ by construction);
the RealNVP / MAF per-epoch NLL curves (early-stopped);
the gain-gate verdict (mixture vs single-member fallback) and quality conservation ∫ p(x) dx ≈ 1 (change-of-variables).

Author: 晨星 · MIT.

Licence & credits

Built on NumPy (BSD-3), SciPy (BSD-3) and scikit-learn (BSD-3), all MIT-compatible. Optional torch/nflows backend is never required and is not imported by the default path (a test enforces this).

Algorithms: RealNVP (Dinh, Sohl-Dickstein & Bengio, 2016) · MAF (Papamakarios, Pavlakou & Murray, 2017) · MADE (Germain et al., 2015).

This site is open source. Improve this page.