Skip to content

Golden-Image Regression Harness

Black-box regression coverage for the capture/rendering paths that unit tests cannot reach. The harness drives the production capture funnel (RenderingManagerFactory → PrefabPreviewManager → CaptureEngine.CaptureStatic) over a fixed scenario catalog, then compares the output PNGs against committed per-pipeline baselines with a tolerance-based pixel diff.

Owns

  • Deterministic fixture generation (GoldenFixtureFactory, GoldenFixtureMaterials)
  • The scenario catalog and its stable ids (GoldenScenarioCatalog)
  • Capture orchestration + baseline/result management (GoldenHarnessRunner, GoldenPaths)
  • Tolerance-based comparison and diff artifacts (GoldenImageComparer)
  • Menu + Test Runner entry points (GoldenHarnessMenu, GoldenHarnessTests)

Does NOT Own

  • Capture, preview, or pipeline implementation (exercises them as a black box)
  • Animation/batch/game-ready export flows (future harness phases)

Pipeline project setup (do this first)

The harness captures whatever the project actually renders, so a mis-configured project bakes wrong baselines.

  • URP: enable alpha processing. A fresh URP project drops alpha in the post-processing uber pass (it uses render-target formats without alpha), so "transparent" captures come out opaque black even though the objects render correctly. Enable URP's alpha-processing option (and keep an eye on HDR, which interacts with it) before capturing baselines. Symptom in a baseline: corner pixels are (0,0,0,255) instead of (_,_,_,0). Snap Studio Pro does not currently warn about this — see the product follow-up note below.
  • Use a machine/GPU you'll keep; baselines are only valid where they were captured.

Workflow

  1. Open a pipeline test project (BuiltIn / URP / HDRP) with verified-correct rendering.
  2. Tools → Rev Gaming → Golden Harness → Capture Baselines — writes Baselines~/<Pipeline>/*.png. Commit these; they are the reference.
  3. Before merging capture/pipeline/preview changes: Run Comparison (or run GoldenHarnessTests in the Test Runner) in each pipeline project. Failures write Results~/<id>_diff.png (magenta = changed).
  4. Intentional rendering change? Eyeball the results, then recapture baselines and commit them with the change.

A

Step 1 says "verified-correct rendering" and it is load-bearing, not boilerplate. On 2026-08-16 the HDRP particles_straight baseline turned out to have been captured while HDRP Straight-alpha reconstruction was broken — both passes cleared to the same colour, so the reference image was the bug. That scenario passed for as long as the defect lived and went red only when it was fixed. The harness was not failing to catch it; it was certifying it, and it failed the person who repaired it.

So when a baseline changes after a fix, the question is which image is correct, not what did I break — and the answer comes from looking, not from the comparison. Conversely, never recapture to make a red run green: on the same day, a Built-in mismatch that had been red all along turned out to be a real finding hiding inside an already-failing test, and recapturing would have erased the only evidence of it.

  1. Recapturing is all-or-nothing — Capture Baselines re-renders every scenario and overwrites. Compare first, capture, then git checkout the baselines that were already passing, so only genuinely new or intentionally changed images land. Built-in in particular rewrites bytes for scenarios that pass, being within tolerance but not identical; committing those buries a little unexplained drift in every capture commit.

User Fixtures (real assets)

Any prefab under Assets/SnapStudioProGoldenFixtures in the host test project joins the run as scenario user_<prefabName> (512² transparent static capture). Use real store/production assets here — skinned characters, alpha foliage, vehicles, weird materials. They stay in the test project (never in this repo); only the rendered baseline PNGs are committed.

Rules of thumb: - Prefab names must be unique and stable — they become baseline filenames. - Static/skinned meshes are ideal. Avoid prefabs whose appearance depends on runtime scripts or auto-playing VFX; user fixtures are captured at load state. - Particle/VFX coverage comes from the built-in seeded particle scenarios.

Determinism notes

  • Baselines are only valid within the machine/GPU/pipeline that captured them; each test project keeps its own Baselines~/<Pipeline> set.
  • Async shader compilation is disabled during runs; each scenario gets a warm-up render before the measured capture.
  • Built-in particle fixtures use fixed random seeds and ParticleSystem.Simulate.
  • static_supersampled_4x pins the super-sampled export resolve (the premultiplied box downscale). Because that path is gated behind Labs Mode plus a per-feature opt-in, the scenario temporarily writes three EditorPrefs and restores them from its finally — so a run leaves Labs Mode exactly as it found it. If a run is killed mid-scenario, check Preferences before trusting the next manual session.
  • The harness prints through Debug, not the category-gated MyLogger, for the same reason the headless capture API does: a batchmode run has no diagnostics window and the output is the whole point. It previously logged under a "Golden" category that was never registered, so every line came out as [MyLogger] Unknown category 'Golden' used. Ignoring log. and headless runs reported nothing — the NUnit assertion was the only reason results were visible.
  • Tolerances (GoldenHarnessRunner): per-channel delta ≤ 2 ignored; fails when

    0.1% of pixels differ. Loosen only with evidence, never to make a red run green.

Folders ending in ~ are invisible to Unity's importer: Baselines~ is committed, Results~ is gitignored scratch.

Product follow-up surfaced by this harness

Snap Studio Pro has no check or warning for URP alpha processing being disabled. A user who exports transparent sprites from a fresh/default URP project gets silently opaque-black output — a real trap for a tool whose game-ready output depends on alpha. Candidate fix: a startup/compatibility check (alongside the existing pipeline-define detection) that warns when the active URP asset would strip capture alpha. Logged here as a candidate for a later phase, not part of the harness itself.