test: add opt-in shifted FP32 Jet CV control

docs/validation/jet_circle_cv.md

@@ -113,3 +113,47 @@ These criteria are fixed before the continuation runs. There is no new physical
 Actual earlier scale-1 pilot cost was 52.3884s launch-to-exit, 46.1414s after READY for800 steps with full-field diagnostics every base step. A naive same-diagnostic3200-step scale-1 extrapolation is about210s; the new876 field samples reduce transfer count but add ray diagnostics and compression. Use conservative planning envelopes210–500s for scale1 and900–3200s for scale2, not timing guarantees. The scale2 conservative envelope accounts for4x cells and4x root steps; sparse sampling mitigates but does not eliminate that cost. Post-READY safety caps are1200s and5400s respectively, replacing the unsuitable fixed1800s assumption. Root owns compilation/startup timeout and serial launch decisions.
 
 Device-envelope estimates remain128/512MiB for s1/s2, at least1GiB free (or twice estimate). Output estimate is under64MiB per run including five primitive snapshots and876 sample records; no all-time populations are saved. Plan/metadata and sampled files are rewritten at each100-reference-time window to retain bounded partial evidence if interrupted. READY still follows compiled warmup and state/time reset, before measured stepping begins.
+
+## Dense-root temporal diagnostic: preparation only
+
+The audited continuation did not resolve time quadrature: all48 audited stride2 RMS changes exceeded their stride1 CV residuals, by factors1.55–11.46. That historical failure and its gates remain unchanged. The following is a measurement-only experiment to test this explanation, not a solver fix, a physics pass, or permission to extend GPU work. The separate periodic global stock-minus-raw mass residual also persists; denser contour quadrature cannot by itself resolve it.
+
+Future launch, only after the computer is online and root approves a serialized bounded run:
+
+```sh
+CUDA_VISIBLE_DEVICES="$VERIFIED_IDLE_GPU" python tests/validation/run_jet_circle_cv.py --repo "$PWD" --output "$NEW_DENSE_S1" --dense-root --steps 800 --scale 1
+CUDA_VISIBLE_DEVICES="$VERIFIED_IDLE_GPU" python tests/validation/run_jet_circle_cv.py --repo "$PWD" --output "$NEW_DENSE_S2" --dense-root --steps 800 --scale 2
+python tests/validation/analyze_jet_cv_continuation.py "$NEW_DENSE_S1" "$NEW_DENSE_S2" --output "$DENSE_COMPARISON_JSON"
+```
+
+This mode fixes Re100,beta.1 and reference t=0..800 (t*≈.9965), with800/3200 root intervals and801/3201 endpoint records. It is mutually exclusive with --continuation and rejects longer runs or other scales/Re. Physical geometry, source manifest, solver, velocity commands, kernel and raw observation path are unchanged. There are no new finest-hop hooks. Existing GPU population downloads occur every root interval for CPU primitive diagnostics; those distributions are discarded rather than retained.
+
+Dense scalar records contain CV endpoint inventories, each flux and split momentum terms, density/speed, physical composite mass, exact cumulative raw partitions and cumulative absolute aperture net mass. Rays remain on the exact old reference cadence:0..100 at dt1, then104..800 at dt4,276 samples for each scale. Schema2 maps their smaller arrays through ray_sample_indices into the dense records. Full primitive snapshots occur only at reference0,100,800. Saved data is capped at32MiB per case; dense arrays are scalar diagnostics, not all-time full fields. Schema/unit metadata, file hashes, complete-schedule checks and matched root time intervals remain mandatory.
+
+The analyzer derives root strides1,2,4 and the exact old cadence from the SAME dense primitive records, always keeping window endpoints and the SAME raw ledger. At scale2 these root strides correspond to reference dt.25,.5,1; the old cadence corresponds to root gaps4/16. Its fourth quadrature result and change from dense are separately identified. Existing residual normalizations and historical files are preserved. No automatic refinement-progress flags are issued for this temporal-only mode. Root-sampled agreement cannot certify finer-than-root time convergence or exclude aliasing; an unresolved outcome stays unresolved, with no automatic longer run.
+
+Three distinct throughput quantities must remain separate:
+
+- Prescribed unsigned nominal mass at rho_ref=1:2*Q_command*window_duration.
+- Observed raw absolute NET aperture mass: sum over root intervals of abs(dm_blowing)+abs(dm_suction), after spatial reduction within each aperture. This retains reversals between root intervals that abs(window-net) loses. Within-aperture or within-root/substep reversals remain unresolved by this ledger. The legacy raw_gross_mass_over_nominal field retains its old abs(window-net)/ONE-aperture-nominal definition; it is not this new quantity.
+- Sparse reconstructed absolute Q/M: true-arc quadrature of abs(u.n) and abs(rho*u.n), using each existing linear/quadratic ray estimate, followed by old-cadence time integration. This is estimator-dependent measured-field evidence, not exact wall truth, not dense-root quadrature, and not a prescribed nominal flow.
+
+Global stock-minus-raw mass balance is reported per root interval and summed per window, in reference mass units. Raw total-minus-wall/aperture partition discrepancies are also separate. Neither is bundled into the CV temporal error, and neither is called a proven raw-extractor defect. Source/geometry/state validity, both signed and gross budgets, old failures, and physics_validation_pass=false remain intact.
+
+Planning envelope from the earlier52-second800-step pilot: s1 60–240s, s2 600–2400s, with conservative post-READY caps1200/5400s. Scale2 has4x cells and4x root observations over the same reference horizon; sparse saving reduces output, not all CPU/PCIe observation cost. These are preparation estimates, not measured timings. Root owns startup/compilation timeout and GPU serialization. No GPU/remote action was taken to prepare this option.
+
+Budget enforcement preflights the projected retained file sizes before replacing each snapshot/window report or the sample+metadata pair. If32MiB would be exceeded, staged replacements are removed, the previous verified sample/hash-metadata pair remains intact, and execution stops. Temporary compression files need additional short-lived disk space and are distinct from the32MiB retained-evidence cap. This protects budget failures; it is not a filesystem crash-recovery transaction. Both raw partition identities, total−wall−aperture and aperture−blowing−suction, are saved separately from stock/raw and CV residuals.
+
+## Optional shifted-FP32 representation control
+
+The dense-root campaign accepts `--shifted-fp32` as an explicit numerical-representation control. Without it, the historical unshifted FP32 default remains unchanged. Use a fresh output directory and append the flag to the reviewed dense command, for example:
+
+```bash
+CUDA_VISIBLE_DEVICES="$VERIFIED_IDLE_GPU" python tests/validation/run_jet_circle_cv.py --repo "$PWD" --output "$NEW_SHIFTED_DENSE_S1" --dense-root --steps 800 --scale 1 --shifted-fp32
+```
+
+Root owns the future GPU launch and scale selection. This option requires the bounded dense campaign; it does not authorize another run. Geometry, Jet profile/amplitude, initial physical seed, viscosity, time horizon, root sampling, sparse rays/fields, normalization, thresholds, budgets and prior failed gates are unchanged. The `case.ddf_shifting` flag and `case.storage_variant` receipt appear in the plan, READY, window report and sample metadata. Preserve the unshifted evidence in its original directory. Compare like-scale records at identical root endpoints, including CV mass/momentum, global stock-minus-raw balance, raw partitions and measured aperture flux. A reduced mass residual would support representation sensitivity; it would not establish physical convergence or prove that every residual is roundoff.
+
+Both production blocked steppers already support this representation: initialization takes physical populations and encodes `g=f-w` to FP32; public download returns physical `f` decoded in FP64. The runner deliberately retains its historical FP32-rounded physical equilibrium seed for BOTH variants rather than also changing initial precision. Encoding introduces a small additional rounding difference: maximum per-population change 1.035e-10 and density change 3.201e-10 at both scales for this seed. Thus the decoded starting states are not bit-identical. The residual initial density offset is therefore not removed. All existing measurement routines receive physical populations and apply no extra weight shift. Snapshots keep the existing primitive-field precision.
+
+The existing `BlockCurvedBoundary` optional-input adapter invokes the same shifted-aware BFL helper as the AMR boundary: wall density restores the unit background, raw mass uses reflected-minus-outgoing (opposite weights cancel), and reaction restores twice the direction weight. No host correction is applied to raw rows. The shifted branch selects `compute_feq_from_momentum` rather than unshifted `compute_feq`, so this control changes existing numerical algebra as well as storage representation. It cannot isolate storage precision alone or identify collision as the sole cause. Block collision preserves the density perturbation in its shifted branch; blocked halo copies and AMR transfers already support shifted storage. This option changes no solver/kernel/API or source manifest. Existing shifted-storage/Jet host tests and the new runner codec tests are CPU evidence only; numerical GPU behavior remains to be checked by the reviewed control.

tests/unit/test_jet_circle_cv.py

@@ -28,6 +28,41 @@ def eq(rho, u, v):
 
 
 class Host(unittest.TestCase):
+    def test_shifted_storage_receipt_preserves_physical_case(self):
+        for scale in (1, 2):
+            base = case(scale)
+            shifted = case(scale, ddf_shifting=True)
+            self.assertFalse(base.pop("ddf_shifting"))
+            self.assertTrue(shifted.pop("ddf_shifting"))
+            self.assertEqual(base.pop("storage_variant")["name"], "unshifted_fp32")
+            self.assertEqual(shifted.pop("storage_variant")["name"], "shifted_fp32")
+            self.assertEqual(base, shifted)
+
+    def test_shifted_physical_download_roundtrip(self):
+        from CelerisLab.lbm.population_storage import _encode_physical, _decode_stored
+        from run_jet_circle_cv import moments
+        from jet_cv_measurements import primitives
+
+        for speed in (0.03, 0.015):
+            seed = state((9, 48, 64), speed)
+            self.assertEqual(seed.dtype, np.float32)
+            plain = _decode_stored(_encode_physical(seed, False), False)
+            decoded = _decode_stored(_encode_physical(seed, True), True)
+            self.assertEqual(decoded.dtype, np.float64)
+            np.testing.assert_array_equal(plain, seed)
+            # Same FP32 physical seed; only encoding quantization may differ.
+            np.testing.assert_allclose(decoded, seed, rtol=0, atol=3e-10)
+            rho, momentum, velocity = moments(decoded)
+            rho2, velocity2 = primitives(decoded, CX, CY)
+            np.testing.assert_array_equal(rho, rho2)
+            np.testing.assert_array_equal(velocity, velocity2)
+            np.testing.assert_allclose(momentum, rho * velocity, atol=1e-17)
+            fluid = np.ones(seed.shape[1:], bool)
+            inv, flux = cv(decoded, fluid, (0, 0), (5, 4, 25, 20), 0.0072)
+            expected, _ = cv(seed, fluid, (0, 0), (5, 4, 25, 20), 0.0072)
+            np.testing.assert_allclose(inv, expected, rtol=0, atol=1e-6)
+            np.testing.assert_allclose(flux, 0, atol=1e-14)
+
     def test_descriptor_and_uniform_closed_flux(self):
         from CelerisLab.lbm.descriptors import D2Q9_EX, D2Q9_EY
 

tests/unit/test_jet_cv_measurements.py

@@ -31,6 +31,111 @@ from analyze_jet_cv_continuation import integrate, analyze
 
 
 class Measurements(unittest.TestCase):
+    def test_budget_failure_preserves_verified_pair(self):
+        from jet_cv_measurements import install_evidence
+
+        with tempfile.TemporaryDirectory() as tmp:
+            path = Path(tmp)
+            data = path / "samples.npz"
+            meta = path / "samples.json"
+            data.write_bytes(b"old-data")
+            meta.write_text("old-meta")
+            staged = path / ".samples.pending.npz"
+            staged_meta = path / ".meta.pending"
+            staged.write_bytes(b"x" * 100)
+            staged_meta.write_text("new-meta")
+            with self.assertRaisesRegex(RuntimeError, "last verified files preserved"):
+                install_evidence({data: staged, meta: staged_meta}, limit=20)
+            self.assertEqual(data.read_bytes(), b"old-data")
+            self.assertEqual(meta.read_text(), "old-meta")
+            self.assertEqual(
+                sorted(p.name for p in path.iterdir()), ["samples.json", "samples.npz"]
+            )
+
+    def test_dense_root_and_exact_old_cadence(self):
+        from jet_cv_measurements import old_sample_steps
+
+        for scale, count in ((1, 801), (2, 3201)):
+            c = case(scale, steps=800)
+            c["dense_root"] = True
+            dense = sample_steps(c)
+            old = old_sample_steps(c)
+            self.assertEqual(len(dense), count)
+            self.assertEqual(len(old), 276)
+            np.testing.assert_array_equal(dense, np.arange(800 * scale**2 + 1))
+            self.assertTrue(np.isin(old, dense).all())
+            np.testing.assert_array_equal(old[:101] / scale**2, np.arange(101))
+            np.testing.assert_array_equal(old[101:] / scale**2, np.arange(104, 801, 4))
+
+    def test_root_interval_absolute_net_mass_reversals(self):
+        from jet_cv_measurements import absolute_aperture_interval_mass
+
+        masks = (np.array([True, False]), np.array([False, True]))
+        rows = np.zeros((2, 4))
+        rows[0, 3] = 1
+        first = absolute_aperture_interval_mass(rows, *masks)
+        rows[0, 3] = -1
+        self.assertEqual(first + absolute_aperture_interval_mass(rows, *masks), 2)
+        self.assertEqual(abs(1 - 1), 0)
+
+    def test_dense_offline_oscillation_preserves_ledger(self):
+        import json
+        from jet_cv_measurements import old_sample_steps
+
+        for s in (1, 2):
+            c = case(s, steps=800)
+            c["dense_root"] = True
+            steps = sample_steps(c)
+            t = steps / s**2
+            n = len(t)
+            ridx = old_sample_steps(c)
+            # Nonzero steady reference fluxes cancel exact stock changes.
+            inv = np.zeros((n, 2, 2, 3))
+            inv[..., 0] = (100 - 0.001 * t[:, None, None]) * s**2
+            inv[..., 1] = -t[:, None, None] * 0.002 * s
+            flux = np.zeros_like(inv)
+            flux[..., 0] = 0.001 + 1e-4 * np.cos(np.pi / 2 * t[:, None, None])
+            flux[..., 1] = 0.002 / s
+            raw = np.zeros((n, 2, 5, 4))
+            with tempfile.TemporaryDirectory() as tmp:
+                path = Path(tmp)
+                (path / f"samples_s{s}.json").write_text(
+                    json.dumps(dict(case=c, sample_file="samples.npz"))
+                )
+                np.savez(
+                    path / "samples.npz",
+                    t_reference=t,
+                    cv_inventory=inv,
+                    cv_flux=flux,
+                    raw_cumulative=raw,
+                    ray_sample_indices=ridx,
+                    arc_estimates=np.zeros((len(ridx), 2, 2, 2, 2)),
+                    arc_absolute_estimates=np.zeros((len(ridx), 2, 2, 2, 2)),
+                    raw_absolute_aperture_mass_cumulative=np.broadcast_to(
+                        2 * t[:, None] * s**2, (n, 2)
+                    ),
+                    global_mass_reference=np.ones((n, 2)) * 100,
+                    level_stats=np.broadcast_to([1, 1, 0, 0.03, 100], (n, 3, 5)),
+                    velocity_error=np.zeros(n),
+                )
+                result = analyze(path)
+                w = result["windows"][1]
+                self.assertTrue(result["complete_schedule"])
+                np.testing.assert_allclose(
+                    np.array(w["normalized_cv_residual"])[0], 0, atol=1e-11
+                )
+                self.assertGreater(
+                    abs(np.array(w["old_cadence_change_from_dense"])[..., 0]).max(),
+                    0.005,
+                )
+                np.testing.assert_allclose(
+                    w["global_stock_raw_balance_sum_reference"], 0
+                )
+                np.testing.assert_allclose(
+                    w["raw_absolute_aperture_net_interval_mass_reference"], 200
+                )
+                np.testing.assert_allclose(w["raw_reaction_coefficient"], 0)
+
     def test_common_physical_schedule(self):
         a = sample_steps(case(1, steps=3200))
         b = sample_steps(case(2, steps=3200))
@@ -134,6 +239,57 @@ class Measurements(unittest.TestCase):
                 snap["uniform_velocity_reference"][0][fluid], 0.01, rtol=1e-7
             )
 
+    def test_dense_recorder_retains_sparse_rays(self):
+        h = 0.5
+        y, x = np.mgrid[:64, :64]
+        x = (x + 0.5) * h
+        y = (y + 0.5) * h
+        fluid = (x - 16) ** 2 + (y - 16) ** 2 > 25
+        fine_leaf = fluid.copy()
+        fine_leaf[:, :4] = False
+        root_leaf = np.zeros((32, 32), bool)
+        root_leaf[:, :2] = True
+
+        def eq(ny, nx):
+            cu = CX[:, None, None] * 0.01
+            return np.broadcast_to(
+                W[:, None, None] * (1 + 3 * cu + 4.5 * cu * cu - 1.5 * 0.01**2),
+                (9, ny, nx),
+            ).copy()
+
+        ug = eq(64, 64)
+        levels = [eq(32, 32), ug.copy()]
+        c = dict(
+            scale=1,
+            dense_root=True,
+            steps=800,
+            center=[16, 16],
+            diameter=10,
+            u=0.01,
+            nu=0.0072,
+            contours=[[9, 9, 23, 23], [7, 7, 25, 25]],
+        )
+        data = {
+            name: [cv(ug, mask, (0, 0), r, c["nu"]) for r in c["contours"]]
+            for name, mask in [("uniform_adapter", fluid), ("amr_api", fine_leaf)]
+        }
+        with tempfile.TemporaryDirectory() as tmp:
+            rec = MeasurementRecorder(
+                c, tmp, fluid, [root_leaf, fine_leaf], (0, 0), CX, CY
+            )
+            for step in (0, 101):
+                rec.capture(step, ug, levels, data, {k: np.zeros(2) for k in data}, 0.0)
+                rec.records[-1]["raw_cumulative"] = np.zeros((2, 5, 4))
+                rec.records[-1]["raw_absolute_aperture_mass_cumulative"] = np.zeros(2)
+            rec.flush()
+            a = np.load(Path(tmp) / "samples_s1.npz")
+            np.testing.assert_array_equal(a["step"], [0, 101])
+            np.testing.assert_array_equal(a["ray_sample_indices"], [0])
+            self.assertEqual(a["arc_estimates"].shape, (1, 2, 2, 2, 2))
+            self.assertTrue(
+                np.all(a["arc_absolute_estimates"] >= abs(a["arc_estimates"]) - 1e-14)
+            )
+
     def test_offline_mass_force_normalization(self):
         import json
 

tests/validation/analyze_jet_cv_continuation.py

@@ -32,7 +32,7 @@ def analyze(directory):
     t = a["t_reference"]
     if not len(t) or t[0] != 0 or np.any(np.diff(t) <= 0):
         raise ValueError("Sample reference times must increase strictly from zero")
-    from jet_cv_measurements import sample_steps
+    from jet_cv_measurements import sample_steps, old_sample_steps
 
     complete_schedule = np.array_equal(t, sample_steps(c) / s**2)
     records = []
@@ -45,7 +45,15 @@ def analyze(directory):
     )  # angular impulse reference = root/s^2 (not used in force tests)
     q0 = 2 * np.pi / 12 * (c["diameter"] / 2) * c["jet_speed"]
     f0 = 0.5 * (c["u"] * s) ** 2 * (c["diameter"] / s)
-    for lo in range(0, 3200, 100):
+    dense_root = c.get("dense_root", False)
+    old_times = old_sample_steps(c) / s**2
+    ray_indices = (
+        a["ray_sample_indices"] if "ray_sample_indices" in a else np.arange(len(t))
+    )
+    ray_times = t[ray_indices]
+    if dense_root and complete_schedule and not np.array_equal(ray_times, old_times):
+        raise ValueError("Sparse ray records do not match the declared old cadence")
+    for lo in range(0, c.get("steps", 3200 * s * s) // (s * s), 100):
         hi = lo + 100
         ids = np.flatnonzero((t >= lo) & (t <= hi))
         if len(ids) < 2 or t[ids[0]] != lo or t[ids[-1]] != hi:
@@ -54,8 +62,14 @@ def analyze(directory):
         delta = inv[last] - inv[first]
         ledger = raw[last] - raw[first]
         cv_variants = []
-        for stride in (1, 2, 4):
-            selected = np.unique(np.r_[ids[::stride], ids[-1]])
+        selections = [np.unique(np.r_[ids[::stride], ids[-1]]) for stride in (1, 2, 4)]
+        if dense_root:
+            selections.append(ids[np.isin(t[ids], old_times)])
+        for selected in selections:
+            if t[selected[0]] != lo or t[selected[-1]] != hi:
+                raise ValueError(
+                    "Every quadrature selection must retain both window endpoints"
+                )
             total = integrate(t[selected], flux[selected])
             m = delta[:, :, 0] + total[:, :, 0]
             j = -delta[:, :, 1:] - total[:, :, 1:]
@@ -68,16 +82,61 @@ def analyze(directory):
                     axis=-1,
                 )
             )
+        ray_ids = np.flatnonzero((ray_times >= lo) & (ray_times <= hi))
         arc = (
-            integrate(t[ids], a["arc_estimates"][ids]) / 100
+            integrate(ray_times[ray_ids], a["arc_estimates"][ray_ids]) / 100
         )  # Q/M rate conversion is unity
         # Each estimator, aperture, Q/M; compare against signed prescribed Q (rho_ref=1 for M only).
         arc_ratio = arc / np.array([q0, -q0])[None, None, :, None]
         global_delta = (
             a["global_mass_reference"][last] - a["global_mass_reference"][first]
         )
+        dense_details = {}
+        if dense_root:
+            absolute = (
+                a["raw_absolute_aperture_mass_cumulative"][last]
+                - a["raw_absolute_aperture_mass_cumulative"][first]
+            ) / s**2
+            stock_raw = np.diff(a["global_mass_reference"][ids], axis=0) - np.diff(
+                raw[ids, :, 2, 3], axis=0
+            )
+            partition = ledger[:, 2] - ledger[:, 0] - ledger[:, 1]
+            dense_details = dict(
+                quadrature_names=[
+                    "root_stride1",
+                    "root_stride2",
+                    "root_stride4",
+                    "exact_old_cadence",
+                ],
+                root_dt_reference=1 / s**2,
+                old_cadence_change_from_dense=(
+                    cv_variants[3] - cv_variants[0]
+                ).tolist(),
+                root_temporal_sensitivity_max=float(
+                    np.max(abs(np.array(cv_variants)[1:3] - cv_variants[0]))
+                ),
+                raw_absolute_aperture_net_interval_mass_reference=absolute.tolist(),
+                prescribed_unsigned_nominal_mass_reference=float(2 * q0 * 100),
+                legacy_raw_gross_field_definition="raw_gross_mass_over_nominal retains abs(window net blowing)+abs(window net suction), divided by ONE nominal aperture throughput; it is not root-interval absolute throughput.",
+                raw_absolute_to_prescribed_unsigned_nominal=(
+                    absolute / (2 * q0 * 100)
+                ).tolist(),
+                global_stock_raw_balance_max_abs_root_interval_reference=np.max(
+                    abs(stock_raw), axis=0
+                ).tolist(),
+                global_stock_raw_balance_sum_reference=stock_raw.sum(axis=0).tolist(),
+                raw_total_minus_wall_aperture_reference=partition.tolist(),
+                raw_aperture_minus_blowing_suction_reference=(
+                    ledger[:, 1] - ledger[:, 3] - ledger[:, 4]
+                ).tolist(),
+                measured_ray_absolute_volume_mass_reference=integrate(
+                    ray_times[ray_ids], a["arc_absolute_estimates"][ray_ids]
+                ).tolist(),
+                measured_ray_cadence="Sparse old cadence; estimator-dependent, not dense-root or exact wall throughput",
+            )
         records.append(
             dict(
+                **dense_details,
                 t_reference=[lo, hi],
                 startup=hi <= 800,
                 normalized_cv_residual=np.array(cv_variants).tolist(),
@@ -97,6 +156,12 @@ def analyze(directory):
         )
     return dict(
         scale=s,
+        dense_root=dense_root,
+        previous_temporal_failure_retained=(
+            "Prior continuation stride2 changes exceeded baseline residual in all48 audited comparisons; this diagnostic does not retroactively change those gates."
+            if dense_root
+            else None
+        ),
         complete_schedule=bool(complete_schedule),
         sample_checksum_verified=checksum_verified,
         windows=records,
@@ -122,8 +187,10 @@ def main():
         results=results,
         physics_validation_pass=False,
     )
-    if sorted(x["scale"] for x in results) == [1, 2] and all(
-        x["complete_schedule"] for x in results
+    if (
+        not any(x.get("dense_root", False) for x in results)
+        and sorted(x["scale"] for x in results) == [1, 2]
+        and all(x["complete_schedule"] for x in results)
     ):
         ordered = sorted(results, key=lambda x: x["scale"])
         summary = []

tests/validation/jet_cv_measurements.py

@@ -10,13 +10,63 @@ LINEAR = np.array([2.5, -1.5, 0.0])
 QUADRATIC = np.array([4.375, -5.25, 1.875])
 
 
+EVIDENCE_LIMIT = 32 * 1024**2
+
+
+def install_evidence(replacements, limit=None):
+    """Preflight retained bytes before replacing files; cap failure preserves old pair.
+
+    Staged compression bytes are temporary disk overhead, not retained evidence.
+    This guards budget failures, not a transactional crash-recovery filesystem.
+    """
+    temporary = set(replacements.values())
+    try:
+        directory = next(iter(replacements)).parent
+        retained = sum(
+            p.stat().st_size
+            for p in directory.iterdir()
+            if p.is_file() and p not in temporary
+        )
+        projected = retained - sum(p.stat().st_size for p in replacements if p.exists())
+        projected += sum(p.stat().st_size for p in temporary)
+        if limit is not None and projected > limit:
+            raise RuntimeError(
+                "Dense-root 32 MiB retained-evidence budget would be exceeded; last verified files preserved"
+            )
+        for target, staged in replacements.items():
+            staged.replace(target)
+    finally:
+        for p in temporary:
+            if p.exists():
+                p.unlink()
+
+
+def write_evidence_text(path, text, *, dense_root=False):
+    staged = path.with_name("." + path.name + ".pending")
+    staged.write_text(text)
+    install_evidence({path: staged}, EVIDENCE_LIMIT if dense_root else None)
+
+
 def sample_steps(c):
     """Same reference times at both scales; dense startup retained."""
+    if c.get("dense_root", False):
+        return np.arange(c["steps"] + 1, dtype=int)
+    return old_sample_steps(c)
+
+
+def old_sample_steps(c):
+    """Exact previous physical cadence, clipped to this campaign's endpoint."""
     s2 = c["scale"] ** 2
-    reference = sorted(set(range(0, 101)) | set(range(100, 3201, 4)))
+    end = c.get("steps", 3200 * s2) // s2
+    reference = sorted(set(range(0, min(100, end) + 1)) | set(range(100, end + 1, 4)))
     return np.array(reference, int) * s2
 
 
+def absolute_aperture_interval_mass(rows, blowing, suction):
+    """Sum absolute aperture NET masses for one root interval, not per-link flux."""
+    return float(abs(rows[blowing, 3].sum()) + abs(rows[suction, 3].sum()))
+
+
 def factors(scale):
     """Lref=one s=1 root length, Tref=one s=1 root time, rho_ref=1."""
     return dict(
@@ -105,7 +155,7 @@ def ray_geometry(fluid, origin, center, radius, h=0.5):
     )
 
 
-def reconstruct_aperture(rho, vel, geo):
+def reconstruct_aperture(rho, vel, geo, *, return_absolute=False):
     field = np.concatenate((rho[None], vel), axis=0)
     x, y, a, b = geo["ix"], geo["iy"], geo["a"], geo["b"]
     ray = (
@@ -115,12 +165,24 @@ def reconstruct_aperture(rho, vel, geo):
         + field[:, y + 1, x + 1] * a * b
     )
     estimates = []
+    absolute = []
     for coeff in (LINEAR, QUADRATIC):
         wall = np.einsum("cajk,k->caj", ray, coeff)
         un = np.einsum("caj,ajc->aj", wall[1:], geo["normal"])
         q = np.sum(un * geo["ds"], axis=1)
         mass = np.sum(wall[0] * un * geo["ds"], axis=1)
         estimates.append(np.stack((q, mass), axis=-1))
+        absolute.append(
+            np.stack(
+                (
+                    np.sum(abs(un) * geo["ds"], axis=1),
+                    np.sum(abs(wall[0] * un) * geo["ds"], axis=1),
+                ),
+                axis=-1,
+            )
+        )
+    if return_absolute:
+        return ray, np.array(estimates), np.array(absolute)
     return ray, np.array(estimates)
 
 
@@ -136,6 +198,8 @@ class MeasurementRecorder:
         self.cx = cx
         self.cy = cy
         self.records = []
+        self.sparse_rays = []
+        self.ray_steps = set(old_sample_steps(c).tolist())
         self.rays = [
             ray_geometry(fluid_u, (0.0, 0.0), np.array(c["center"]), c["diameter"] / 2),
             ray_geometry(leaves[1], origin_a, np.array(c["center"]), c["diameter"] / 2),
@@ -179,8 +243,16 @@ class MeasurementRecorder:
         record["cv_inventory"] = []
         record["cv_flux"] = []
         record["momentum_flux_terms"] = []
-        record["arc_ray_values"] = []
-        record["arc_estimates"] = []
+        save_ray = not c.get("dense_root", False) or step in self.ray_steps
+        ray_record = dict(
+            sample_index=len(self.records),
+            arc_ray_values=[],
+            arc_estimates=[],
+            arc_absolute_estimates=[],
+        )
+        if not c.get("dense_root", False):
+            record["arc_ray_values"] = []
+            record["arc_estimates"] = []
         record["nearest_mass_flux"] = [
             nearest_flux[k] for k in ("uniform_adapter", "amr_api")
         ]
@@ -203,9 +275,18 @@ class MeasurementRecorder:
                 rtol=2e-10,
             )
             record["momentum_flux_terms"].append(terms)
-            ray, estimates = reconstruct_aperture(rho, vel, self.rays[idx])
-            record["arc_ray_values"].append(ray)
-            record["arc_estimates"].append(estimates)
+            if save_ray:
+                ray, estimates, absolute = reconstruct_aperture(
+                    rho, vel, self.rays[idx], return_absolute=True
+                )
+                ray_record["arc_ray_values"].append(ray)
+                ray_record["arc_estimates"].append(estimates)
+                ray_record["arc_absolute_estimates"].append(absolute)
+                if not c.get("dense_root", False):
+                    record["arc_ray_values"].append(ray)
+                    record["arc_estimates"].append(estimates)
+        if c.get("dense_root", False) and save_ray:
+            self.sparse_rays.append(ray_record)
         self.records.append(record)
         if step // s**2 in (0, 100, 800, 1600, 3200) and step % s**2 == 0:
             fields = {}
@@ -223,7 +304,11 @@ class MeasurementRecorder:
                 fields[name + "_origin_reference"] = np.asarray(origin) / s
                 fields[name + "_h_reference"] = np.array(h / s)
             file = self.output / f"fields_s{s}_tref{step//s**2}.npz"
-            np.savez_compressed(file, **fields)
+            staged = file.with_name("." + file.name + ".pending.npz")
+            np.savez_compressed(staged, **fields)
+            install_evidence(
+                {file: staged}, EVIDENCE_LIMIT if c.get("dense_root", False) else None
+            )
             self.snapshots.append(
                 dict(
                     file=file.name,
@@ -236,19 +321,32 @@ class MeasurementRecorder:
     def flush(self):
         path = self.output / f"samples_s{self.c['scale']}.npz"
         arrays = {k: np.array([r[k] for r in self.records]) for k in self.records[0]}
+        if self.c.get("dense_root", False):
+            arrays["ray_sample_indices"] = np.array(
+                [r["sample_index"] for r in self.sparse_rays], dtype=int
+            )
+            for key in ("arc_ray_values", "arc_estimates", "arc_absolute_estimates"):
+                arrays[key] = np.array([r[key] for r in self.sparse_rays])
         for i, g in enumerate(self.rays):
             for k in ("normal", "ds", "positions"):
                 arrays[f"ray_{i}_{k}"] = g[k]
         arrays["ray_offsets_fine_cells"] = OFFSETS
         arrays["linear_coefficients"] = LINEAR
         arrays["quadratic_coefficients"] = QUADRATIC
-        np.savez_compressed(path, **arrays)
+        staged = path.with_name("." + path.name + ".pending.npz")
+        np.savez_compressed(staged, **arrays)
         metadata = dict(
-            schema=1,
+            schema=2 if self.c.get("dense_root", False) else 1,
             case=self.c,
+            dense_root=bool(self.c.get("dense_root", False)),
+            ray_sample_index_key=(
+                "ray_sample_indices" if self.c.get("dense_root", False) else None
+            ),
+            raw_absolute_throughput_definition="Cumulative sum of abs(blowing NET dm)+abs(suction NET dm) per root interval; unresolved within-aperture/substep reversals; not abs(window net).",
+            absolute_throughput_definition="Quadrature integral of abs(reconstructed u.n), abs(reconstructed rho*u.n); not abs(signed integral), not prescribed nominal, not exact wall truth.",
             reference_conversion=factors(self.c["scale"]),
             sample_file=path.name,
-            sample_sha256=hashlib.sha256(path.read_bytes()).hexdigest(),
+            sample_sha256=hashlib.sha256(staged.read_bytes()).hexdigest(),
             sample_count=len(self.records),
             snapshots=self.snapshots,
             level_stats_columns=[
@@ -280,6 +378,10 @@ class MeasurementRecorder:
             ],
             warning="Extrapolation and nearest-node values are estimator-dependent, not exact wall values. Fields/ray arrays use root-lattice units except explicitly named reference arrays.",
         )
-        (self.output / f"samples_s{self.c['scale']}.json").write_text(
-            json.dumps(metadata, indent=2) + "\n"
+        meta_path = self.output / f"samples_s{self.c['scale']}.json"
+        meta_staged = meta_path.with_name("." + meta_path.name + ".pending")
+        meta_staged.write_text(json.dumps(metadata, indent=2) + "\n")
+        install_evidence(
+            {path: staged, meta_path: meta_staged},
+            EVIDENCE_LIMIT if self.c.get("dense_root", False) else None,
         )

tests/validation/run_jet_circle_cv.py

@@ -18,11 +18,24 @@ CY = np.array([0, 0, 0, 1, -1, 1, -1, -1, 1])
 W = np.array([4 / 9] + [1 / 9] * 4 + [1 / 36] * 4)
 
 
-def case(scale=1, re=100, steps=800):
+def case(scale=1, re=100, steps=800, ddf_shifting=False):
     u = 0.03 / scale
     d = 24.084 * scale
     return dict(
         scale=scale,
+        ddf_shifting=bool(ddf_shifting),
+        storage_variant=dict(
+            name="shifted_fp32" if ddf_shifting else "unshifted_fp32",
+            device_populations="g=f-w" if ddf_shifting else "f",
+            device_dtype="float32",
+            equilibrium_path=(
+                "compute_feq_from_momentum" if ddf_shifting else "compute_feq"
+            ),
+            initial_physical_seed="historical FP32-rounded equilibrium, identical in both variants",
+            downloaded_populations="physical f, decoded by public stepper.download()",
+            downloaded_dtype="float64" if ddf_shifting else "float32",
+            raw_exchange="physical reaction and mass; no host weight correction",
+        ),
         re_d=re,
         beta=0.1,
         nx=192 * scale,
@@ -226,7 +239,7 @@ def run_case(c, ctx, output, provenance, continuation=False):
         collision="SRT",
         streaming="double_buffer",
         store_precision="FP32",
-        ddf_shifting=False,
+        ddf_shifting=c.get("ddf_shifting", False),
         les_enabled=False,
     )
     fine_cfg = LBMConfig(
@@ -237,7 +250,7 @@ def run_case(c, ctx, output, provenance, continuation=False):
         collision="SRT",
         streaming="double_buffer",
         store_precision="FP32",
-        ddf_shifting=False,
+        ddf_shifting=c.get("ddf_shifting", False),
         les_enabled=False,
     )
     obj = SimObject(
@@ -350,6 +363,10 @@ def run_case(c, ctx, output, provenance, continuation=False):
             for name in ("uniform_adapter", "amr_api")
         }
 
+        from jet_cv_measurements import absolute_aperture_interval_mass
+
+        cumulative_absolute_aperture_mass = np.zeros(2)
+
         def capture(step):
             ug = uniform.download()
             levels = amr.download()
@@ -381,6 +398,10 @@ def run_case(c, ctx, output, provenance, continuation=False):
                         for name in ("uniform_adapter", "amr_api")
                     ]
                 )
+                if c.get("dense_root", False):
+                    recorder.records[-1][
+                        "raw_absolute_aperture_mass_cumulative"
+                    ] = cumulative_absolute_aperture_mass.copy()
             return data, error, aperture
 
         previous, _, previous_aperture = capture(0)
@@ -427,7 +448,17 @@ def run_case(c, ctx, output, provenance, continuation=False):
             np.testing.assert_allclose(
                 ar.sum(axis=0)[:2], obs["total"]["impulse"], atol=1e-13, rtol=1e-13
             )
-            for name, rr in [("uniform_adapter", ur), ("amr_api", ar)]:
+            np.testing.assert_allclose(
+                ar[:, 3].sum(), obs["total"]["mass_into_fluid"], atol=1e-13, rtol=1e-13
+            )
+            for backend_index, (name, rr) in enumerate(
+                [("uniform_adapter", ur), ("amr_api", ar)]
+            ):
+                cumulative_absolute_aperture_mass[
+                    backend_index
+                ] += absolute_aperture_interval_mass(
+                    rr, masks["blowing"], masks["suction"]
+                )
                 for k, mask in masks.items():
                     subtotal = rr[mask].sum(axis=0)
                     accumulated[name][k] += subtotal
@@ -519,8 +550,12 @@ def run_case(c, ctx, output, provenance, continuation=False):
                     physics_validation_pass=False,
                     precision="FP32 evolution, FP64 host diagnostics",
                 )
-                (output / f"re{c['re_d']}_s{c['scale']}.json").write_text(
-                    json.dumps(payload, indent=2) + "\n"
+                from jet_cv_measurements import write_evidence_text
+
+                write_evidence_text(
+                    output / f"re{c['re_d']}_s{c['scale']}.json",
+                    json.dumps(payload, indent=2) + "\n",
+                    dense_root=c.get("dense_root", False),
                 )
                 print(
                     json.dumps(
@@ -552,16 +587,33 @@ def main():
         action="store_true",
         help="Fixed measurement campaign: Re100, scale1/2, 3200 reference steps",
     )
+    p.add_argument(
+        "--dense-root",
+        action="store_true",
+        help="Bounded ref0..800 scalar diagnostics at every root interval; sparse rays/fields",
+    )
+    p.add_argument(
+        "--shifted-fp32",
+        action="store_true",
+        help="Opt-in shifted numerical-representation control for --dense-root; historical unshifted default unchanged",
+    )
     p.add_argument("--plan", action="store_true")
     a = p.parse_args()
+    if a.shifted_fp32 and not a.dense_root:
+        p.error("--shifted-fp32 requires the bounded --dense-root campaign")
     if a.steps <= 0 or a.steps > 3200:
         p.error("steps must be 1..3200")
     if a.re == 400 and not a.allow_re400:
         p.error("Re400 requires explicit conditional-campaign opt-in")
+    if a.dense_root:
+        if a.continuation or a.scale not in (1, 2) or a.re != 100 or a.steps != 800:
+            p.error(
+                "--dense-root requires --re 100 --scale 1/2 --steps 800, without --continuation"
+            )
     if a.continuation:
         if a.scale not in (1, 2) or a.re != 100 or a.steps != 3200:
             p.error("--continuation requires --re 100 --scale 1/2 --steps 3200")
-    c = case(a.scale, a.re, a.steps)
+    c = case(a.scale, a.re, a.steps, ddf_shifting=a.shifted_fp32)
     if a.continuation:
         from jet_cv_measurements import sample_steps, factors
 
@@ -575,12 +627,28 @@ def main():
             post_ready_budget_seconds=1200 if a.scale == 1 else 5400,
             runtime_estimate_seconds=[210, 500] if a.scale == 1 else [900, 3200],
         )
+    if a.dense_root:
+        from jet_cv_measurements import sample_steps, old_sample_steps, factors
+
+        c.update(
+            dense_root=True,
+            measurement_schema=2,
+            reference_conversion=factors(a.scale),
+            post_ready_budget_seconds=1200 if a.scale == 1 else 5400,
+            runtime_estimate_seconds=[60, 240] if a.scale == 1 else [600, 2400],
+        )
+        c.update(
+            sample_count=len(sample_steps(c)),
+            sparse_ray_sample_count=len(old_sample_steps(c)),
+            scalar_sample_dt_reference=1 / a.scale**2,
+            raw_interval_dt_reference=1 / a.scale**2,
+        )
     plan = dict(
         case=c,
         required_revision=PIN,
         required_free_device_mib=1024,
         estimated_peak_device_mib=128 * a.scale**2,
-        estimated_output_mib=64 if a.continuation else 2,
+        estimated_output_mib=32 if a.dense_root else (64 if a.continuation else 2),
         max_steps=c["steps"],
         gpu_launch_owner="root; this runner does not select or reserve a GPU",
     )
@@ -612,7 +680,9 @@ def main():
             free, total = cuda.mem_get_info()
         if free < max(1024, 2 * plan["estimated_peak_device_mib"]) * 1024**2:
             raise RuntimeError("Insufficient free device memory")
-        run_case(c, ctx, a.output, provenance, continuation=a.continuation)
+        run_case(
+            c, ctx, a.output, provenance, continuation=a.continuation or a.dense_root
+        )
     print(
         json.dumps(
             dict(status="COMPLETE_DIAGNOSTIC_ONLY", physics_validation_pass=False)