# OID v2 pipeline and recovery contract

This document is the operational specification for the audit-bound OID result. Mathematics and formal claim rules are in [`OID_METHOD.md`](OID_METHOD.md); fold values and interpretation are in [`RESULTS.md`](RESULTS.md). The final authority is `runs/active-two-scene-v2/OID_PHASE4_AUDIT_20260723T1510Z.json`, SHA-256 `9358105c7199793a2fd810885044295350da165a0f8e9026734ca1a51fc52ed8`.

## 1. Research question and scope

OID asks which active correction-field directions linearly generate or reconstruct a declared incremental observable through a fitted propagator. The two frozen cases are `karman_re100` and `illusion_1.0L`; each has the historically declared realizations `r1`, `r2`, and `r3`. This is not evidence of broad stochastic independence. LORO conclusions are conditional on these runs and fixed operating conditions.

For Kármán nomenclature, `re_code=100` uses reference length `2D`: with `D=20`, `U0=0.01`, and `nu=0.004`, `Re_code=U0(2D)/nu=100` while physical cylinder `Re_D=U0D/nu=50`.

## 2. Physical collection contract

### Same-parent branches and numerical twins

Each realization begins from its own declared parent checkpoint. Three branches descend from that same parent with identical exogenous forcing and clocks:

1. `controlled`: PPO action branch (`q_ctl`);
2. `zero_baseline`: zero-action blocked branch (`q_blk`) used in subtraction;
3. `baseline_twin`: an independently executed zero-action branch used only to test numerical reproducibility.

The twin is evidence, not the subtraction baseline. Its field and telemetry relative-RMS values must satisfy the fixed 0.01 thresholds; all six recorded twin gates pass. Parent/checkpoint, branch, field, telemetry, paired-collection, and twin-gate hashes are bound through `runs/ACTIVE_EVIDENCE.json` and canonical metadata.

### Dual clock and exact pairing

The control clock and field clock are distinct. A branch records `control_interval`, `field_interval`, exact `sample_ids`, `lattice_steps`, and stored `times`. In the frozen products, Kármán has 363 field samples at 317-step spacing (control interval 800), while Illusion has 642 field samples at 257-step spacing (control interval 600). Field snapshots and field-level telemetry are captured on the same field lattice. Controlled and baseline arrays pair only when IDs, lattice steps, and times are exactly equal; equal array indices are insufficient. A positive sample lag pairs `x(t)` with an actually present `y(t+lag)` in the same segment and discards the tail—no padding, nearest matching, wraparound, or cross-segment pairing. Circular wraparound is used only inside the explicitly labelled null.

The active correction and observables are

- `dq_ctl = q_ctl - q_blk`;
- `delta_forces = forces_ctl - forces_blk` (six LBM-force channels);
- `delta_sensors = sensors_ctl - sensors_blk` (six velocity channels normalized by `U0`).

Absolute controlled telemetry and target error are different observables and cannot substitute for these deltas.

### GPU policy for collection

Historical CFD collection used conda environment `pycuda_3_10`, physical GPU 2 exposed as logical GPU 0, and the shared CFD lease. Parent and branch commands require an explicit physical `--device 2`; branch output is immutable/no-clobber. Collection is frozen for the current evidence. New collection must create new parents and destinations and must not modify the archived historical payload.

## 3. Canonical product

`rebuild_phase3_canonical.py` consumes the staged `formal-three` payload, verifies the archived mask defect is present, and writes a corrected `oid-canonical-paired/v1` product per scene/realization. It uses:

- full Legacy field axes `(x,y)` and components `ux,uy`;
- crop `x=360:1000`, all `y`, then stride 8;
- reduced shape `80 x 64` and C-order flattening, `ux` then `uy`;
- the actual Legacy solid rasterization and outer solid rows;
- Kármán counts 5,025 fluid/95 solid; Illusion counts 5,024 fluid/96 solid;
- uniform lattice-cell quadrature, sampled-cell area 64, zero weight on solid cells;
- `state_weights` repeated consistently for both velocity components.

Every product contains `dq_ctl`, both telemetry deltas, IDs, lattice steps, times, mask, cell metric, and state weights plus array hashes, source file/content hashes, geometry declaration, delta contract, exact timeline contract, and reproducibility evidence. It refuses an existing destination.

## 4. Formal Phase 3–5 analysis

### Snapshot POD, LR/LE, and metrics

For each outer fold, two realizations train and one is held out. All means, scales, metric snapshot POD, rank/lag selection, propagators, and null/control selection are training-only. Weighted snapshot POD forms the sample-space eigensystem rather than a dense field metric; one maximum-rank fit is cached for each training set and lag, then truncated for candidate ranks. It preserves the full-spectrum energy denominator and zero-weight mask behavior.

The fitted Schlegel map is `C_tau: a(t) -> b(t+tau)`. LR-OID uses the least-residual reconstruction inverse; LE-OID uses the least-state-energy inverse. Generated-range restriction is explicit when the map is rank-deficient. Each fold records right-inverse and projection defects, ranks, thresholds, condition number, retained energy, model arrays, and cross-fold subspace diagnostics. The formal numerical status is `truncated`, with no ridge/loading/shrinkage and one prespecified `rcond=1e-12` for covariance, propagator, generated range, and LR/LE solves.

The observable metric is `H=I` in physical incremental channel coordinates. The primary statistic is pooled H-weighted held-out R². Per-channel R² and the unweighted mean of channel R² are supplementary.

### LORO rank and lag selection

The fixed candidates are state ranks `(4, 8, 12)` and sample lags `(0, 1, 2)`. Selection maximizes the training-only primary score. Reports preserve full rank/lag sensitivity. A sample lag is not a lattice-step count: reports also store lattice-step and physical-time lag using the scene spacing and the convective `D/U0` scale.

### Periodic-excluded circular-shift null

Each outer fold runs 999 circular shifts. For each training realization, a deterministic multichannel rFFT estimates its dominant period after per-channel centering and normalized nonzero-frequency power. With exclusion width `w=max(2, 1+max(candidate lags))=3` samples, admissible shifts exclude the circular-zero neighborhood and neighborhoods of positive integer period multiples. Period unavailable or fewer than two admissible shifts fails closed.

Each shifted surrogate reruns complete rank/lag selection; the held-out realization never determines the shift set or selection. The upper-tail Monte Carlo value is `(1 + count(null >= observed))/1000`, so resolution is 0.001. These are circular time shifts, not Fourier “phase surrogates.” Approximate stationarity/periodicity remains an assumption.

### Reselected realization derangement

The negative control deranges realization-to-observable correspondence in training, then independently reselects rank and lag over the full candidate grid. Reusing the observed model selection is invalid. A job is promoted only if every fold satisfies:

`observed H-weighted R² > reselected deranged-control H-weighted R² + 0.01`.

A failed scientific gate does not invalidate artifact integrity; it yields conditional non-promotion.

## 5. Artifacts and expected statuses

- Canonical: `canonical/<scene>/<r>/paired_product.{json,npz}`, schema `oid-canonical-paired/v1`, audit status `verified`.
- Analysis: `analysis/<scene>/<observable>/phase_3_5_report.json`, `phase_3_5_models.npz`, and `verification.json`; schemas `oid-phase3-5-loro/v2` and `oid-phase3-5-verification/v1`.
- Audit: explicit fresh `oid-phase4-audit/v3` JSON. Expected frozen result: overall `conditional`; canonical gate `pass`; formal integrity `pass`; sensors `promoted`; forces `not_promoted`; formal-complete gate `conditional`/fail-closed.
- Summary: `summary/phase4_summary.json`, schema `oid-phase4-compact-summary/v2`, plus three PNG/PDF pairs. Figures embed audit/report hash bindings; no field-mode plot is authorized.
- Provenance: `runs/ACTIVE_EVIDENCE.json`, schema `oid-active-evidence/v1`; relocation manifests bind archived bytes.
- Conditional freeze: `provenance/freeze/OID_CONDITIONAL_FREEZE_20260723T173300Z.json`, schema `oid-conditional-freeze/v1`. It is an exact evidence-package inventory, not a strict `COMPLETE` marker. It can bind ignored canonical/model NPZ payloads present locally or in the checksum-bound archive while the repository exposes compact JSON/PNG/PDF evidence.

`archive/legacy_oid_v1/`, `archive/v2_development_code/`, `archive/v2_development_runs/`, prior `runs/formal-*`, and relocated source paths are provenance-only. They are not alternative active products, and archived code must not be executed or imported as an active implementation.

## 6. Exact commands

Run from repository root. Analysis, audit, and summary are CPU-only; disable CUDA explicitly. Commands use fresh output paths because all publishing stages are no-clobber.

### Verify archive without restoring

```bash
sha256sum src/OID_analysis/archive/RUNS_RELOCATION_MANIFEST.json
sha256sum /home/frank14f/optane/DynamisLab-backups/OID_analysis-runs-phase0-20260723/RUNS_RELOCATION_MANIFEST.json
cmp src/OID_analysis/archive/RUNS_RELOCATION_MANIFEST.json \
  /home/frank14f/optane/DynamisLab-backups/OID_analysis-runs-phase0-20260723/RUNS_RELOCATION_MANIFEST.json
```

Both hashes must be `97a7611a7788d473081fb0c5498fd868d927f055d70910e818a9007eb5e69e67`; `cmp` must be silent. For the older development archive, verify its `source_manifest.sha256` as documented in `archive/RELOCATION.md` (local manifest SHA-256 `0f4a7c25b0e76ec60a9c2229260917444d2420fccf101b091a5aa0b2f34a32aa`).

### Stage restore

```bash
mkdir -p /tmp/oid-runs-restore
rsync -a --checksum --dry-run \
  /home/frank14f/optane/DynamisLab-backups/OID_analysis-runs-phase0-20260723/payload/src/OID_analysis/runs/ \
  /tmp/oid-runs-restore/
rsync -a --checksum \
  /home/frank14f/optane/DynamisLab-backups/OID_analysis-runs-phase0-20260723/payload/src/OID_analysis/runs/ \
  /tmp/oid-runs-restore/
```

Verify staged repository-relative files against `RUNS_RELOCATION_MANIFEST.json`. Never restore over the compact evidence tree, never use `--delete`, and review every destination conflict.

### Canonical rebuild

```bash
PYTHONPATH="src:$PYTHONPATH" CUDA_VISIBLE_DEVICES="" \
  conda run -n pycuda_3_10 python3 \
  src/OID_analysis/scripts/rebuild_phase3_canonical.py \
  --payload /tmp/oid-runs-restore \
  --output /tmp/oid-active-two-scene-v2/canonical
```

### Formal analysis

```bash
PYTHONPATH="src:$PYTHONPATH" CUDA_VISIBLE_DEVICES="" \
OPENBLAS_NUM_THREADS=2 OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 NUMEXPR_NUM_THREADS=2 \
  conda run -n pycuda_3_10 python3 \
  src/OID_analysis/scripts/run_phase3_5_formal.py \
  --canonical /tmp/oid-active-two-scene-v2/canonical \
  --analysis /tmp/oid-active-two-scene-v2/analysis --all
```

`--all` launches four jobs with at most three processes; each BLAS pool must report two threads. To recover one failed job, remove only its incomplete fresh staging destination after inspection, then rerun with `--scene <scene> --observable <observable>`.

### Audit

The audit expects the canonical active layout and manifest bindings. The module is rooted at `src/OID_analysis/v2`, so `python -m v2.audit` requires `PYTHONPATH=src/OID_analysis` (not `PYTHONPATH=src`). For a reproducibility audit, stage a complete repository-shaped copy and choose a fresh timestamped output:

```bash
PYTHONPATH="src/OID_analysis:$PYTHONPATH" CUDA_VISIBLE_DEVICES="" \
  conda run -n pycuda_3_10 python3 -m v2.audit \
  --repo /path/to/staged/DynamisLab \
  --output /path/to/staged/DynamisLab/src/OID_analysis/runs/active-two-scene-v2/OID_PHASE4_AUDIT_<UTC>.json
```

For read-only verification of the frozen tree, recompute its SHA-256 and compare with the value at the top of this document. Do not overwrite the final audit.

### Compact summary

The script reads the fixed final audit name and four reports, verifies their hashes, and requires a nonexistent output directory:

```bash
PYTHONPATH="src:$PYTHONPATH" CUDA_VISIBLE_DEVICES="" \
  conda run -n pycuda_3_10 python3 \
  src/OID_analysis/scripts/plot_phase4_summary.py \
  --output /tmp/oid-phase4-summary
```

### Conditional freeze generation and test

Build only to a fresh path outside active evidence. Rebuilding does not upgrade the status to `COMPLETE` and will reflect current source/doc bytes; it does not replace the published freeze unless separately reviewed.

```bash
PYTHONPATH="src:$PYTHONPATH" CUDA_VISIBLE_DEVICES="" \
  conda run -n pycuda_3_10 python3 \
  src/OID_analysis/scripts/build_freeze_manifest.py \
  --created-utc <UTC> --output /tmp/OID_CONDITIONAL_FREEZE_<UTC>.json

PYTHONPATH="src:$PYTHONPATH" CUDA_VISIBLE_DEVICES="" \
  conda run -n pycuda_3_10 python3 -m pytest -q \
  src/OID_analysis/tests/test_freeze_manifest.py
```

### Full active tests

```bash
PYTHONPATH="src:$PYTHONPATH" CUDA_VISIBLE_DEVICES="" \
  conda run -n pycuda_3_10 python3 -m pytest -q \
  src/OID_analysis/tests/test_v2.py \
  src/OID_analysis/tests/test_geometry.py \
  src/OID_analysis/tests/test_schlegel_oid.py \
  src/OID_analysis/tests/test_phase_3_5.py \
  src/OID_analysis/tests/test_parent.py \
  src/OID_analysis/tests/test_audit.py
```

## 7. Recovery and fail-closed rules

1. Preserve the compact tree and both relocation manifests before recovery.
2. Verify archive manifests, then restore only into an empty staging directory.
3. Rebuild corrected canonical products into an empty destination; never copy the archived defective canonical mask as the final product.
4. Run formal jobs into an empty analysis root. Existing destinations are evidence or incomplete attempts and are never overwritten.
5. Verify every report/model/verification binding before audit.
6. Audit to a fresh explicit file. `fail` indicates integrity/contract failure; `conditional` may mean truthful scientific non-promotion or the formal-complete freeze.
7. Generate summary to a fresh directory only after audit/report hash verification.
8. If telemetry is control-end-only, zero between control boundaries, timeline-mismatched, nonfinite, or missing six channels, quarantine it; do not repair by interpolation.
9. If a period cannot be estimated, admissible shifts are insufficient, a source hash differs, exact mask counts differ, BLAS/GPU policy differs, or a destination exists, stop and investigate.
10. Never synthesize `COMPLETE.json`. A new formally complete study requires newly initialized parents and an immutable run-level manifest/transition under the strict loader.
