# Observable-inferred decomposition (OID): method contract

This is the single normative method specification for OID in this repository. It separates the mathematical construction of Schlegel et al. (2012) from the present implementation status and from CCD. The trusted scientific scope is the one declared in `README.md`; archived v1 products are provenance only.

## 1. Source and notation

The primary source is Schlegel et al., *On least-order flow representations for aerodynamics and aeroacoustics*, J. Fluid Mech. (2012), already reproduced locally as `papers/Sch12.md`. The derivation below follows its equations (2.8), (2.11)–(2.15), (2.17)–(2.27), and Appendix equations (A1)–(A4), (A9). The state POD may be computed by Sirovich's snapshot method: this is a computational route to the same metric POD subspace, not a different OID objective. Repository notation is:

- state snapshot \(x(t)=\Delta q_{ctl}(t)\), not the paper's observable \(q\);
- state POD coefficients \(a(t)\in\mathbb R^r\);
- observable coefficients \(b(t)\in\mathbb R^m\);
- state and observable metrics \(G\succeq0\) and \(H\succeq0\);
- fitted linear propagator \(C_\tau:a(t)\mapsto b(t+\tau)\).

The renamed state avoids confusing a flow field with an observable. Column-vector formulas are used below; stored sample matrices have samples in rows.

## 2. Correction-field state contract

The default OID state is the active correction

\[
x(t)=\Delta q_{ctl}(t)=q_{ctl}(t)-q_{blk}(t),
\]

where each field contains the same declared components, spatial support, units, geometry, mask/quadrature, and lattice time. A valid time-paired subtraction requires controlled and blocked branches descended from the declared parent state, identical exogenous forcing and sampling protocol, and exactly equal `lattice_steps`; equal array indices alone are insufficient. Flattening order and component order are part of the artifact schema. Means, POD, scaling, propagator, rank choice, and regularization must be fitted on development data only.

A phase-conditioned state answers a different question:

\[
x_\phi=\mathbb E[q_{ctl}\mid\phi_{ctl}]-\mathbb E[q_{blk}\mid\phi_{blk}].
\]

It permits distinct phase sequences but requires a common phase definition, bin edges, support and minimum bin populations. It must not be presented as an instantaneous paired correction. Cropping or subsampling is admissible only when recorded in the run contract; physical inner products must retain the corresponding mask and quadrature weights.

The three-field interpretation remains: \(q_{in}\) is incident flow, \(q_{blk}\) is passive blockage, and \(q_{ctl}\) is controlled flow. Thus \(q_{blk}-q_{in}\) is blockage, whereas \(q_{ctl}-q_{blk}\) is active correction. OID here decomposes the latter unless a different state is explicitly named.

## 3. Incremental observables

The observable must be declared before fitting. For a paired correction analysis, the preferred definitions are

\[
y_F(t)=F_{ctl}(t)-F_{blk}(t),\qquad
 y_s(t)=s_{ctl}(t)-s_{blk}(t),
\]

with force component/object ordering and sensor channel ordering recorded. An illusion/task-error observable may instead be

\[
y_{err}(t)=s_{ctl}(t)-s_{tar}(t)
\]

(or the analogous force error), but its sign, target, normalization and timeline must be explicit. Absolute controlled telemetry \(F_{ctl},s_{ctl}\), incremental telemetry, and target error are different scientific questions and cannot share a label or be substituted after analysis. Force-OID and signature-OID are separate decompositions.

All centering and nondimensionalization parameters are learned on development samples and applied unchanged to validation. Near-constant channels must be removed or assigned an explicit scale policy; silently replacing their scientific meaning through standardization is forbidden.

## 4. Metrics \(G\) and \(H\)

Define the state and observable inner products

\[
\langle a_1,a_2\rangle_G=a_1^TGa_2,
\qquad
\langle b_1,b_2\rangle_H=b_1^THb_2.
\]

\(G\) represents the physical state norm: for velocity fields it is induced by component layout, fluid mask and cell quadrature; in an orthonormal POD basis, \(G=I\). \(H\) defines the observable goal functional and relative channel units: \(Q_H(b)=\mathbb E[b^THb]\). Identity metrics are valid only after justified nondimensionalization. Diagonal inverse-variance weighting is a statistical choice, not a physical law. Both matrices, their coordinate system, and any whitening must be stored; changing either changes the OID problem.

For a metric-aware state POD, modes satisfy \(\Phi^TW\Phi=I\), where \(W\) is the field metric, and \(a=\Phi^TW(x-\bar x)\). Observable POD modes \(v_i\) satisfy

\[
\Sigma_{bb}H v_i=\lambda_i v_i,
\qquad v_i^THv_j=\delta_{ij},
\]

and are ordered by non-increasing \(\lambda_i\). Equivalently, POD may be performed after an explicitly stored square-root whitening. Ties define an invariant subspace, not unique individual modes.

## 5. Derivation from Schlegel et al. (2012)

Schlegel equation (2.12), including equation (2.27) for delay, is the OID assumption

\[
b(t+\tau)=C_\tau a(t)+\varepsilon(t).
\]

It is a propagator or fitted linear observable map. With centered development matrices \(A=[a(t_j)^T]_j\) and \(B_\tau=[b(t_j+\tau)^T]_j\), ordinary LSE gives

\[
\widehat C_\tau=B_\tau^TA(A^TA)^+,
\]

with weighted/regularized variants declared when used. This is Schlegel equation (2.8) in POD coordinates. The induced OID product (equations (2.19)–(2.22)) is

\[
\langle a_1,a_2\rangle_A
   =(C_\tau a_1)^TH(C_\tau a_2),
\qquad
R^{OID}_{jk}=K^{-1}a_j^TC_\tau^THC_\tau a_k.
\]

Its nonzero eigenvalues equal the observable POD eigenvalues under an exact surjective map. Schlegel equations (2.14)–(2.15) then pull each ordered observable POD direction back into state coordinates:

\[
d_i=C_\tau^-v_i,\qquad \psi_i=\Phi d_i.
\]

The inverse is non-unique when \(m<r\). Writing \(P=C_\tau^-C_\tau\), equations (A1)–(A2) require \(P^2=P\) and \(C_\tau P=C_\tau\). The two OID variants select different inverses.

### LR-OID: least residual

LR-OID minimizes expected state reconstruction residual (A3):

\[
P_{LR}=\arg\min_{P=C_\tau^-C_\tau}
  \mathbb E\|a-Pa\|_G^2.
\]

For nonsingular empirical covariances and a full-row-rank exact map, the corresponding regression inverse is

\[
C_{LR}^-=\Sigma_{ab}\Sigma_{bb}^{-1},
\qquad
\Sigma_{ab}=\mathbb E[ab^T],
\quad \Sigma_{bb}=\mathbb E[bb^T].
\]

With pseudoinverses it is the minimum-training-residual LSE reconstruction \(a\approx C_{LR}^-b\). LR modes are \(d_i^{LR}=C_{LR}^-v_i\). This is the EPOD-coincident, observer-oriented construction described by equations (2.24) and (A3): it reconstructs the empirically most probable *linear* state, not a causal state without further assumptions.

### LE-OID: least energy

LE-OID minimizes projected state norm for the same generated observable (A4):

\[
C_{LE}^-b=\arg\min_z z^TGz
\quad\text{subject to}\quad C_\tau z=b.
\]

If \(G\succ0\) and \(C_\tau\) has full row rank,

\[
C_{LE}^-=G^{-1}C_\tau^T
  (C_\tau G^{-1}C_\tau^T)^{-1}.
\]

Thus \(d_i^{LE}=C_{LE}^-v_i\). For \(G=I\), this is the Moore–Penrose minimum-norm inverse. It identifies the least-\(G\)-energy state displacement capable of producing the linearly represented observable; it does not prove that a real actuator can realize that displacement or that reducing modal energy will close the control objective.

The exact pulled-back directions are the columns \(d_i=C_\tau^-v_i\), and they must be retained as such: in generated observable coordinates they obey \(C_\tau d_i=v_i\) (or the corresponding range-restricted identity below). Arbitrary unit normalization or Gram–Schmidt orthogonalization in the state metric generally changes that equation. State norms and overlaps are therefore reported as diagnostics, e.g. \(d_i^TGd_i\) and \(d_i^TGd_j\), without altering the exact pullback. If normalized or orthogonalized vectors are desired for visualization or regression, define and store an explicit invertible coefficient transform \(T\), with \(\widetilde D=DT\) and generated observables \(C_\tau\widetilde D=VT\); these transformed vectors and coefficients are not the original per-direction pullbacks. A common scalar rescaling of \(G\) changes reported energies but not LE directions, whereas changing relative state weights changes the LE problem. Ordering remains the observable POD ordering \(\lambda_1\ge\lambda_2\ge\cdots\), not state energy, inverse gain, or cross-covariance singular values. Sign is arbitrary only when the matching observable direction is sign-flipped too; comparisons use sign-invariant MAC, principal angles, or projector similarity, and near-degenerate modes are compared as subspaces.

## 6. Generated-range coordinates under truncation

Let the requested observable-POD coefficient vector be \(b\in\mathbb R^m\), and let \(C_\tau\in\mathbb R^{m\times r}\) have numerical rank \(k<m\). A truncated construction does not claim to invert all of \(b\). Write \(U_k\in\mathbb R^{m\times k}\) for the retained left-singular basis of \(C_\tau\), and define

\[
z=U_k^Tb,\qquad C_g=U_k^TC_\tau,\qquad C_gd_i=e_i.
\]

Thus \(z\) is a generated-range coordinate, not the original channel vector and, when \(U_k\) rotates the observable POD basis, not an individual observable-POD coefficient. Prediction in the requested observable-POD space is \(\widehat b=U_kC_ga\); the discarded component \((I-U_kU_k^T)b\) is ungenerated truncation error and must not be credited to the inverse. Physical observables are recovered only after lifting \(\widehat b\) through the stored observable-POD basis and undoing its centering/scaling. LR/LE identities, inverse defects, mode labels and goal fractions must state whether they refer to \(z\), \(\widehat b\), or physical channels. The projector \(U_kU_k^T\), its coordinate convention, and the retained/discarded goal fractions are part of the result.

## 7. Propagator is not cross-covariance

The standardized cross-covariance

\[
K_{ab}=K^{-1}A_{std}^TB_{std}
\]

measures contemporaneous or delayed linear association. Its SVD gives PLS-type directions. It is not \(C_\tau\): it omits the state covariance inverse, does not define a state-to-observable map, and supplies neither the LR nor LE pseudoinverse. CCA additionally whitens both covariance matrices and answers a correlation question. Therefore current `v2.analysis.fit_pls_type` products are **PLS-type baselines**, not Schlegel OID. They may coincide with special whitened, zero-lag reductions but that algebraic coincidence does not establish the OID interpretation.

## 8. Delay contract

A nonnegative delay pairs \(x(t)\) only with \(y(t+\tau)\) at an actually present lattice step and within the same segment. The candidate integer used to index retained snapshots is the **sample lag** \(\ell\), matching report fields such as `sample_lag` and `candidate_sample_lags`; it is not itself a lattice-step count unless snapshots were saved every lattice step. For uniform sampling, report `lattice_step_lag = sample_lag * lattice_step_spacing` and `physical_time_lag = sample_lag * physical_time_spacing`, together with both spacings. Missing tails are discarded: no padding, wraparound, nearest-index pairing, or cross-segment pairing. This no-wrap rule governs physical delay pairing; the separately labelled circular time-shift null below deliberately wraps only the null observable. A delay fixed from physics is distinguished from a delay selected on development data. If selected by maximizing resolution or validation score, the search range and selection uncertainty are part of the model; validation data cannot choose \(\tau\). A single fitted delay is a reduced approximation to distributed propagation times, exactly as cautioned in Schlegel §2.6.

## 9. Numerical status labels

Every reported decomposition receives one status:

- **exact**: declared finite-dimensional model, unregularized full-rank solves, complete retained state and observable POD spaces, and numerical rank/conditioning checks pass. “Exact” is algebraic, not physical truth.
- **truncated**: state POD, observable POD, propagator, or inverse uses an explicitly declared rank cutoff/TSVD. Report retained state energy, observable goal fraction, cutoff and rank sensitivity.
- **regularized**: ridge/Tikhonov, covariance loading, shrinkage, or another penalty is used. Report penalty, metric, selection protocol and sensitivity. If truncation is also used, label **truncated + regularized**.
- **PLS-type/CCA diagnostic**: no Schlegel pseudoinverse construction; never relabel as LR-OID or LE-OID.

Use stable SVD/eigendecomposition rather than literal matrix inversion. One relative cutoff must govern every rank decision and pseudoinverse within a fitted decomposition: for a matrix \(M\), retain \(\sigma_j>rcond\,\sigma_1\). The default is \(rcond=\epsilon_{mach}\max(M.shape)\); an overridden value is a prespecified or development-selected hyperparameter and must be stored and included in sensitivity analysis. Apply the same declared \(rcond\) convention to the state covariance solve, \(C_\tau\), generated-range restriction, and LR/LE Gram solves; do not mix library defaults or choose a cutoff from the held-out realization. For symmetric eigensystems, use the analogous threshold relative to the largest nonnegative eigenvalue after any declared metric whitening. Report \(rcond\), absolute thresholds, effective ranks, smallest retained singular values and condition estimates. An algebraically exact training solution with poor held-out behavior remains empirically unsupported.

The formal Phase 3--5 runner prespecifies \(rcond=10^{-12}\), uses no ridge, loading, shrinkage, or other regularization, and labels every fold **truncated**. That same \(10^{-12}\) cutoff is passed unchanged to the state-covariance pseudoinverse, propagator generated-rank decision, and LR/LE Gram rank decisions and pseudoinverses. Each fold reports the requested observable rank, effective observable rank, generated rank, retained state rank and energy, propagator threshold, smallest retained singular value and condition estimate.

## 10. What is provable and what is empirical

Given the stated finite-dimensional assumptions, one can prove: observable conservation \(C_\tau P=C_\tau\) for a valid generalized inverse; LR minimum expected linear reconstruction residual; LE minimum \(G\)-norm subject to the linear constraint; and transfer of observable-POD ordering/goal resolution under an exact surjective map.

Data must establish: adequacy and stability of the linear map, effective surjectivity/rank, delay, generalization, robustness to POD rank/metric/regularization, physical reproducibility, and interpretation of modes as mechanisms. Correlation, prediction, or modal overlap alone does not prove causality, controllability, necessity, or controller use. A contiguous holdout from one trajectory measures temporal extrapolation within one realization, not independent-realization generalization.

## 11. OID–CCD boundary

OID asks which correction-state directions reconstruct or generate a declared observable under \(C_\tau\), with LR/LE distinguished by their pseudoinverses and metrics. CCD asks how two multivariate processes/subspaces co-vary after covariance whitening (in this project, commonly correction versus target, action, force, or delayed signature). CCD/CCA directions optimize normalized correlation; OID modes optimize observable goal resolution through a propagator. Shared collection, POD, delay pairing, or a similar zero-lag subspace does not make the methods interchangeable. Results may be compared only after matching state, observable, clocks, mask, metric, training split and rank.

## 12. Validation and claim policy

A paper-eligible OID claim requires, at minimum:

1. immutable input identities and run manifest; exact timeline/segment checks and a declared state/observable contract;
2. development-only fitting of means, scales, PODs, \(C_\tau\), delay, ranks, penalties and the rank cutoff;
3. held-out reconstruction/prediction against POD and mean baselines, plus reported residuals for \(b-C_\tau a\), \(a-C_{LR}^-b\), and the LE constraint;
4. rank, metric, delay and regularization sensitivity; subspace rather than signed-mode comparison where appropriate;
5. independent realizations for generalization claims, with uncertainty across realizations; contiguous holdout must be labelled as such;
6. explicit status (`exact`, `truncated`, `regularized`, or diagnostic) and separate LR/LE results;
7. per-channel held-out scores and one declared metric-weighted aggregate. For channel \(j\), report \(R_j^2=1-\sum_t e_{tj}^2/\sum_t(y_{tj}-\bar y_{j,dev})^2\), including negative values and the zero-variance policy. For the aggregate, declare \(H\) and report \(R_H^2=1-\sum_t e_t^THe_t/\sum_t(y_t-\bar y_{dev})^TH(y_t-\bar y_{dev})\). An unweighted mean of channel-wise \(R^2\) is a supplementary summary, not the metric-weighted primary score, unless that choice is explicitly justified as the scientific metric;
8. selection-adjusted circular time-shift null tests as specified below when significance against temporal association is claimed;
9. claim language bounded by evidence: “associated,” “linearly reconstructible,” or “minimum-norm under the fitted map” until interventions or controlled counterfactuals justify causal/control claims.

### Circular time-shift null

Call these **circular time shifts**, not “phase surrogates.” Machine-readable report names use the `circular_time_shift` stem: `circular_time_shift_null_contract`, `circular_time_shift_surrogate_scores`, and `circular_time_shift_surrogate_selections`. Within each training realization and uninterrupted segment, roll the complete multichannel observable sequence relative to the field sequence by an admissible integer offset measured in saved samples, with wraparound. A common offset for the channel vector preserves every channel's sample distribution and circular autocorrelation (and preserves contemporaneous relations among observable channels), while disrupting contemporaneous field–observable pairing under a stationary or periodic null. This is a Theiler-style surrogate-data test in rationale, but the randomization here is a circular shift, not Fourier phase randomization.

The admissible shift set is fixed before any null score is computed. Estimate a dominant period separately from each **training** realization's complete multichannel observable; the held-out realization is never used. For an \(N\times p\) record, subtract each channel mean, compute the deterministic real FFT along time, discard the zero-frequency bin, square magnitudes, and normalize each channel's nonzero-bin powers by that channel's total nonzero spectral energy. Average these normalized spectra across channels, choose the first maximum nonzero bin \(k\), and define the period as \(N/k\) samples. The peak concentration is the selected value of that channel-averaged normalized spectrum. A channel with no nonzero spectral energy makes the period unavailable and the formal null fails closed.

Let \(w=\max(2,1+\max\{\ell:\ell\text{ is a candidate sample lag}\})\). Among integer offsets \(0,\ldots,N-1\), exclude the circular-zero neighborhood (offsets with circular distance at most \(w\) from zero) and every offset whose ordinary sample distance is at most \(w\) from any positive integer multiple of the estimated period that lies within \(w\) of the record. Draw uniformly from the remaining realization-specific offsets. Fewer than two admissible offsets fails closed. Report \(w\) in samples and physical time, each estimated period in samples and time, peak concentration, dominant bin, and exact excluded/admissible sets and counts. Wraparound creates an artificial endpoint adjacency for nonperiodic records, so the test is credible only for approximately stationary records or records whose endpoints are physically periodic. Drift, transients, pseudoperiodicity, uncertain period and coherence times comparable to record length can make the null anti-conservative or powerless and must be diagnosed rather than hidden.

For a formal test use 999 independently drawn admissible circular shifts per outer fold. Each surrogate repeats the complete development-only selection procedure—including candidate rank, delay, regularization and any other tuned choice—before evaluation with the same held-out scoring rule. The upper-tail Monte Carlo value is \(p=(1+\#\{T_s\ge T_{obs}\})/(999+1)\); declare sidedness and statistic in advance. Reusing the observed model selection for shifted data is not selection-adjusted. Exploratory runs with fewer shifts are diagnostics, not formal p-values.

With exactly three realizations, leave-one-realization-out (LORO) yields three dependent outer-fold estimates; each model is trained on only two realizations, and training-only selection has extremely low realization-level resolution. The three folds do not estimate a broad population distribution, justify asymptotic error bars, or establish robustness across operating conditions. Report all three fold scores, selected hyperparameters, null p-values and mode-subspace comparisons without treating snapshots as independent replicates. Aggregate language is limited to these three declared realizations; stronger generalization requires more independent realizations.

The realization-derangement negative control repeats the complete training-only rank/lag selection on the deranged training observables and uses that reselected model for its held-out score. Its fixed formal gate is
\[
R^2_{H,\mathrm{observed}}>R^2_{H,\mathrm{deranged}}+0.01
\]
in every fold. Report the deranged control's selected rank, selected lag, and full rank/lag sensitivity; reusing the observed selection is invalid.

Failed gates, null results and unsupported channels remain reported. Archived v1 numbers, synthetic contract tests, pilot runs, and immutable audit artifacts are provenance or diagnostics, not substitutes for these gates.

## References

- Sirovich, L. (1987), “Turbulence and the dynamics of coherent structures. Part I: Coherent structures,” *Quarterly of Applied Mathematics* 45, 561–571. This is the snapshot-POD grounding used here.
- Schlegel, M. et al. (2012), “On least-order flow representations for aerodynamics and aeroacoustics,” *Journal of Fluid Mechanics* 697, 367–398. The repository-local reproduction is `papers/Sch12.md`.
- Theiler, J., Eubank, S., Longtin, A., Galdrikian, B. & Farmer, J. D. (1992), “Testing for nonlinearity in time series: The method of surrogate data,” *Physica D* 58, 77–94. It grounds the surrogate-data testing logic; the exact null used here is the explicitly defined circular-shift randomization above.
