# OID Analysis Knowledge Base

## Document role

Same as sibling projects (SR_analysis, CCD_analysis): confirmed facts, hard rules, critical caveats, and current results. Does NOT contain execution plans (see OID_notes.md).

## Companion documents

- `OID_notes.md` -- execution plan, task tracking, current priority
- `analysis_notes.md` -- project-wide analysis task book
- `analysis_knowledge.md` -- project-wide analysis knowledge

---

## CRITICAL RULES

### Rule 1: OID default object is Delta-q_ctl, NOT raw full field

OID operates on:

\[
\Delta q_{ctl} = q_{ctl} - q_{blk}
\]

Raw full-field OID is NOT the default. The only exception is when explicitly comparing correction-field vs raw-field POD performance.

### Rule 2: force-OID and signature-OID must be reported SEPARATELY

They should NOT be merged into a single observable. If they differ, this is a potential mechanism result, not a failure.

### Rule 3: Steady cloak is NOT a periodic future-signature problem

Its primary questions are:
- Which correction structures suppress natural shedding?
- Which structures restore the mean wake?
- Which structures correlate with recirculation collapse?

Do NOT force it into the periodic signature-OID template.

### Rule 4: Three-field decomposition is MANDATORY before OID

Every scene requires:
- `q_in`: incident reference field
- `q_blk`: fixed pinball field (zero rotation)
- `q_ctl`: controlled pinball field (DRL or open-loop)

And the derived fields:
- `Delta_q_blk = q_blk - q_in`: passive blockage
- `Delta_q_ctl = q_ctl - q_blk`: active correction

### Rule 5: No field cropping

All fields must be full 1280x512 resolution. ROI masking is done only at the analysis stage (POD) using spatial masks, not by saving cropped fields.

### Rule 6: Model naming conventions (inherited from SR/CCD)

- "2U" in model name means S_DIM=14 (2 extra target force observations). NOT 2x velocity. u0 is ALWAYS 0.01.
- "1U" means S_DIM=12. NOT 1x velocity.
- nu=0.004 unless Vis suffix in model name (e.g. "02Vis" = nu*0.02 = 0.00008).
- SAMPLE_INTERVAL per diameter: 0.75L=400, 1.0L=600, 1.5L=800 (no S suffix = default 800).

### Rule 7: action_bias != preset_action

- `action_bias` (e.g. [0, -2, 2]): constant added to scaled DRL action: `omega = (action*scale + bias) * U0`
- `preset_action` (e.g. [0, 0, 0, 0, -1*U0, 1*U0]): fixed Omega array to warm up FIFO before inference
These are DIFFERENT values and purposes.

### Rule 8: GPU state contamination prevention

- Running PPO inference after other CFD on the same GPU degrades similarity
- Minimum 4NX/U0 steps between different PPO scenes on the same GPU
- Prefer separate GPUs for Karman (device 1) and Illusion (device 3) scenes
- Fresh Context per collection (don't reuse FlowField across scenes)

### Rule 9: context.push()/pop() around every run() call

Both SR and CCD do this. Action smoother is internal to run(). Pattern:

```python
ff.context.push()
ff.run(SAMPLE_INTERVAL, temp)
ff.context.pop()
```

---

## Current Results (Complete Phase 2, 2026-06-22)

### steady_cloak

| Metric | Value |
|--------|-------|
| Correction-field POD energy (5 modes) | 99.97% |
| Rank sensitivity | cosine sim = 1.000 |
| N snapshots | 100 (POD) |
| Force-OID mode 1 singular value | 0.544 |
| Force-OID mode 2 singular value | 0.074 |
| Force-OID cum_corr (1 mode) | 0.880 |
| Force-OID vs Suppression-OID mode 1 overlap | 0.763 |
| **Re-analysis: Full-field RMS reduction** | **99.43%** |
| **Re-analysis: Recirculation area collapse** | **38.5%** (Ar_ctl=1234 vs Ar_blk=2008) |
| **Re-analysis: Recirculation length collapse** | **3.2%** (Lr_ctl=269 vs Lr_blk=278) |
| **Re-analysis: Fy RMS reduction** | **83.3%** |
| Status | **Re-analysis complete**. Strong suppression (99.4% RMS reduction). Recirculation area shrinks but length barely changes -- suggests the bubble becomes narrower but not shorter. Force-OID and suppression structures are related but distinct (0.763 overlap). |

### karman_re100

| Metric | Value |
|--------|-------|
| Correction-field POD energy (5 modes) | 99.9% |
| Rank sensitivity | cosine sim = 1.000 |
| N snapshots | 500 |
| Phase 1-2 | Complete |
| **Phase 3: Force-OID** | S=[0.966, 0.724]; cum_corr(1)=0.572, cum_corr(2)=1.0 |
| **Phase 4a: Signature-OID (current)** | S=[1.235, 0.582, 0.383]; cum_corr(3)=0.965 |
| **Phase 4a: Signature-OID (delayed)** | S=[1.253, 0.670, 0.515]; cum_corr(3)=0.967 |
| **Phase 4b: Signature-PCD** | S=[0.942, 0.817, 0.595]; cum_corr(3)=0.485 (whitened) |
| **Phase 6: Force prediction** | OID(m=2) R2=0.750 vs POD(m=2) R2=0.418 -- OID wins |
| **Phase 6: Future sig prediction** | All R2 ~ 0 (delayed sensor error near-zero variance) |
| **Phase 7: Whitebox** | obs->act R2=0.956, OID->act R2=0.225, OID+force->act R2=0.233 |
| Status | **Phase 3-7 complete**. Strong OID advantage for force prediction. OID-only does not capture full control law. |

### illusion scenes (3 diameters)

| Metric | 0.75L | 1.0L | 1.5L |
|--------|-------|------|------|
| Correction-field POD energy (5 modes) | 99.93% | 99.91% | 97.9% |
| Rank sensitivity | 1.000 | 1.000 | 1.000 |
| N snapshots (POD) | 100 | 100 | 100 |
| Force-OID S[0] | 0.699 | 1.447 | 0.981 |
| Force-OID S[1] | 0.671 | 0.832 | 0.407 |
| Signature-OID (delayed) S[0] | 2.880 | 3.402 | 2.848 |
| Signature-OID (delayed) S[1] | 2.134 | 2.321 | 1.893 |
| Force prediction: OID(m=2) R2 | **0.435** | **0.671** | **0.640** |
| Force prediction: POD(m=2) R2 | -2.426 | -0.237 | 0.264 |
| Signature prediction: Sig-OID(m=2) R2 | **0.661** | **0.586** | **0.315** |
| Signature prediction: POD(m=2) R2 | -0.034 | -0.160 | 0.060 |
| Force-vs-Signature OID mode 1 overlap | **-0.082** | **-0.495** | **-0.932** |

**Key observation**: As target diameter increases, force-OID and signature-OID modes systematically diverge (overlap: -0.082 to -0.932). Supports hypothesis that force-relevant and signature-relevant structures are distinct.

### Cross-scene: Force-OID vs Signature-OID mode overlap

| Scene | Cosine similarity | Interpretation |
|-------|-----------------|----------------|
| steady_cloak | +0.763 | Force and suppression structures highly related |
| karman_re100 | -0.034 | Nearly orthogonal |
| illusion_0.75L | -0.082 | Near-orthogonal (small target) |
| illusion_1.0L | -0.495 | Moderately separated |
| illusion_1.5L | -0.932 | Strongly separated |

The monotonic trend from + (steady) through 0 (Karman) to -- (illusion, growing with diameter) suggests a **systematic mechanism**: force and signature involve increasingly different correction structures as the task transitions from suppression to preservation to retuning.

### Robustness: POD Rank Sensitivity

| Scene | r=6 | r=8 | r=10 | r=12 | r=16 | std | Verdict |
|-------|-----|-----|------|------|------|-----|---------|
| steady_cloak | -0.49 | -0.78 | -0.76 | -0.73 | -0.68 | 0.10 | Sign consistent |
| karman_re100 | 0.14 | -0.04 | -0.03 | 0.01 | -0.05 | **0.07** | **Stable (near zero)** |
| illusion_0.75L | -0.20 | 0.08 | -0.08 | -0.50 | 0.12 | **0.26** | **Unstable -- needs more data** |
| illusion_1.0L | -0.44 | -0.47 | -0.50 | -0.44 | -0.42 | **0.03** | **Very stable** |
| illusion_1.5L | -0.97 | -0.96 | -0.93 | -0.93 | -0.91 | **0.02** | **Very stable** |

### Robustness: Karman tau_c Sensitivity

| tau_c | 0 | 10 | 15 | 20 | 25 | 30 | 40 | 60 |
|-------|---|---|---|----|----|----|----|-----|
| Overlap | 0.31 | 0.12 | 0.12 | 0.11 | 0.14 | 0.14 | 0.15 | 0.19 |
| Sig R2 | 0.28 | 0.31 | 0.32 | 0.33 | 0.33 | 0.31 | 0.30 | 0.26 |

Overlap stays near-orthogonal for ALL delays. The Karman force-sig separation is NOT a delay-misalignment artifact.

### White-box Chain (Karman, from Phase 7)

| Model | Action R2 | Meaning |
|-------|:--------:|---------|
| obs -> act | **0.956** | PPO baseline uses raw sensor observations |
| force-OID coord -> act | **0.225** | OID finds observable-relevant, not action-relevant structures |
| OID + force -> act | 0.233 | Adding force doesn't help |

### Figures Generated

All figures in `src/OID_analysis/data/derived/figures/`:

| Fig | File | Description |
|-----|------|-------------|
| 1 | `fig1_force_sig_overlap.png` | **Flagship**: force-OID vs sig-OID overlap across 5 scenes (signed + absolute) |
| 2 | `fig2_rank_sensitivity.png` | POD rank sensitivity (r=6,8,10,12,16), 5 subplots |
| 3 | `fig3_oid_vs_pod_r2.png` | OID vs POD prediction R2 for force and signature |
| 4 | `fig4_tauc_sensitivity.png` | Karman tau_c sensitivity sweep (10 delays) |
| 5 | `fig5_pod_energy.png` | Correction-field POD energy capture |
| 6 | `fig6_steady_metrics.png` | Steady cloak suppression metrics |
| 7 | `fig7_whitebox_summary.png` | White-box chain: obs->z->act |

---

## Known Bugs (All FIXED)

| Bug | Description | Fix |
|-----|-------------|-----|
| Hardcoded paths | Collection scripts wrote to hardcoded dirs instead of data_dir_for_scene() | Switched to data_dir_for_scene() |
| Karman blk overwrite | karman_blk.py wrote to karman_re100/ directory, corrupting controlled data | Fixed path to karman_blk/ |
| Field size mismatch | PPO replay used ROI cropping while baselines were full field | Removed cropping, auto-align in compute_delta |
| Wrong POD loading | All Phase 3-7 scripts expected .npy but save() created .npy.npz | Changed all pod_fp to .npy.npz |
| Steady cloak negative R^2 | Used time-series R^2 for near-steady signal | Expert: replace with RMS reduction / Lr collapse metrics |

---

## OID Discipline (inherited from task book)

1. OID identifies directions within chosen POD subspace -- it CANNOT recover structures truncated by POD rank reduction.
2. All OID results must specify: chosen POD basis rank, rank sensitivity, delay definition (if applicable).
3. OID is a structural diagnosis tool, NOT an automatic mechanism generator.
4. OID results must be interpreted within the physical layers: correction field -> body-connected near wake -> downstream descendant structures.
5. If force-OID and signature-OID differ, first check rank sensitivity and data length before discussing mechanism.

---

## Key References for OID Interpretation

| Reference | Role in project |
|-----------|----------------|
| [Sch12] Schlegel et al. (OID) | Observable-Inferred Decomposition framework. Provides cross-covariance SVD for identifying observable-relevant structures in POD subspace. |
| [Lyu23] Lyu et al. (CCD/PCD) | Canonical Correlation Decomposition with delay embedding. Used for signature-PCD with whitening. |
| [Kan17b] Kantsios et al. (wake-to-force) | Body-connected near wake as primary force determinant. Supports force-OID window choice. |
| [Che19, Che21b] Chen-Liu (vorticity dynamics) | Rotation first rewrites near-body source terms. Supports `act -> near-body correction` causal ordering. |
| [Tad10] Tadmor et al. (observability) | Low-dimensional state for flow control. Supports OID-to-whitebox chain. |