# OID Analysis Handover Notes

## For the incoming agent

---

## Quick Overview

You are taking over the **OID analysis line** of the DynamisLab fluidic pinball project. This is one of three parallel analysis pipelines:

- **SR/SINDy**: `obs -> act` white-box control law extraction (most mature)
- **CCD**: `structure -> force/signature` phase-aligned correlation decomposition (correction-field analysis in progress)
- **OID**: `Delta-q_ctl -> structure -> force/signature` full-time-series observable-related decomposition (this line)

OID has been fully implemented as an independent project under `src/OID_analysis/`. All 5 scenes (steady cloak, Karman cloak, illusion 0.75/1.0/1.5L) have been analyzed end-to-end through Phases 1-7.

---

## Start Here (in order)

1. **`docs/OID_analysis_results.md`** -- Full project report with 7 figures. Read this first. It explains OID concepts, all results, and all caveats.

2. **`src/OID_analysis/README.md`** -- Engineering entry point. How to run, directory structure, common pitfalls.

3. **`src/OID_analysis/OID_knowledge.md`** -- Confirmed facts, critical rules (15+ rules), current results, bug history.

4. **`src/OID_analysis/OID_notes.md`** -- Task tracking, open items.

5. **`docs/ccd_correction_field_report.md`** -- CCD report (sibling project, important cross-reference)
6. **`docs/SR_analysis_results.md`** -- SR report (control law white-box results)

---

## The Key Concept

OID (Observable-Inferred Decomposition) answers: "Which flow structures most affect a chosen **observable** (force, sensor error)?"

It does this by:
1. Starting with a POD basis (unified coordinate system, but ranked by energy not task relevance)
2. Computing cross-covariance between POD coefficients and the observable
3. SVD to find directions in POD space that best correlate with the observable

OID operates on **correction fields** Delta-q_ctl = q_ctl - q_blk (controlled field minus baseline zero-rotation field), NOT on raw controlled fields.

---

## The Flagship Result

Force-relevant and signature-relevant correction structures **systematically separate** across control tasks:

```
steady_cloak (+0.763) --> Karman (-0.034) --> illusion 0.75L (-0.082) --> illusion 1.0L (-0.495) --> illusion 1.5L (-0.932)
```

where the number is cosine similarity between force-OID mode 1 and signature-OID mode 1.
The monotonic trend from same-channel to strongly opposite is the project's most compelling new physical finding.

OID also consistently beats POD for predicting force and signature in all scenes.

---

## Key Files

| File | Purpose |
|------|---------|
| `src/OID_analysis/configs.py` | Scene definitions (12 scenes) |
| `src/OID_analysis/utils/analysis.py` | POD, OID, PCD, statistics (CPU, no GPU) |
| `src/OID_analysis/utils/cfd_interface.py` | Re-exports from CCD (GPU) |
| `src/OID_analysis/analysis/phase3_force_oid.py` | Force-OID implementation |
| `src/OID_analysis/analysis/phase4a_signature_oid.py` | Signature-OID implementation |
| `src/OID_analysis/analysis/phase4b_signature_pcd.py` | PCD whitened cross-correlation |
| `src/OID_analysis/analysis/robustness_analysis.py` | Rank/tau_c/window robustness |
| `src/OID_analysis/analysis/make_figures.py` | Generate all 7 figures |
| `src/OID_analysis/scripts/collect_illusion_qblk.py` | **Important**: illusion-position q_blk (separate from cloak) |

---

## Environments

```bash
# CFD data collection (GPU required, 2 GPUs available: 1 and 3)
conda run -n pycuda_3_10 python src/OID_analysis/scripts/...

# Analysis (CPU only)
conda run -n sr_env python3 src/OID_analysis/analysis/...
```

---

## Open Items (Priority Order)

### P0 - Should fix next
1. **Illusion 0.75L rank instability** (std=0.26 across r=6..16). Likely needs longer time series. Current 100 snapshots for POD may be insufficient.
2. **Karman future-signal R2~0** -- currently near zero. Consider reformulating the signature observable as phase-error instead of direct sensor error.

### P1 - Important but not blocking
3. **OID mode-to-field visualization** -- OID spatial modes are computed but not plotted. Would show whether force-sig separation maps to different physical regions (near-body vs downstream).
4. **causal-PCD** -- Need a separate action-PCD to get action-related z_act coordinates for the `obs -> z -> act` chain.

### P2 - Enhancement
5. **Cross-validation** across multiple independent rollouts
6. **Vortex scenes** extension (data collected in SR but not in OID)
7. **CCD cross-validation** of the force-sig separation trend using phase-aligned data

---

## Data to Keep / Not Commit

**Commit these** (small, reproducible):
- All `.py` files in `src/OID_analysis/`
- All `.md` files in `src/OID_analysis/`
- `docs/OID_analysis_results.md`
- JSON configs in `data/configs/`

**DO NOT commit** (large, regeneratable):
- All `.npz`, `.npy` in `data/`
- All `.png` in `data/derived/figures/`
- Check `.gitignore` for proper exclusion

---

## Context on SR and CCD

- **SR/SINDy** is the most mature line. Best result: Karman cross-Re unified backbone achieves 94.4% of PPO closed-loop performance. New phase-state features for illusion achieve 100.2% of PPO with zero action memory. The SR agent wrote a detailed README at `src/SR_analysis/README.md`.

- **CCD** is in correction-field transition. Round 5 (raw-field baseline) is frozen. Round 6 (correction-field) Phase 1-2 complete. The key CCD finding that cross-validates OID: zone-restricted analysis shows force-sig structures are almost orthogonal in the sensor zone at zero lag (O=0.01) but converge after convective delay (O=0.72-0.92).

- The **correction-field framework** (Delta-q_ctl) is the shared analysis object across all three lines. All analyses should use it.

---

## Final Notes

- The OID analysis code is self-contained. It re-uses `CCD_analysis.utils.cfd_interface` for GPU operations but has its own analysis utilities and configs.
- The `data_dir_for_scene()` function in `configs.py` is the single source of truth for all data paths. Never hardcode paths.
- The most likely next useful step is: (a) fix the 0.75L rank instability, (b) produce the mode-to-field plots, (c) feed OID coordinates into SR's SINDy framework for the `obs -> z -> act` test.