# CCD_analysis: Observable-Correlated Decomposition for Flow Control

## Overview

This directory implements the **CCD (Cross-Correlation Decomposition)** analysis
pipeline for the DynamisLab fluidic pinball project. While POD ranks modes by
fluctuation energy, CCD ranks modes by their correlation with a chosen
observable (force, action, or sensor signature), making it the right tool for
answering "which flow structures does the controller actually modulate?"

The pipeline covers four reference cases at Re=100 (code convention, Re_D=50):

| Case | Control | Target Type | Source |
|------|---------|-------------|--------|
| **pinball** | None (uncontrolled) | Periodic | Open-loop CFD |
| **steady_cloak** | Constant rotation (rear 5.1xU0) | Steady | Open-loop CFD |
| **karman_re100** | DRL PPO (d1a3o12_re100) | Periodic | PPO inference |
| **illusion_1L** | DRL PPO (d1a3o14_250525_imit_1L_2U_600S) | Periodic | PPO inference |

For background:
- `ccd_notes.md` -- execution plan and methodological discussion
- `ccd_knowledge.md` -- confirmed facts, lessons learned, and pitfalls

## Directory Structure

```
CCD_analysis/
  configs.py                 # Unified scene metadata (4 cases)
  configs/
    config_cuda.json         # Legacy CFD CUDA config (copied from CelerisLab)
    config_flowfield.json    # Legacy CFD flow field config (copied)
  utils/
    __init__.py              # Non-pycuda exports (resampling, POD, CCD)
    cfd_interface.py         # LegacyCelerisLab wrapper (requires pycuda_3_10)
    resampling.py            # Phase resampling, POD, CCD algorithms (CPU-only)
  data/
    pinball/pinball/         # Uncontrolled pinball: sensors.npz, fields.npz
    steady_cloak/steady_cloak/ # Steady cloak: sensors.npz, fields.npz
    karman/karman_re100/     # Karman cloak: target.npz, norm.json, controlled.npz
    illusion/illusion_1L/    # Illusion: target.npz, norm.json, controlled.npz
    resampled/               # Phase-resampled data (24 pts/cycle)
    ccd/                     # CCD results (ccd_results.json)
    steady/                  # Steady metrics (steady_metrics.json)
  scripts/
    collect_karman.py        # Karman cloak PPO inference -> data/karman/
    collect_illusion.py      # Illusion PPO inference -> data/illusion/
    collect_pinball.py       # Pinball baseline -> data/pinball/
    collect_steady_cloak.py  # Steady cloak open-loop -> data/steady_cloak/
    resample.py              # Phase resampling for periodic cases
  ccd/
    run_ccd.py               # POD + force/action CCD computation
  steady/
    run_steady.py            # Steady cloak metrics
```

## Key Design Decisions

### 1. Scene Metadata Driven

All scene parameters are defined once in `configs.py`, not hard-coded in
scripts. Each scene dict contains geometry, DRL parameters, and inference
settings. Adding a new scene means adding one dict.

### 2. Verified CFD Interface

`utils/cfd_interface.py` is adapted from `SR_analysis/utils/cfd_interface.py`
(which was itself verified against `analysis_crossre`). It contains the
environment-building functions that exactly replicate the legacy training
environments:

- `build_karman_cloak_env()` -- mirrors `legacy_env_karman_cloak_standard.py`
- `add_pinball()` -- norm computation + bias-action FIFO, configurable for
  Karman (7 objects) and Illusion (6 objects) layouts
- `build_observation()`, `scale_action()` -- DRL obs/action helpers
- `compute_similarity()` -- lag-compensated DTW for reward validation

### 3. Data / Analysis Separation

- `data/` -- raw sensor/force/action/field arrays (.npz), one-time generation
- `ccd/`, `steady/` -- analysis results, regeneratable from `data/`
- `scripts/` -- inference pipelines that produce `data/`

### 4. Two-Pass Collection (PPO cases)

For DRL cases (karman, illusion), data collection uses a two-pass strategy:

1. **Closed-loop pass**: Run PPO inference, record `controlled.npz` with
   actions/sensors/forces/rewards, validate similarity against target
2. **Open-loop replay** (optional): Reset to checkpoint, replay saved actions
   without PPO, collect dense field snapshots for CCD

This decouples field sampling from PPO state management, ensuring the DRL
observation pipeline is not disturbed by field I/O.

### 5. Validation Gate

Each PPO case computes a similarity score (lag-compensated DTW between
controlled sensor signals and target reference). Only passing cases
(similarity >= 0.80 for Karman, >= 0.70 for Illusion) should proceed to CCD.

## Verified Data Quality

| Scene | Similarity | Notes |
|-------|-----------|-------|
| karman_re100 | 0.950 | Verified against analysis_crossre reference |
| illusion_1L | ~0.84 | Below thesis 0.975; under investigation |
| steady_cloak | N/A (steady) | Sensor std=0.000344, no residual shedding |
| pinball | N/A (baseline) | St=0.1125 at Re=100 (code) |

## Regeneration Commands

All commands run from repo root (`/home/frank14f/DynamisLab`).

### Data Generation (requires GPU, pycuda_3_10 env)

```bash
# Pinball baseline (uncontrolled, 6 objects)
conda run -n pycuda_3_10 python src/CCD_analysis/scripts/collect_pinball.py --device 2

# Steady cloak (open-loop constant rotation)
conda run -n pycuda_3_10 python src/CCD_analysis/scripts/collect_steady_cloak.py --device 2

# Karman cloak re100 (PPO, 7 objects)
conda run -n pycuda_3_10 python src/CCD_analysis/scripts/collect_karman.py --device 2 --steps 200

# 1L Illusion (PPO, 2U=0.02)
conda run -n pycuda_3_10 python src/CCD_analysis/scripts/collect_illusion.py --device 2 --steps 200
```

### Resampling (no GPU needed)

```bash
python3 src/CCD_analysis/scripts/resample.py
```

### CCD Analysis (no GPU needed)

```bash
python3 src/CCD_analysis/ccd/run_ccd.py

# Steady metrics
python3 src/CCD_analysis/steady/run_steady.py
```

## Pipeline Workflow

```
                    ┌─────────────────────┐
                    │   configs.py         │
                    │  (scene metadata)    │
                    └────────┬────────────┘
                             │
              ┌──────────────┼──────────────┐
              ▼              ▼              ▼
    ┌─────────────────┐ ┌──────────┐ ┌──────────┐
    │ collect_pinball │ │collect_  │ │collect_  │
    │ collect_steady  │ │karman.py │ │illusion  │
    │ _(open-loop)    │ │(PPO)     │ │ .py(PPO) │
    └────────┬────────┘ └────┬─────┘ └────┬─────┘
             │               │            │
             ▼               ▼            ▼
    ┌──────────────────────────────────────────┐
    │           data/{scene_id}/{scene_name}/   │
    │  sensors.npz, forces.npz, fields.npz      │
    │  controlled.npz, target.npz, norm.json    │
    └──────────────────┬───────────────────────┘
                       │
                       ▼
              ┌────────────────┐
              │ scripts/       │
              │ resample.py    │
              │ (24 pts/cycle) │
              └───────┬────────┘
                      │
                      ▼
              ┌────────────────┐
              │ data/resampled/│
              └───────┬────────┘
                      │
         ┌────────────┴────────────┐
         ▼                         ▼
  ┌──────────────┐        ┌──────────────┐
  │ ccd/run_ccd  │        │ steady/      │
  │ POD + CCD    │        │ run_steady   │
  └──────┬───────┘        └──────┬───────┘
         │                       │
         ▼                       ▼
  ┌──────────────┐        ┌──────────────┐
  │ data/ccd/    │        │ data/steady/ │
  │ ccd_results  │        │ steady_      │
  │ .json        │        │ metrics.json │
  └──────────────┘        └──────────────┘
```

## Known Issues and Caveats

1. **Illusion similarity below thesis** -- The 1L illusion achieves ~0.84
   similarity vs the thesis value of 0.975. The vorticity field shows partial
   but not perfect wake matching. Possible causes: harmonics-based target
   reconstruction may differ subtly from training, or the PPO needs longer
   warm-up. Data is still useful for CCD as a "partial illusion" reference.

2. **Karman cloak uses exactly the reference code** -- `utils/cfd_interface.py`
   is adapted from `SR_analysis/utils/cfd_interface.py`, which was verified
   against `analysis_crossre/scripts/phase1_infer.py`. The similarity of 0.95
   matches the reference.

3. **No empty channel reference for steady metrics** -- The steady cloak
   analysis currently lacks a clean parabolic channel reference flow. This
   affects the E_mean calculation. Generate via a separate FlowField with
   no bodies and a dummy sensor.

4. **CCD results are preliminary** -- Once data collection is validated, the
   `ccd/run_ccd.py` script computes POD and CCD. Results should be
   cross-checked with visual field inspection before drawing conclusions.

5. **Resampled field quality depends on source data** -- The phase resampling
   step uses linear interpolation. If the original field sampling rate is too
   low (< 12 pts/cycle), resampled fields will have interpolation artifacts.
   Currently all cases use raw sampling that gives ~18-25 pts/cycle.

## File Reference

| File | Purpose |
|------|---------|
| configs.py | Unified scene metadata (4 cases) |
| utils/cfd_interface.py | LegacyCelerisLab wrapper, env builders, DTW |
| utils/resampling.py | Period detection, phase resampling, POD, CCD |
| utils/__init__.py | Non-pycuda exports |
| scripts/collect_karman.py | Karman cloak PPO inference |
| scripts/collect_illusion.py | Illusion PPO inference |
| scripts/collect_pinball.py | Pinball baseline |
| scripts/collect_steady_cloak.py | Steady cloak open-loop |
| scripts/resample.py | Phase resampling pipeline |
| ccd/run_ccd.py | POD + CCD computation |
| steady/run_steady.py | Steady cloak metrics |
