# Reproduction Knowledge Document

> **Purpose**: Complete record of all experience, pitfalls, and findings from reproducing legacy DRL pinball control results on the new CelerisLab CFD solver.
> **Date**: 2026-06-21 (original reproduce), updated 2026-07-12 (inlet fix verified)
> **Next step**: Train new PPO models from scratch on the new CelerisLab solver. See `phase2_open_loop.py` (open-loop CFD validation) and `phase3_reproduce.py` (DRL inference with legacy-compat config) for the current reproduce pipeline.

---

## 1. Overall Picture

### What was attempted

Take old PPO models (trained on `LegacyCelerisLab/FlowField`) and run them on the new `CelerisLab/Simulation` solver, comparing sensor signals, forces, normalized observations, DTW similarity, and reward values against `SR_analysis` reference data.

### Conclusion

**Old PPO models cannot be perfectly transferred to the new CelerisLab solver.** Two independent LBM implementations produce systematically different plant dynamics (especially in vortex-dominated wake regions). Retraining on the new solver is the only complete solution.

### Best results achieved (Karman Cloak, legacy norm)

| Metric | Our result | Reference | Coverage |
|--------|:----------:|:---------:|:--------:|
| Action correlation (aF/aB/aT) | +0.83 / +0.81 / +0.63 | — | Good |
| Front_fy correlation | +0.90 | — | Excellent |
| DTW similarity | 0.916 | 0.954 | 96% |
| Sensor correlation | 0.69 ~ 0.72 | — | Moderate |
| Reward | 0.412 | 0.644 | 64% |

### All scenes summary

| Scene | DTW similarity | Action corr | Key issue |
|-------|:--------------:|:-----------:|-----------|
| Steady Cloak | N/A (open-loop) | N/A | Fully works |
| Karman Cloak (legacy norm) | 0.916 | 0.63-0.83 | Rear fy std 6x larger |
| Karman Cloak (new norm) | poor | -0.07~-0.04 | Uncorrelated actions |
| Illusion 1L (legacy norm) | 0.912 | 0.29-0.52 | Moderate correlation |
| Illusion 0.75L (legacy norm) | 0.926 | -0.32~+0.50 | Rear action sign inverted |
| Illusion 1.5L (legacy norm) | 0.894 | 0.08-0.15 | Low correlation |
| Vortex Lamb (legacy norm) | 0.955 | -0.22~-0.26 | Norm 16% diff |
| Vortex Taylor (legacy norm) | 0.979 | 0.01-0.39 | Norm 60% diff |

---

## 2. Critical API Differences

### 2.1 Omega: Surface Velocity vs Angular Velocity + Sign Inversion

**This was the most impactful bug.**

| Solver | Kernel code | Meaning |
|--------|------------|---------|
| **Legacy** `FlowField` | `Uw = action[id_obj] * (y_c - y) / radius` | action = tangential surface velocity |
| **New** `Simulation` | `Uw = -omega * ry` | `set_body(id, omega=val)` = angular velocity |

The new CelerisLab kernel has `Uw = -omega * ry` — **the leading minus sign means `omega > 0` produces CW rotation**, opposite to the documented convention. The old `FlowField` has no minus sign.

**Corrected conversion**:
```python
omega = -surface_vel / radius
```
Where `surface_vel = (action_norm * scale + bias) * U0` and `radius = L0/2 = 10`.

### 2.2 Sensor Area Normalization

| Solver | Physical meaning | Value |
|--------|-----------------|-------|
| **Legacy** `obs[i]` | `sum_cells(u) / steps` | No area division |
| **New** `read_sensor(id, normalize=True)` | `sum_cells(u) / (cell_count * steps)` | Area- AND time-averaged |

**Conversion**: `legacy_equiv = new_value * cell_count`

Sensor cell count for radius=5: **78 cells** (verified empirically).

Key insight: if using `legacy norm` with new sensor values, you MUST convert sensors to old-equiv first:
```python
sensor_old_equiv = sim.read_sensor(id, normalize=True) * 78.0
```

### 2.3 Action Smoothing

Legacy `FlowField.run()` has built-in exponential smoothing with `weight=0.1`:
```
pinned = 0.9 * pinned + 0.1 * target
```

New CelerisLab has NO built-in smoothing. Must implement manually.

**Critical: EMA must be initialized to the bias values, NOT zeros.**
In legacy code, `self.action` retains the last bias value after the bias FIFO phase, so the first DRL inference step starts from a smoothed bias state:

```python
ema = EMA(weight=0.1)
ema.reset(bias_omega.copy())  # NOT ema.reset(np.zeros(3))
```

### 2.4 Snapshot/Restore Timing

**The snapshot must be taken AFTER the bias FIFO, not before.**

Legacy `cfd_interface.py` `add_pinball()` does:
1. Stabilize pinball (zero action)
2. `get_ddf()` + `save_ddf()` — saves zero-action state temporarily for norm
3. Collect norm, apply_bias, then does `get_ddf()` + `save_ddf()` again — **overwrites DDF with bias-state**
4. So `reset()` → `restore_ddf()` goes to **post-bias** state

Correct behavior:
```python
sim.run(warmup)                     # stabilize
sim.snapshot()                       # OPTIONAL: temp save for norm
# ... collect norm ...
sim.restore()                        # back to pre-bias state
# ... bias FIFO ...
sim.snapshot()                       # OVERWRITE: now at post-bias state
# DRL inference starts here:
sim.restore()                        # goes to post-bias state
```

### 2.5 Runtime Body Addition (sync_bodies)

```python
n_before = sim.bodies.count
sim.add_body("circle", center=(x, y, 0.0), radius=r)  # returns -1 (staged)
sim.sync_bodies()                                      # commit
pinball_ids = list(range(n_before, n_before + 3))       # real IDs discovered AFTER sync
```

### 2.6 PyCUDA + PyTorch Context Conflict

NVIDIA V100 has only one CUDA context per process. Solution: load PPO model on CPU:
```python
model = PPO.load(path, env=dummy_env, device="cpu")  # CPU avoids context conflict
```

---

## 3. Body ID and Obs Layout

### 3.1 Karman Cloak (7 objects: dist_cyl + 3 sensors + 3 pinball)

**Add order**: dist_cyl(0) → sensor0(1) [y=+2L0] → sensor1(2) [y=0] → sensor2(3) [y=-2L0] → front(4) → top(5) [y=+0.75L0] → bottom(6) [y=-0.75L0]

**Legacy obs** (14 values):
```
[0:2]  = dist_cyl fx, fy
[2:8]  = sensor0_ux,uy, sensor1_ux,uy, sensor2_ux,uy
[8:14] = front_fx,fy, top_fx,fy, bottom_fx,fy
```
**Training uses** `obs[2:14]` → skips dist_cyl forces.

**Action -> body mapping**:
| Action index | Bias (U0 mult) | Body ID | Cylinder |
|:-----------:|:--------------:|:-------:|----------|
| 0 | 0 | 4 | Front |
| 1 | -4 | 5 | Top (rear, +y) |
| 2 | +4 | 6 | Bottom (rear, -y) |

**Normalized obs order**: `[forces(6)/norm, sensors(6)/norm]` clipped [-1,1].

### 3.2 Illusion/Vortex (6 objects: 3 sensors + 3 pinball)

**Add order**: sensor0(0) → sensor1(1) → sensor2(2) → front(3) → top(4) → bottom(5)

**Obs**: `[s0_ux,uy, s1_ux,uy, s2_ux,uy, front_fx,fy, top_fx,fy, bottom_fx,fy]`
Training uses `obs[0:12]` (all channels).

**Normalized obs order (S_DIM=12)**: same as Karman: forces first, sensors second.
**Normalized obs order (S_DIM=14, Illusion)**: forces(6) + sensors(6) + target_cd(1) + target_cl(1)

---

## 4. Scene Parameters

### 4.1 Scene settings

| Scene | S_DIM | Action scale | Action bias | SI | CONV_LEN | MaxSteps | Objects |
|-------|:----:|:------------:|:-----------:|:--:|:--------:|:--------:|:-------:|
| Steady Cloak | 12 | 8 | [0, -5.1, 5.1] | 800 | 30 | 500 | 6 |
| Karman Cloak | 12 | 8 | [0, -4, 4] | 800 | 30 | 500 | 7 |
| Illusion 0.75L | 14 | 8 | [0, -2, 2] | 400 | 36 | 500 | 6 |
| Illusion 1L | 14 | 8 | [0, -2, 2] | 600 | 36 | 500 | 6 |
| Illusion 1.5L | 14 | 8 | [0, -2, 2] | 800 | 36 | 500 | 6 |
| Vortex Lamb | 12 | 4 | [0, -4, 4] | 800 | 30 | 150 | 6 |
| Vortex Taylor | 12 | 4 | [0, -4, 4] | 800 | 30 | 150 | 6 |

### 4.2 Bias init actions (for FIFO initialization - may differ from DRL bias!)

| Scene | Init bias (surface_vel) | DRL bias |
|-------|:----------------------:|:--------:|
| Karman | [0, -4, 4] * U0 | Same |
| Illusion | **[0, -1, 1] * U0** | [0, -2, 2] * U0 |
| Vortex | [0, -4, 4] * U0 | Same |

### 4.3 Norm formulas

All scenes use:
- `force_norm_fact = 6 * max(|forces|)`
- `sens_deviation[i] = mean(sensor_i)`
- `sens_norm_fact[i] = 5 * max(|sensor_i - mean|)`

### 4.4 Omega conversion per scene

```python
# Karman, Steady, Illusion:
target_omega = -(action * 8.0 + bias) * U0 / 10.0

# Vortex:
target_omega = -(action * 4.0 + bias) * U0 / 10.0
```

---

## 5. All Bugs Found & Fixed

| # | Bug | Symptom | Fix |
|---|-----|---------|-----|
| **1** | Omega sign inverted | force fy means wrong sign, front_fy anti-correlated (-0.97) with ref actions open-loop | `omega = -surface_vel / R` |
| **2** | Snapshot before bias FIFO | Inference starts from zero-action flow, DRL sees wrong obs | `sim.snapshot()` after bias FIFO |
| **3** | EMA initialized to zeros | First DRL step applies 10% of target action = no effect | `ema.reset(bias_omega)` |
| **4** | Sensor not converted to old-equiv | Sensors have intrinsic area normalization, old norm doesn't expect it | `new_sensor * cell_count(78)` |
| **5** | Wrong body ID after sync_bodies | `set_body(-1, ...)` raises KeyError | `pinball_ids = list(range(n_before, n_before+3))` |
| **6** | `snapshot()` before `restore()` | `RuntimeError: No snapshot to restore` | Always snapshot after warmup |

---

## 6. File Structure (after cleanup)

```
reproduce/
├── __init__.py
├── REPRODUCE_KNOWLEDGE.md            ← This file
├── core/
│   ├── __init__.py
│   ├── action_wrapper.py             # EMA smoother + omega conversion
│   ├── obs_normalizer.py             # Norm computation helpers
│   └── dtw_metrics.py                # DTW similarity + harmonics
├── configs/
│   ├── __init__.py
│   ├── scene_params.py               # All scene parameter definitions
│   └── model_inventory.py            # Model loading (DummyEnv + Sin)
├── run_all_cases.py                  # Steady + Karman Cloak reproduction
├── run_illusion_vortex.py            # Illusion + Vortex reproduction
└── output/
    ├── PHASE4_SUMMARY.md             # Cross-scene comparison report
    ├── steady_cloak/                 # Final steady cloak output
    ├── karman_cloak/                 # Final Karman cloak output
    ├── illusion_1L/                  # Final Illusion 1L output
    ├── illusion_075L/                # Final Illusion 0.75L output
    ├── illusion_15L/                 # Final Illusion 1.5L output
    ├── vortex_lamb/                  # Final Vortex Lamb output
    └── vortex_taylor/                # Final Vortex Taylor output
```

---

## 7. Reference Data Locations

| Scene | Path |
|-------|------|
| Karman | `src/SR_analysis/data/karman/karman_re100/` |
| Illusion 1L | `src/SR_analysis/data/illusion/illusion_1L/` |
| Illusion 0.75L | `src/SR_analysis/data/illusion/illusion_0.75L/` |
| Illusion 1.5L | `src/SR_analysis/data/illusion/illusion_1.5L/` |
| Vortex Lamb | `src/SR_analysis/data/vortex/vortex_lamb/` |
| Vortex Taylor | `src/SR_analysis/data/vortex/vortex_taylor/` |
| Steady | `src/SR_analysis/data/steady/steady/` |

Each contains: `controlled.npz`, `uncontrolled.npz` (Karman only), `target.npz`, `norm.json`, `config.json`, `result.json`

---

## 8. Recommended Workflow for New Training

```
1. Create env with new CelerisLab Simulation
   ├─ Add bodies in legacy order
   ├─ initialize(), warmup (4*NX/U0 steps)
   ├─ Record target (150 x SI steps, sensors converted to old-equiv)
   ├─ Add pinball via sync_bodies()
   ├─ Warmup pinball (zero action)
   ├─ Collect zero-action FIFO → compute norm
   ├─ Bias FIFO (EMA smoother, proper bias values)
   ├─ sim.snapshot()  ← AFTER bias FIFO!
   └─ save save_states

2. Create gym.Env wrapper (observation_space, action_space, step, reset)
   ├─ step(): apply_action_smoothed → read_obs → normalize → build_obs → compute_reward
   ├─ reset(): sim.restore(), fifo = save_states.copy()
   └─ reward: same formula as legacy (DTW + force terms)

3. Train PPO (same as legacy)
   ├─ Network: MlpPolicy, Sin activation, 64×64
   ├─ PPO hyperparams: lr=3e-4(actor)/4e-4(critic), n_steps=2048, batch_size=64
   ├─ Train for 500-1000 episodes (360-1500 timesteps per iter)
   └─ Load PPO model on CPU (avoids CUDA context conflict)

4. Evaluate
   ├─ Run deterministic inference
   ├─ Compare reward curves across seeds
   └─ Check DTW similarity against target
```
