Frank14f/DynamisLab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: project-wide cleanup — consolidate docs, remove obsolete code, update .gitignore

Browse Source 
- Remove obsolete docs (OID_handover, SR_analysis_results, ccd_* handover)
- Remove CCD legacy output_redux and old scripts
- Remove SR old sindy scripts and compare modules
- Update .gitignore to cover all analysis-generated outputs
- Retain all active code in OID/SR/CCD analysis directories

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Frank14f
2026-06-28 16:52:48 +08:00
co-authored by Cursor
parent c918ac0de4
commit 56e3c78a83
73 changed files with 1365 additions and 11072 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/HANDOVER_SR_ANALYSIS.md
-81
View File
@@ -1,81 +0,0 @@
# SR Analysis Pipeline — Handover Notes (2026-06-15)
## 交接人 → 接手人
### 当前管线状态
| 模块 | 状态 | 说明 |
|------|:----:|------|
| Illusion 0.75L + 1L phase-state + abs | **已验证** | 闭环 0.974 / 0.958，无动作历史，可进 PySR |
| Illusion 1.5L | **边界 case** | bang-bang 机制，线性 SINDy 不适用 |
| Karman phase-state + abs | **0.699** | 优于 deriv 模式，但低于旧 v23 的 0.901 |
| Karman old v23 (a_lag) | **0.901** | 保留作基线对照 |
| Karman 泛化测试 | **已完成** | Re70/150/300/25 约 0.54-0.60 |
| PySR 符号回归 | **有 shell** | 需要修复 run_pysr.py 后重新跑 |
| Vortex 偏移扩展 | **未做** | 低优先级 |
### 核心文件改动（2026-06-14~15）
| 文件 | 改动类型 |
|------|----------|
| `utils/feature_builder.py` | **新增** PHASE_STATE_KEYS, ILLUSION_PHASE_KEYS, KARMAN_EXPANDED_KEYS, obs dynamics, error-state, mu_Cl_tot |
| `utils/sindy_fitter.py` | **新增** compute_action_deriv, get_feature_matrix_deriv(output_mode) |
| `sindy/run_all_v2.py` | **新增** --deriv, --phase, --karman-expand, --karman-mu, --output-mode, --augment-level CLI |
| `validate/run_closed_loop.py` | **新增** predict_v23_deriv, mode="abs", load_sindy_coefs返回mode |
| `validate/run_closed_loop_illusion.py` | **修改** 支持 predict_v23_deriv, 自动检测模式 |
| `validate/eval_rollout.py` | **新文件** 离线多步 rollout 评估 |
| `scripts/plot_sr_results.py` | **新文件** 结果可视化图表 |
| `docs/SR_analysis_results.md` | **新文件** 完整分析报告 |
| `docs/figures/SR_analysis/fig*.png` | **新文件** 6 张图表 |
### 关键设计决策（接手前必读）
1. **phase-state 特征** = `u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear` (6维)
2. **ILLUSION_PHASE_KEYS** = phase-state + `Cd_err, Cl_err, dCd_err/dt, dCl_err/dt` (10维)
3. **绝对动作输出** `output_mode="absolute"` 优于导数 `"deriv"`，无积分累积
4. **v23 结构**始终默认：front no-bias, rear shared-head
5. **时间一阶导**统一除以 `dt_c = SAMPLE_INTERVAL/2000`，跨场景可比较
6. `controlled.npz` 中新加了 `target_forces` 字段，illusion 场景必须有
7. **FIFO bias ≠ DRL action bias**，1U vs 2U 不要混淆
### 常用命令速查
```bash
# 拟合 + 验证 Illusion phase-state + absolute (完整流程)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes illusion_0.75L,illusion_1L --deriv --phase --output-mode absolute
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop_illusion.py \
--scene illusion_0.75L --device 0 --steps 320 \
--sindy-results src/SR_analysis/sindy/illusion/sindy_results_deriv.json
# 拟合 + 验证 Karman phase-state + absolute
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes karman_re100 --deriv --phase --output-mode absolute
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scene karman_re100 --device 0 --steps 200 --mode abs \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json
# 离线 rollout 评估
python3 src/SR_analysis/validate/eval_rollout.py \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json \
--scene karman_re100
# PySR (需要先修复滞后的 bug！)
conda run -n sr_env python src/SR_analysis/sindy/run_pysr.py --scene illusion_1L
# 画图
python3 scripts/plot_sr_results.py
```
### 目前最适合推进的方向
1. **Illusion PySR 符号回归**（0.75L + 1L separate → 公式比较）
2. **Karman 状态补强**（配合 CCD/OID 分析找出缺失的状态量，再回 SR）
3. **Karman 跨 Re 联合**（在 phase-state + mu 基础上做跨 Re 联合拟合 + 泛化）
### 注意重新运行 run_pysr.py
当前 `run_pysr.py` 有路径/导入问题，接手后需先确认：
- `env_sr` 或 `sr_env` 环境内的 PySR 可用性
- whitelist 特征与 `controlled.npz` 中的字段匹配（特别是 illusion 的 `target_forces`）
- 滞后构造正确（`a_prev[1:] = actions_phys[:-1]`）
docs/OID_handover.md
-136
View File
@@ -1,136 +0,0 @@
# OID Analysis Handover Notes
## For the incoming agent
---
## Quick Overview
You are taking over the **OID analysis line** of the DynamisLab fluidic pinball project. This is one of three parallel analysis pipelines:
- **SR/SINDy**: `obs -> act` white-box control law extraction (most mature)
- **CCD**: `structure -> force/signature` phase-aligned correlation decomposition (correction-field analysis in progress)
- **OID**: `Delta-q_ctl -> structure -> force/signature` full-time-series observable-related decomposition (this line)
OID has been fully implemented as an independent project under `src/OID_analysis/`. All 5 scenes (steady cloak, Karman cloak, illusion 0.75/1.0/1.5L) have been analyzed end-to-end through Phases 1-7.
---
## Start Here (in order)
1. **`docs/OID_analysis_results.md`** -- Full project report with 7 figures. Read this first. It explains OID concepts, all results, and all caveats.
2. **`src/OID_analysis/README.md`** -- Engineering entry point. How to run, directory structure, common pitfalls.
3. **`src/OID_analysis/OID_knowledge.md`** -- Confirmed facts, critical rules (15+ rules), current results, bug history.
4. **`src/OID_analysis/OID_notes.md`** -- Task tracking, open items.
5. **`docs/ccd_correction_field_report.md`** -- CCD report (sibling project, important cross-reference)
6. **`docs/SR_analysis_results.md`** -- SR report (control law white-box results)
---
## The Key Concept
OID (Observable-Inferred Decomposition) answers: "Which flow structures most affect a chosen **observable** (force, sensor error)?"
It does this by:
1. Starting with a POD basis (unified coordinate system, but ranked by energy not task relevance)
2. Computing cross-covariance between POD coefficients and the observable
3. SVD to find directions in POD space that best correlate with the observable
OID operates on **correction fields** Delta-q_ctl = q_ctl - q_blk (controlled field minus baseline zero-rotation field), NOT on raw controlled fields.
---
## The Flagship Result
Force-relevant and signature-relevant correction structures **systematically separate** across control tasks:
```
steady_cloak (+0.763) --> Karman (-0.034) --> illusion 0.75L (-0.082) --> illusion 1.0L (-0.495) --> illusion 1.5L (-0.932)
```
where the number is cosine similarity between force-OID mode 1 and signature-OID mode 1.
The monotonic trend from same-channel to strongly opposite is the project's most compelling new physical finding.
OID also consistently beats POD for predicting force and signature in all scenes.
---
## Key Files
| File | Purpose |
|------|---------|
| `src/OID_analysis/configs.py` | Scene definitions (12 scenes) |
| `src/OID_analysis/utils/analysis.py` | POD, OID, PCD, statistics (CPU, no GPU) |
| `src/OID_analysis/utils/cfd_interface.py` | Re-exports from CCD (GPU) |
| `src/OID_analysis/analysis/phase3_force_oid.py` | Force-OID implementation |
| `src/OID_analysis/analysis/phase4a_signature_oid.py` | Signature-OID implementation |
| `src/OID_analysis/analysis/phase4b_signature_pcd.py` | PCD whitened cross-correlation |
| `src/OID_analysis/analysis/robustness_analysis.py` | Rank/tau_c/window robustness |
| `src/OID_analysis/analysis/make_figures.py` | Generate all 7 figures |
| `src/OID_analysis/scripts/collect_illusion_qblk.py` | **Important**: illusion-position q_blk (separate from cloak) |
---
## Environments
```bash
# CFD data collection (GPU required, 2 GPUs available: 1 and 3)
conda run -n pycuda_3_10 python src/OID_analysis/scripts/...
# Analysis (CPU only)
conda run -n sr_env python3 src/OID_analysis/analysis/...
```
---
## Open Items (Priority Order)
### P0 - Should fix next
1. **Illusion 0.75L rank instability** (std=0.26 across r=6..16). Likely needs longer time series. Current 100 snapshots for POD may be insufficient.
2. **Karman future-signal R2~0** -- currently near zero. Consider reformulating the signature observable as phase-error instead of direct sensor error.
### P1 - Important but not blocking
3. **OID mode-to-field visualization** -- OID spatial modes are computed but not plotted. Would show whether force-sig separation maps to different physical regions (near-body vs downstream).
4. **causal-PCD** -- Need a separate action-PCD to get action-related z_act coordinates for the `obs -> z -> act` chain.
### P2 - Enhancement
5. **Cross-validation** across multiple independent rollouts
6. **Vortex scenes** extension (data collected in SR but not in OID)
7. **CCD cross-validation** of the force-sig separation trend using phase-aligned data
---
## Data to Keep / Not Commit
**Commit these** (small, reproducible):
- All `.py` files in `src/OID_analysis/`
- All `.md` files in `src/OID_analysis/`
- `docs/OID_analysis_results.md`
- JSON configs in `data/configs/`
**DO NOT commit** (large, regeneratable):
- All `.npz`, `.npy` in `data/`
- All `.png` in `data/derived/figures/`
- Check `.gitignore` for proper exclusion
---
## Context on SR and CCD
- **SR/SINDy** is the most mature line. Best result: Karman cross-Re unified backbone achieves 94.4% of PPO closed-loop performance. New phase-state features for illusion achieve 100.2% of PPO with zero action memory. The SR agent wrote a detailed README at `src/SR_analysis/README.md`.
- **CCD** is in correction-field transition. Round 5 (raw-field baseline) is frozen. Round 6 (correction-field) Phase 1-2 complete. The key CCD finding that cross-validates OID: zone-restricted analysis shows force-sig structures are almost orthogonal in the sensor zone at zero lag (O=0.01) but converge after convective delay (O=0.72-0.92).
- The **correction-field framework** (Delta-q_ctl) is the shared analysis object across all three lines. All analyses should use it.
---
## Final Notes
- The OID analysis code is self-contained. It re-uses `CCD_analysis.utils.cfd_interface` for GPU operations but has its own analysis utilities and configs.
- The `data_dir_for_scene()` function in `configs.py` is the single source of truth for all data paths. Never hardcode paths.
- The most likely next useful step is: (a) fix the 0.75L rank instability, (b) produce the mode-to-field plots, (c) feed OID coordinates into SR's SINDy framework for the `obs -> z -> act` test.
docs/SR_analysis_results.md
-225
View File
@@ -1,225 +0,0 @@
# SR Analysis: Phase-State SINDy Results Report
> Date: 2026-06-15
> Project: DynamisLab — Active hydrodynamic cloaking and illusion using DRL on a fluidic pinball.
> Analysis pipeline: SINDy (STLSQ) + feature engineering for interpretable control law extraction.
---
## Executive Summary
Three key findings from the SR analysis pipeline:
1. **Illusion 0.75L + 1L: New route validated.** Phase-state features + error-state + absolute action output achieves closed-loop similarity of **0.96+** (97% of PPO) with **zero action history features**. This proves that physically meaningful control laws can be extracted without relying on action memory.
2. **Illusion 1.5L: Regime shift identified.** The 1.5L target exhibits bang-bang/saturated control (alpha range [-8, 8], autocorrelation r=0.07). Linear SINDy is fundamentally inadequate (R2=0.12). This is a regime boundary, not a modelling failure.
3. **Karman: State representation still incomplete.** The same phase-state approach reaches 0.699 (vs 0.901 baseline with action history). The problem is not the output form (derivative vs absolute) but insufficient input state information for the Karman scene.
---
## 1. Methodology
### 1.1 Feature Architecture (Final)
Three feature levels were tested for fitting `alpha = f(state)` or `d(alpha)/dt = g(state)`:
| Level | Features | Dim | Description |
|-------|----------|:---:|-------------|
| Static | u_m, u_a, u_c, v_a, Cd_tot, Cd_rear, Cl_tot, Cl_diff | 8 | Current-step physics only, no memory |
| Phase-state | u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear (+error terms) | 6+4 | Oscillation phase + rate + drag feedback |
| Full-lag | Static + lag-1 of all 8 | 16 | Brute-force temporal context (baseline) |
### 1.2 Output Modes
Two output targets were compared:
- **Derivative mode**: predict d(alpha)/dt, then integrate `alpha(t) = alpha(t-1) + dt_c * dalpha/dt`
- **Absolute mode**: predict alpha directly, no integration needed
### 1.3 Evaluation Metrics
Models are evaluated on:
1. **One-step R2**: fit quality on training data
2. **Offline multi-step rollout**: 1/5/20/50 step recursive prediction on held-out PPO data
3. **CFD closed-loop**: full CFD environment with SINDy control law, DTW similarity vs target
---
## 2. Illusion Results
### 2.1 Three-Scene Summary
| Scene | S | Old v23 (a_lag) | New phase+error+abs | PPO baseline | % of PPO | Features | Action history? |
|------|:--:|:----------------:|:-------------------:|:------------:|:--------:|----------|:---------------:|
| 0.75L | 400 | 0.908 | **0.974** | 0.972 | 100.2% | 10-dim (ILLUSION_PHASE) | **No** |
| 1L | 600 | 0.962 | **0.958** | 0.973 | 98.5% | 10-dim (ILLUSION_PHASE) | **No** |
| 1.5L | 800 | 0.926 | **N/A** | 0.945 | — | — | Bang-bang regime |
**Key insight**: The 0.75L scene's new route **outperforms** the old v23 (0.974 vs 0.908), while 1L matches it within 1.5%. This definitively proves that "action history is not necessary."
### 2.2 Illusion 1L Front Coefficients (Phase-State, Absolute Output)
```
alpha_F = f(u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear, Cd_err, Cl_err, dCd_err/dt, dCl_err/dt)
R2 = 0.987
```
Key contributors (sorted by |coef|): Cd_rear > Cd_tot > Cd_err > Cl_tot > Cl_err > dCl_err/dt > dCd_err/dt
The formula is dominated by **drag-based feedback** (Cd_tot, Cd_rear, Cd_err), with the oscillation phase (u_a, Cl_tot) providing secondary modulation.
### 2.3 Illusion 1.5L — Regime Shift Evidence
| Metric | 1L | 1.5L | Ratio |
|--------|:--:|:----:|:-----:|
| Alpha range (front) | [-0.36, 1.11] | **[-8.0, 8.0]** | 10x |
| 1-step autocorrelation | 0.957 | **0.065** | — |
| d(alpha)/dt std | 0.12 | **6.07** | 50x |
| Linear SINDy R2 | 0.987 | **0.124** | — |
The 1.5L controller operates at saturation limits (maximum rotation speed), flipping between extremes. This is consistent with the frequency-doubling control strategy reported in the confirmation report.
---
## 3. Karman Results
### 3.1 Ablation Study (re100)
| Configuration | Feat | Output | R2 | Closed-loop | % of PPO |
|--------------|:----:|:-----:|:--:|:----------:|:--------:|
| old v23 (a_lag1 dominant) | 14+3 | alpha | 0.996 | **0.901** | 94.4% |
| **Phase-state -> abs (best new)** | **6** | **alpha** | **0.965** | **0.699** | 73.3% |
| Phase-state -> deriv | 6 | dalpha/dt | 0.837 | 0.656 | 68.8% |
| Phase-state + mu -> abs | 9 | alpha | 0.979 | 0.700 | 73.4% |
| Expanded 10-dim -> abs | 10 | alpha | 0.980 | **0.580** | 60.8% |
| Full-lag -> deriv | 16 | dalpha/dt | 0.939 | 0.619 | 64.9% |
| Static -> deriv | 8 | dalpha/dt | 0.321 | 0.745 | 78.1% |
**Key insight from ablation**:
1. **Phase-state + absolute output is the best new route** (0.699), but still well below the action-history baseline.
2. Adding extra static features (expanded 10-dim) **hurts** closed-loop despite higher R2 — classic covariate shift.
3. The static->deriv paradox: low R2 (0.321) but good closed-loop (0.745), because no-memory models are naturally robust to rollout divergence.
4. Mu modulation doesn't help at single-Re; its value will appear in cross-Re fitting.
### 3.2 Cross-Re Generalization (old v23 model)
| Re | Type | Closed-loop | Note |
|:--:|:----:|:----------:|------|
| 50 | Training | 0.582 | Low-frequency shedding |
| 100 | Training | **0.901** | Default / best |
| 200 | Training | 0.793 | Moderate degradation |
| 400 | Training | 0.664 | High-Re challenge |
| 25 | Unseen (subcritical) | 0.567 | Below Hopf bifurcation |
| 70 | Unseen | 0.577 | Between Re50 and Re100 |
| 150 | Unseen | 0.595 | Between Re100 and Re200 |
| 300 | Unseen | 0.541 | Outer extrapolation |
### 3.3 One-Step R2 vs Closed-Loop Paradox
A key phenomenon discovered during this work: **high one-step R2 does not predict good closed-loop performance** when temporal features (lags, derivatives) are present in the input. This is because:
- **Training**: features use ground-truth PPO observations
- **Deployment**: features use SINDY-controlled observations (distribution shift)
- Temporal features amplify this shift recursively
---
## 4. Roadmap to Next Steps
### 4.1 Illusion: Proceed to PySR
The 0.75L and 1L scenes are ready for symbolic regression. Recommended input:
```python
ILLUSION_PHASE_KEYS = [
"u_a", "du_a_dt", # oscillation phase
"Cl_tot", "dCl_tot_dt", # lift dynamics
"Cd_tot", "Cd_rear", # drag feedback
"Cd_err", "Cl_err", # force error
"dCd_err_dt", "dCl_err_dt", # error dynamics
]
```
Output: absolute alpha (non-dimensional action), no integration.
First do separate PySR for each scene, then compare formula structure for shared backbone.
### 4.2 Karman: CCD/OID for state completion
The 0.699 ceiling suggests missing state variables. Candidate directions:
- Recirculation zone length / reattachment point
- Wake centerline deflection
- POD mode coefficients for phase completion
- CCD modes correlated with control action
### 4.3 SR / CCD / OID Integration Framework
```
┌─────────────────────────────────────┐
│ Control Objective │
│ (stealth / illusion / erase) │
└──────────┬──────────────────────────┘
│
┌──────────▼──────────┐
│ DRL Policy │
│ (PPO + Sin act) │
└──────────┬──────────┘
│
┌────────────────┼────────────────┐
│ │ │
┌────────▼──────┐ ┌─────▼──────┐ ┌──────▼─────────┐
│ SINDy / SR │ │ CCD / OID │ │ Validation │
│ obs -> act │ │ obs -> z │ │ CFD closed- │
│ white-box │ │ structure │ │ loop + DTW │
│ control law │ │ analysis │ │ │
└────────┬──────┘ └─────┬──────┘ └──────┬─────────┘
│ │ │
└────────────────┼─────────────────┘
│
┌──────────▼──────────┐
│ Interpretable │
│ Control Mechanics │
│ obs -> z -> act │
│ -> structure -> │
│ -> signature │
└─────────────────────┘
```
---
## 5. Code Changes Summary
### Files Modified (8 core files)
| File | Changes |
|------|---------|
| `src/SR_analysis/utils/feature_builder.py` | PHYSICS_FEAT_KEYS, ILLUSION_ERR_KEYS, PHASE_STATE_KEYS, ILLUSION_PHASE_KEYS, KARMAN_EXPANDED_KEYS; obs dynamics (lag1 + derivative); error-state computation; 1D target_forces support |
| `src/SR_analysis/utils/sindy_fitter.py` | compute_action_deriv(); get_feature_matrix_deriv() with output_mode="deriv"|"absolute"; include_mu support |
| `src/SR_analysis/sindy/run_all_v2.py` | run_single_scene_deriv(); run_joint_karman_deriv(); --deriv, --phase, --karman-expand, --karman-mu, --output-mode, --augment-level CLI |
| `src/SR_analysis/validate/run_closed_loop.py` | predict_v23_deriv() with output_mode; mode="abs" branch; load_sindy_coefs returns "mode" |
| `src/SR_analysis/validate/run_closed_loop_illusion.py` | Support predict_v23_deriv; auto-detect SINDy mode from coefs |
| `src/SR_analysis/utils/__init__.py` | Export new constants and functions |
| `src/SR_analysis/sindy/wrap_joint.py` | Parameterized for karman/illusion |
| `src/SR_analysis/validate/eval_rollout.py` | **New**: offline multi-step rollout evaluation |
### Key Design Principles
1. **No action history in features** for all new routes (PHASE_STATE_KEYS, ILLUSION_PHASE_KEYS)
2. **Time-normalized derivatives** for cross-scene compatibility: `dx/dt = (x(t) - x(t-1)) / dt_c`
3. **Error-state** for Illusion: Cd_err, Cl_err encode current-to-target deviation
4. **Output flexibility**: both derivative and absolute modes supported
---
## 6. Figures
All figures in `docs/figures/SR_analysis/`:
- **fig1_illusion_comparison.png**: Illusion 3-scene comparison bar chart
- **fig2_karman_ablation.png**: Karman re100 ablation across 7 configurations
- **fig3_karman_generalization.png**: Karman cross-Re generalization (training vs unseen)
- **fig4_r2_vs_closedloop.png**: One-step R2 vs closed-loop paradox scatter
- **fig5_illusion_coefficients.png**: Illusion 1L front feature coefficients
- **fig6_roadmap.png**: Research progress roadmap
docs/ccd_correction_field_report.md
-412
View File
@@ -1,412 +0,0 @@
# CCD Analysis Report: Correction-Field Decomposition of Illusion Control
> **What this report is**: A self-contained summary of the CCD (Canonical Correlation Decomposition) analysis pipeline applied to the fluidic pinball illusion control problem. It assumes no prior knowledge of CCD or the project details — everything is explained from the ground up.
>
> **What this report is NOT**: A complete physics investigation. It is a progress report documenting what analysis was done, what was found, what it means, and where the open questions are.
---
## 1. The Problem in Plain Language
### 1.1 The physical system
Imagine three identical cylinders arranged in a triangle pointing upstream, placed in a channel with water flowing past them. Each cylinder can spin independently at a controlled speed. When the cylinders do NOT spin, the flow behind them forms a chaotic oscillating wake (a "von Karman vortex street").
The DRL controller can rotate the three cylinders at different speeds to change this wake. The goal of **illusion control** is: "make the flow field downstream of the three cylinders look like the flow field that would be produced by a single cylinder of a different size."
We test three target sizes: a cylinder of diameter 0.75 (smaller than the pinball cylinders), 1.0 (same size), and 1.5 (larger).
### 1.2 What the controller sees and does
- **Observations (input to controller)**: The forces on each cylinder (drag and lift) + the flow velocity measured at 3 points downstream
- **Actions (output of controller)**: 3 rotation speeds (one per cylinder), updated every 800 simulation timesteps
- **Reward (what the controller is trained to maximise)**: How closely the downstream sensors match the target cylinder's signal, plus how closely the total forces on the pinball match the target cylinder's forces
### 1.3 The key insight that changed everything
The naive approach is to ask: "does the controlled flow look like the target flow?" But this is not what the controller does. The controller works by **modifying the existing pinball wake**. A better question is: "what extra change does the controller add on top of the uncontrolled pinball wake, and does that change look like the change needed to transform the pinball wake into the target wake?"
This leads to the **correction-field framework**:
| Symbol | Meaning | How to think of it |
|--------|---------|-------------------|
| `q_in` | Clean channel flow (no pinball) | The baseline |
| `q_blk` | Pinball, no rotation | What the pinball does to the flow by its mere presence |
| `q_ctl` | Pinball with DRL control | The controlled flow |
| `q_tar` | The target cylinder alone | The flow we wish we had |
| `dq_blk = q_blk - q_in` | **Blockage field**: how the pinball disturbs the channel | Pinball's "mess" |
| `dq_ctl = q_ctl - q_blk` | **Correction field**: what control adds on top of the pinball | Controller's "fix" |
| `dq_tar = q_tar - q_blk` | **Target correction**: what change would turn pinball into target | The required "fix" |
The main question becomes: does `dq_ctl` (the actual fix) look like `dq_tar` (the required fix)?
---
## 2. What is CCD? (For the Non-Expert)
### 2.1 The core idea
**Proper Orthogonal Decomposition (POD)** finds the flow patterns that contain the most energy. It answers: "what are the dominant oscillating structures in this flow?"
**Canonical Correlation Decomposition (CCD)** finds the flow patterns that are most correlated with a specific quantity you care about (an "observable"). It answers: "what flow structures most determine the force on the cylinders?" or "what flow structures most determine the downstream sensor reading?"
The difference is crucial. Imagine a jet engine: the most energetic flow structures might be in the turbulent exhaust, but the structures that generate noise might be much weaker and completely different. POD would miss them because it ranks by energy, not by relevance to noise.
### 2.2 How CCD works (simplified)
1. **Take snapshots**: Record 96 velocity field snapshots of `dq_ctl` at evenly spaced times over 4 vortex shedding cycles
2. **Build a reference basis**: Use POD to find the main energy-containing structures in the TARGET correction field `dq_tar` — this gives us a coordinate system defined by what the target looks like
3. **Project into this basis**: Express the CONTROLLED correction field `dq_ctl` in terms of the target's structures
4. **Pick an observable**: Choose something we care about — the total lift force (`SigmaFy`), the cylinder rotation speeds (`action`), or the future sensor error (`signature`)
5. **Find the correlated patterns**: CCD finds the directions in flow-structure-space that best predict/correlate with the observable
6. **Measure compactness (m80)**: How many such directions do we need to capture 80% of the correlation? m80=1 means a single flow pattern explains most of the observable. m80=4 means we need more patterns.
7. **Measure overlap (O_k)**: Do two cases (e.g., target vs illusion) use the same flow patterns to generate the observable? O=1 means identical, O=0 means completely different.
### 2.3 Validation: how do we know CCD is meaningful?
We use **Leave-One-Cycle-Out (LOCO) cross-validation**:
- We have 4 shedding cycles of data
- Train CCD on 3 cycles, predict the observable on the held-out 1 cycle
- Compute R2 (how well the prediction matches reality)
- Repeat for each cycle as the held-out set
- If R2 > 0.4-0.5, the CCD patterns are stable and predictive
---
## 3. Data Quality and Preprocessing
Before any analysis, the raw flow fields must be phase-aligned — each snapshot corresponds to the same phase in the vortex shedding cycle across all cases. This is done by detecting the dominant shedding frequency, finding cycle boundaries, and extracting 24 evenly-spaced snapshots per cycle for 4 cycles = 96 snapshots total.
All cases pass quality gates:
- **Strict gate**: cycle-to-cycle period variation (CV_T) < 10%
- **Relaxed gate**: CV_T < 12%
| Case | Gate | Points/cycle | Interpolation factor | Strouhal |
|------|------|-------------|---------------------|----------|
| target_cylinder 0.75L | strict | 30.0 | 0.80 | 0.128 |
| target_cylinder 1.0L | strict | 24.8 | 0.97 | 0.133 |
| target_cylinder 1.5L | strict | 25.8 | 0.93 | 0.143 |
| illusion 0.75L | strict | 29.9 | 0.80 | — |
| illusion 1.0L | strict | 24.5 | 0.98 | — |
| illusion 1.5L | strict | 24.2 | 0.99 | — |
| pinball (uncontrolled) | strict | 21.4 | 1.12 | 0.113 |
The interpolation factor (rho) indicates how close each case is to having an integer number of snapshots per cycle. rho=1 is perfect; all cases have rho <= 1.12, meaning almost no interpolation artifacts.
---
## 4. The Five Analysis Lines
We answer five questions, each requiring a different observable for CCD:
| Line | Observable | Question | Symbol |
|------|-----------|----------|--------|
| **Force line (primary)** | Total lift force | Which correction structures most determine the lift? | `SigmaFy` |
| **Force line (secondary)** | Total drag force | Which correction structures most determine the drag? | `SigmaFx` |
| **Action line** | 3 cylinder rotation speeds | Which correction structures does the controller directly modulate? | `[Omega1, Omega2, Omega3]` |
| **Signature line** | Future sensor error | Which correction structures most determine whether downstream sensors will match the target? | `e_s(t+tau) = s_ctl(t+tau) - s_tar(t+tau)` |
The **signature line** has an extra dimension: time delay tau. We try tau=0 (instantaneous sensor error) and tau=tau_c (the time it takes for flow structures to convect from the pinball to the sensors, about 3-4 simulation steps).
---
## 5. Master Results
### 5.1 One-sentence summary per diameter
| Diameter | Summary |
|----------|---------|
| **1.0L** | The controller's correction direction is **nearly identical to the target's required correction** (O=0.913), and a single flow pattern captures 80% of the force-relevant correction. This is the "natural scale" case. |
| **0.75L** | The controller's correction only **partially aligns** with the target's required correction (O=0.564). The force and sensor-error structures are strongly separated in space at any instant, but converge after convective propagation. |
| **1.5L** | This is **not a failure but a special mechanism**. The controller achieves 94% sensor similarity using a qualitatively different strategy: the cylinder rotation commands have very weak correlation with the target-basis structures (action sigma1 = 0.28 vs 1.1-1.4), the correction energy is concentrated near the cylinders, and the shedding phase drifts over time. |
### 5.2 The master table
All values at r=6 (6 POD modes retained for the reference basis), unless noted.
| Metric | 0.75L | 1.0L | 1.5L |
|--------|-------|------|------|
| **O(dqctl, dqtar) — how similar are the corrections?** (1=identical, 0=orthogonal) | **0.564** | **0.913** | **0.667** |
| force_fy m80 — how many patterns needed for 80% of force? (lower = more concentrated) | 2 | 1 (at r=8/10) | 2 |
| action m80 — how many patterns for 80% of action correlation? | 2 | 3 | 3 |
| **action sigma1 — strength of action correlation** (higher = actions more tied to target-basis structures) | 1.39 | 1.13 | **0.28** |
| signature m80 (zero delay) | 3 | 3 | 2 |
| signature m80 (convective delay) | 2 | 3 | 2 |
| **O(force, sig) at zero lag** (1=force and sensor error use same structures) | **0.413** | **0.551** | — |
| **O(force, sig) at convective delay** | **0.806** | **0.768** | — |
| Phase drift (cycle-to-cycle period variation) | low | low | **high** |
| Body-wake KE / sensor-zone KE (higher = more correction energy near cylinders) | 0.73 | 1.17 | **2.58** |
### 5.3 Validation: can we trust these numbers?
**LOCO cross-validation R2 (m80 reconstruction, r=6):**
| Observable | 0.75L | 1.0L | Threshold | Verdict |
|-----------|-------|------|-----------|---------|
| force_fy (lift) | 0.65 +- 0.08 | 0.64 +- 0.02 | > 0.4 | PASS |
| force_fx (drag) | 0.38 +- 0.23 | 0.43 +- 0.11 | > 0.4 | WARNING |
| signature (zero lag) | 0.50 +- 0.09 | 0.49 +- 0.04 | > 0.4 | PASS |
| signature (convective delay) | 0.51 +- 0.09 | 0.53 +- 0.03 | > 0.4 | PASS |
The standard deviation across the 4 folds is small (0.02-0.09), meaning the patterns are stable across different data subsets. The **drag channel (force_fx) is unreliable** for detailed claims — its R2 is borderline and its variance is high. **All other channels pass validation.**
---
## 6. The Force vs Signature Separation: The Most Important Finding
### 6.1 Why this matters
One of the fundamental questions in flow control is: "are the flow structures that generate forces the same as the flow structures that determine what a downstream sensor sees?" If they are the SAME, then controlling the force automatically controls the sensor signal. If they are DIFFERENT, then the controller must manage two separate sets of structures.
**Our finding**: they are SEPARATED at any instant, but CONVERGE after the flow has time to convect downstream.
**Evidence — full-field CCD:**
| tau (convective delay in steps) | 0.75L O(force, sig) | 1.0L O(force, sig) |
|--|--|--|
| 0 (instantaneous) | **0.413** (separated) | **0.551** (partial) |
| ~3-4 (convective delay) | **0.806** (shared) | **0.768** (shared) |
At tau=0, the force-relevant and sensor-error-relevant structures share only 41-55% of their modal directions. After the flow convects downstream (tau=3-4), they share 77-81%. This makes physical sense: at any snapshot, the forces are determined by what is happening near the cylinders, while the sensor error is determined by what is further downstream. But after the near-body structures have had time to propagate downstream, they become the same thing.
### 6.2 Where does this separation happen spatially?
We divided the flow field into three zones and ran CCD separately in each:
| Zone | x-range (pixels) | What's there |
|------|-----------------|-------------|
| **near_body** | 350-500 | Around the cylinders (located at x=380-406) |
| **body_wake** | 500-700 | Just downstream of cylinders |
| **sensor_zone** | 580-650 | Where the velocity sensors measure the flow |
**0.75L — the separation is dramatic:**
| Zone | O(force, sig) at tau=0 | O(force, sig) at tau=tau_c |
|------|----------------------|---------------------------|
| near_body | 0.262 (separated) | 0.827 (shared) |
| body_wake | 0.269 (separated) | **0.917** (shared) |
| **sensor_zone** | **0.010 (NEARLY ORTHOGONAL)** | 0.722 (shared) |
In the sensor zone at zero lag, the force and signature structures are **effectively perpendicular** (O=0.01). This is the cleanest possible demonstration that force-relevant and sensor-error-relevant structures live in different spatial regions at any given instant. After the convective delay, the body_wake shows the strongest coupling (O=0.917), meaning the near-wake structures jointly determine future forces AND future sensor readings.
**1.0L — more uniform, less separation:**
| Zone | O(force, sig) at tau=0 | O(force, sig) at tau=tau_c |
|------|----------------------|---------------------------|
| near_body | 0.596 | 0.596 |
| body_wake | 0.509 | 0.483 |
| sensor_zone | 0.594 | **0.730** |
At the natural scale (1.0L), force and signature are more intrinsically linked across all zones. There is no zone with near-zero overlap. This makes sense: when the target shedding frequency matches the pinball's natural frequency, the same structures that produce forces are also those that the downstream sensors detect.
---
## 7. 1.5L Special Mechanism
The 1.5L case is not a failure (94.2% sensor similarity) but it operates differently:
1. **Action correlation is dramatically weaker**: sigma1 = 0.28 (vs 1.13-1.39 for other diameters). In the target's structural coordinate system, the cylinder rotation commands have very little explanatory power.
2. **Phase drift**: The shedding period varies significantly over time (CV_T across windows is high), unlike the stable periodic shedding of 0.75L and 1.0L.
3. **Correction energy concentrated near cylinders**: The KE ratio (body_wake / sensor_zone) = 2.58 (vs 0.73 for 0.75L, 1.17 for 1.0L). The controller is applying larger corrections near the cylinders, not just modifying the downstream wake.
4. **Signature coupling is stronger than force coupling**: The signature-line sigma1 (1.29) is greater than the force-line sigma1 (0.91). The correction field is more tightly tied to future sensor error than to instantaneous force.
---
## 8. Comparison: Correction Field vs Raw Field
Why go to the trouble of computing `dq_ctl = q_ctl - q_blk` instead of just working with `q_ctl` directly?
| Measure | Raw field (q_ctl) | Correction field (dq_ctl) | What it tells us |
|---------|------------------|--------------------------|-------------------|
| 1.0L O(target, illusion) | 0.919 | 0.913 | Similar — 1.0L is clean either way |
| 0.75L O(target, illusion) | 0.673 | **0.564** | Raw field was **contaminated** — ~16% of the apparent overlap was just baseline similarity |
| 1.0L force m80 | 2 | **1** | Correction field is more concentrated — the controller's ADDED structures are simpler than the full flow |
| LOCO R2 force_fy | 0.66-0.71 | 0.64-0.65 | Comparable — correction field doesn't degrade predictability |
**Conclusion**: Correction-field is the superior primary analysis object. The raw field can still be useful for historical comparison, but all mechanism claims should be based on correction-field analysis.
---
## 9. What We Learned About Steady Cloak
The steady cloak case (open-loop constant-speed rotation of the rear cylinders) was also analysed. The result: **it does not work well**. The RMS fluctuation suppression is essentially 0%, and the residual after cancellation is 81% of the original blockage. The downstream sensor region does better (13% residual) but that is mostly because the wake naturally recovers with distance.
This case is not suitable as a primary mechanism demonstration. A closed-loop steady cloak (using DRL) would be needed for meaningful analysis.
---
## 10. Limitations
1. **POD-reduced CCD**: All CCD results are constrained to the subspace spanned by the first 6-10 POD modes of the target correction field. If the controller uses structures that have very low energy in the target's natural basis, they will be truncated and invisible to CCD. For a "true" full-field CCD, significantly more data would be needed.
2. **The drag channel is unreliable**: force_fx (drag) fails LOCO validation (R2 ~0.4 with high variance). This is because the force reward in the DRL training matches drag statistics (mean, variance) but not the instantaneous waveform. Drag CCD results should only be used for O_k trend comparisons, not for mechanism claims.
3. **1.5L force-signature overlap not computed**: The force-vs-signature comparison could not be run for 1.5L because of technical issues with the cross-correlation computation. This is a gap that should be filled.
4. **Karman cloak not analysed**: The data for Karman cloak (vortex street incoming) has been collected but the analysis was deferred. The physical question is different (distortion compensation vs target generation) and the full pipeline is ready for when this becomes a priority.
---
## 11. Available Figures and How to Read Them
### Figure 1: Force sanity check — `sanity_force_{diam}L.png`
**Three files**: one per diameter.
**What it shows**: The raw force time series comparison between the target cylinder (red) and the controlled pinball illusion (blue). Four panels per figure:
- Top-left: Total drag force Fx over time
- Top-right: Total lift force Fy over time
- Bottom-left: Fx scatter plot (target vs illusion), with correlation r annotated
- Bottom-right: Fy scatter plot, with correlation r annotated
**How to read it**: The diagonal dashed line in the scatter plots indicates perfect tracking. For 1.0L, the Fy scatter (bottom-right) shows points clustering near the diagonal with r=0.82 — the controller tracks the lift waveform. For 1.5L, the Fy scatter shows r=-0.30 — the lift is negatively correlated, meaning the controller is doing something fundamentally different. The Fx scatter for all diameters shows r near zero — the controller matches mean drag but not the drag waveform.
**Look at**: The Fy scatter correlation coefficients. 0.75L: r=0.38 (weak positive), 1.0L: r=0.82 (strong positive), 1.5L: r=-0.30 (negative!).
---
### Figure 2: Correction field comparison — `corr_illusion_{diam}L_ctl_vs_tar.png`
**Three files**: one per diameter.
**What it shows**: A 2x2 panel comparing the controller's correction (`dq_ctl`, left column) with the target's required correction (`dq_tar`, right column).
- Top row: Mean streamwise velocity (ux) of the correction field
- Bottom row: RMS magnitude of the correction field
**How to read it**: Look at the spatial patterns in the mean ux panels (top row). If the left and right panels look similar in structure (red/blue pattern), the controller is adding a correction that resembles what the target requires. For 1.0L, they look nearly identical. For 0.75L, there are similarities but also clear differences in the wake region. The RMS panels (bottom row) show where the fluctuations are — bright regions indicate high unsteadiness in the correction.
**Look at**: How similar the top-left and top-right panels are. The more similar, the more the controller's correction "knows" what the target needs.
---
### Figure 3: Correction field maps — `corr_illusion_{diam}L_dq_ctl_(control_correction).png`
**Three files**: one per diameter, plus similar files for `dq_blk` and vorticity.
**What it shows**: Three panels of the `dq_ctl` correction field:
- Left: Mean streamwise velocity (ux)
- Centre: Mean cross-stream velocity (uy)
- Right: RMS magnitude
**How to read it**: Red in the ux panel means the controller is ACCELERATING the flow at that point; blue means DECELERATING. The RMS panel shows where the control is most unsteady. The pinball cylinder positions are at approximately x=380-406 (visible as blank regions).
**Look at**: The ux panel — where does the controller add positive (red) vs negative (blue) momentum? For 0.75L and 1.0L, there is a strong dipole pair in the wake. For 1.5L, the pattern is shifted and the amplitude is larger.
---
### Figure 4: CCD mode 1 — `ccd_mode1_fy_{diam}L_{target,illusion}.png`
**Four files**: 2 diameters x 2 cases (target and illusion).
**What it shows**: The first (most important) CCD mode for the force-fy line (lift), expressed as a velocity field. Left panel = ux component, right panel = uy component. Red = positive, blue = negative.
**How to read it**: This is the single flow pattern that is most correlated with the lift force. If the target and illusion panels look similar, it means the controller is using the same kind of flow pattern to generate lift as the target cylinder naturally uses.
**Look at**: Compare the 1.0L target mode with the 1.0L illusion mode — they should look very similar (consistent with O=0.913). Compare 0.75L target with 0.75L illusion — more differences expected (O=0.564).
---
### Figure 5: POD phase portraits — `pod_phase_portraits_target_basis.png`
**One file, three panels** (0.75L, 1.0L, 1.5L).
**What it shows**: The scatter of the first two POD coefficients (a1, a2) in the target-only basis. Red dots = target cylinder, blue dots = illusion (controlled), green dots = pinball (uncontrolled).
**How to read it**: Each dot represents one snapshot (96 per case). The spread of dots shows the "attractor" — the region of flow state space occupied by each case. If the blue dots overlap with the red dots, the illusion dynamics are similar to the target dynamics. If the blue dots are in a completely different region (like 1.5L), the controller is operating in a different dynamical regime.
**Look at**: For 1.0L, blue (illusion) should largely overlap with red (target) and be separated from green (pinball). For 0.75L, the separation is smaller. For 1.5L, the pattern may look different entirely.
---
### Figure 6: Overlap heatmap — `Ok_heatmap_fy_r6.png`
**One file, at r=6 POD rank**.
**What it shows**: A 3x3 heatmap with columns = diameters (0.75L, 1.0L, 1.5L) and rows = comparison pairs (target-illusion, target-pinball, illusion-pinball). Colour = O_1 (the modal overlap of the first CCD mode).
**How to read it**: Each cell tells you how similar two cases are in their force-relevant flow structures. Dark cells (values near 0.9) mean the two cases use nearly identical lift-generating structures. Light cells (values near 0.2) mean they use very different structures.
**Look at**: The top row (target-illusion overlap) across diameters. For 0.75L: ~0.67, for 1.0L: ~0.92, for 1.5L: ~0.62. The progression shows the controller's force strategy diverging from the target's as the target size moves away from the pinball's natural scale.
---
### Figure 7: Cross-diameter overlap — `cross_diameter_overlap_fy.png`
**One file**.
**What it shows**: A 3x3 heatmap showing how similar the illusion's force-CCD direction is between different diameters, when all are projected into the 1.0L target-only POD basis.
**How to read it**: Each cell shows O(diameter_i, diameter_j) — how aligned the force-relevant structures are between illusions at different target sizes. All values along the diagonal are 1.0 (a case is identical to itself). Off-diagonal values show cross-diameter similarity.
**Look at**: The O(0.75L, 1.0L) = ~0.85, O(0.75L, 1.5L) = ~0.96, O(1.0L, 1.5L) = ~0.92. Interestingly, the two "off-natural-scale" cases (0.75L and 1.5L) are MORE similar to each other in the 1.0L basis than either is to 1.0L. This suggests they use a similar "deviant" strategy.
---
### Figure 8: z_1 verification — `z1_verification_fy_{diam}L.png`
**Two files**: 0.75L and 1.0L.
**What it shows**: The temporal coefficient of the first CCD mode (z_1, blue) overlaid with the normalised total lift force (red). Top panel: raw z_1(t). Bottom panel: both signals normalised and overlaid.
**How to read it**: If the blue and red lines track each other well in the bottom panel, the CCD mode is successfully capturing the lift-related structures. This is a sanity check — it shows that CCD found something real.
**Look at**: The overlap between the blue dashed and red solid lines in the bottom panel. Good tracking = CCD is working correctly.
---
### Figure 9: 1.5L special diagnostics (3 files)
- **`15L_raw_timeseries.png`**: Raw sensor, force, and action time series for 1.5L. Shows sensor tracking (how well illusion = target for sensors), force comparison, and the DRL action signals.
- **`15L_windowed_periodicity.png`**: The cycle-to-cycle period variation (CV_T) over time for 1.5L. If CV_T exceeds the dashed lines, the shedding is not perfectly periodic. This confirms the "phase drift" behaviour.
- **`15L_overlap_summary.png`**: A bar chart comparing O(target, illusion) across diameters. The 1.5L bar is annotated as "special mechanism."
**How to read the periodicity figure**: The top panel shows CV_T over time — values below 0.10 (red dashed line) indicate stable periodic shedding. If values frequently exceed this, the shedding period is drifting. The middle panel shows the cycle period itself. The bottom panel shows the dominant frequency. Together, they reveal whether the flow is stably periodic or drifting.
---
### Figure 10: Steady cloak cancel test — `steady_cloak_cancel_test.png`
**One file**.
**What it shows**: Three panels comparing `dq_blk` (the blockage field — pinball's disturbance), `dq_ctl` (the control correction), and `dq_ctl + dq_blk` (the residual — what's left after control tries to cancel blockage).
**How to read it**: If the control perfectly cancels the blockage, the right panel (dq_ctl + dq_blk) should be near zero everywhere. Blue/red patterns in the right panel indicate incomplete cancellation. The presence of strong colour shows the control does not fully restore the flow.
**Look at**: The third panel — if it's mostly blank (near zero), the cancellation is working well. For this case, it is NOT blank, confirming the open-loop steady cloak does not effectively cancel the pinball disturbance.
---
## 12. Summary of Conclusions
1. **The correction-field framework is the correct way to analyse this problem**. It isolates what the controller actually changes, removing baseline similarity contamination.
2. **1.0L illusion is a low-rank, target-aligned correction**. O(dqctl, dqtar)=0.913, m80=1. When the target matches the pinball's natural scale, the controller modulates the existing shedding channel in a near-optimal way.
3. **Force and signature structures are spatially separated at zero lag but converge after convective delay**. This is confirmed by both full-field CCD (O=0.41-0.55 at tau=0, rising to 0.77-0.81 at tau=tau_c) and by zone-restricted CCD (sensor zone shows O=0.01 for 0.75L at tau=0).
4. **1.5L is a genuine special mechanism**, not a failure. It achieves 94% sensor similarity despite weak action coupling, strong phase drift, and a near-body-focused correction pattern.
5. **The drag channel (force_fx) is unreliable for mechanism claims**. It fails validation and should only be used for trend comparisons.
6. **The open-loop steady cloak is ineffective** (0% fluctuation suppression) and should not be a focus for mechanism analysis.
---
## 13. Data and Code Availability
All analysis scripts: `src/CCD_analysis/correction_analysis/*.py`
All results: `src/CCD_analysis/data/ccd/*.json`
All figures: `src/CCD_analysis/data/figures/*.png`
81 total figures, 8 JSON result files
Key result files:
- `ccd_results.json` — raw-field CCD (Round 5 baseline)
- `correction_ccd_results.json` — correction-field force/action CCD
- `correction_validation_results.json` — LOCO validation
- `signature_ccd_results.json` — signature-line CCD
- `15L_correction_results.json` — 1.5L analysis
- `zone_ccd_results.json` — zone-restricted CCD
- `steady_metrics.json` — steady cloak quantitative metrics
docs/ccd_handover.md
-74
View File
@@ -1,74 +0,0 @@
# CCD Direction Handover
## Agent Background
This agent worked on the CCD (Canonical Correlation Decomposition) analysis pipeline for the DynamisLab fluidic pinball project. The work spanned approximately 7-8 hours over 2026-06-14/15, covering Round 5 (raw-field baseline) and Round 6 (correction-field framework).
## Work Summary
### What was accomplished
1. **Data pipeline overhaul**: Replaced the old `resampled.npz` (interpolated) format with `fields_aligned.npz` (96 non-interpolated raw field snapshots) + `phase_plan.json`. Implemented `load_aligned_fields()` in `utils/resampling.py` as the unified data loader.
2. **Raw-field CCD baseline (Round 5)**: `ccd/run_ccd.py` and `ccd/validate.py` rewritten for the new data format. Target-only POD basis, per-force observable (SigmaFy primary), Q_delay=6, per-case z-score. 90 CCD entries, LOCO validation passed for force_fy (R2 0.66-0.71).
3. **Correction-field framework (Round 6)**: Shifted analysis object from `q_ctl` to `dq_ctl = q_ctl - q_blk` (the control correction field). Built `correction_analysis/compute_correction_fields.py` for unified q_in/q_blk/q_ctl/q_tar + dq_* field computation.
4. **Completed analyses in round 6**:
- Force/action CCD on dq_ctl (0.75L, 1.0L)
- Signature line CCD (0.75L, 1.0L) with tau scan (0, geom, corr)
- 1.5L force/action/signature CCD + phase drift diagnostics
- Steady cloak quantitative metrics
- Zone-restricted CCD (near_body, body_wake, sensor_zone) for 0.75L and 1.0L
- Snapshot POD speedup (SVD on 96x96 instead of 1310720x96)
5. **Karman reference data collected**: karman_q_in (vortex street without pinball) and karman_q_blk (pinball in vortex street, no control) — both 96 aligned frames.
6. **Documentation**:
- `docs/ccd_correction_field_report.md` — comprehensive 412-line report explaining everything from scratch, including 10-figure reading guide
- `docs/sr_ccd_oid_mapping.md` — cross-pipeline mapping (DRAFT - needs verification from SR and OID directions)
- `src/CCD_analysis/ccd_knowledge.md` — updated with final results
- `src/CCD_analysis/ccd_notes.md` — updated with completion status
### Key findings
1. **1.0L**: O(dqctl, dqtar)=0.913, force_fy m80=1 — the controller's correction nearly perfectly matches the target's required correction, and it's highly concentrated.
2. **Force vs Signature separation**: O(force,sig)=0.41-0.55 at tau=0 (separated), rising to 0.77-0.81 at tau=tau_c (shared). Zone-CCD shows 0.75L sensor_zone has O=0.01 at tau=0 (near orthogonal) and body_wake has O=0.917 at tau=tau_c.
3. **1.5L special mechanism**: O=0.667, action sigma1=0.28 (1/4 of others), strong phase drift, correction concentrated near-body.
### What is not done
1. **Karman cloak analysis** — data is ready (q_in, q_blk, q_ctl all have fields_aligned.npz), correction-field framework supports it, but analysis was deferred. Different physical question: distortion compensation vs target generation.
2. **1.5L force-vs-signature overlap** — 0.75L and 1.0L have O(force,sig) values, 1.5L has signature m80 but no overlap comparison.
3. **SR-CCD-OID mapping** — `docs/sr_ccd_oid_mapping.md` was written without reading SR and OID reports. Needs correction.
4. **Mixed-basis sensitivity** — deferred sensitivity check (currently target-only basis).
## Quick Start for Your First Commands
```bash
# Read the comprehensive report
less docs/ccd_correction_field_report.md
# Read the knowledge base
less src/CCD_analysis/ccd_knowledge.md
# Explore results
ls src/CCD_analysis/data/ccd/*.json
python3 -c "import json; r=json.load(open('src/CCD_analysis/data/ccd/correction_ccd_results.json')); print(f'{len(r)} entries'); [print(k) for k in list(r.keys())[:5]]"
# Check available figures
ls src/CCD_analysis/data/figures/*.png | wc -l
```
## Environment
- All CPU analysis: `conda run -n pycuda_3_10`
- GPU collection: same environment, devices 2 or 3
- LegacyCelerisLab (FlowField) needed for GPU scripts
- Python 3.10+, numpy, matplotlib, scipy (via conda)
- `sys.path.insert(0, 'src')` needed for imports
docs/sr_ccd_oid_mapping.md
-109
View File
@@ -1,109 +0,0 @@
# SR-CCD-OID Cross-Pipeline Mapping
## Purpose
This document maps the three analysis pipelines (SINDy-SR, CCD, OID) onto a unified chain. They are NOT competing approaches — they answer different questions at different positions along the control-to-signature pathway.
## Unified Control Analysis Chain
```
obs --[SR/SINDy]--> act --[CFD/physics]--> dq_ctl --[CCD/OID]--> force/signature
^ |
|_________________________________________________________________________|
closed loop
```
| Link | What happens | Which analysis |
|------|-------------|----------------|
| obs -> act | DRL policy maps sensor readings to control actions | **SR/SINDy** (white-box control law extraction) |
| act -> dq_ctl | Actions modify the flow field; the change relative to uncontrolled baseline is `dq_ctl` | CFD / data collection |
| dq_ctl -> force | Which correction structures most project to cylinder forces | **CCD** (force line), **OID** |
| dq_ctl -> signature | Which correction structures most determine future sensor mismatch | **CCD** (signature line), **OID** |
## Pipeline Comparison Table
| Aspect | SR / SINDy | CCD | OID / PCD |
|--------|-----------|-----|-----------|
| **Primary question** | How does the controller map observations to actions? | Which correction structures correlate most with force/action/signature? | What is the unified low-dimensional coordinate that captures observable-related structure? |
| **Input data** | Dimensionless obs and actions (time series) | `dq_ctl` fields (N snapshots x 2*NX*NY grid) + observable time series (force/action/sensor error) | POD coefficients of `dq_ctl` + observable time series |
| **Output** | Sparse symbolic control law (e.g. `a_F = 0.3*sin(u_s1)`) | CCD mode directions W, modal overlaps O_k, compactness m80, LOCO R2 | Low-dimensional coordinate z(t), observable reconstruction error |
| **Key method** | STLSQ threshold grid, G-equivariant constraints, SIN activation | POD-reduced CCD (Lyu23-inspired) | Observability Gramian / canonical correlation |
| **Current maturity** | Medium — cross-Re shared backbone found, G-equivariance validated | **Highest** — correction-field framework complete for illusion 0.75L/1.0L/1.5L with force/action/signature lines | Low-medium — framework defined, needs data alignment with CCD |
| **Validation** | Leave-one-Re-out cross-validation, closed-loop replay | LOCO (4-fold), blocked split, R2_m80 | pending alignment |
| **Key result** | Karman cloak cross-Re shared backbone exists (R2 > 0.9 for holdout 200) | 1.0L O(dqctl,dqtar)=0.913, m80=1; force/sig separated at tau=0, shared at tau_c | pending |
## Maturity by Scene
| Scene | SR/SINDy | CCD | OID |
|-------|----------|-----|-----|
| Karman cloak re50/100/200/400 | **Existing** (cross-Re backbone) | Data ready, analysis deferred | Not started |
| Illusion 0.75L | Existing | **Complete** (force/action/sig) | Partial |
| Illusion 1.0L | Existing | **Complete** (force/action/sig) | Partial |
| Illusion 1.5L | Existing | **Complete** (force/action/sig, special mechanism) | Not started |
| Steady cloak | Existing | Partial (quantitative metrics done) | Not started |
| Vortex cloak (lamb/taylor) | Existing | Not started | Not started |
## How They Assemble Into a Paper Chapter
### Chapter Structure Proposal
#### 1. Control Law Extraction (SR/SINDy)
- *Question*: What is the map from sensor observations to cylinder rotations?
- *Deliverable*: Symbolic control law for each scene, cross-scene comparison of feature usage
- *Evidence*: Leave-one-out validation, G-equivariance error < 10%
#### 2. Correction Field Analysis (CCD)
- *Question*: What flow structures does the controller actually modulate?
- *Deliverable*:
- Correction-field decomposition (`dq_ctl`)
- Force line: O(dqctl,dqtar) across diameters
- Action line: compactness m80
- Signature line: force-sig separation at zero lag, convergence at convective delay
- 1.5L special mechanism
- *Evidence*: LOCO validation R2 > 0.4 for all lines
#### 3. Low-Dimensional Coordinate (OID)
- *Question*: Can we describe controller-relevant structures in a unified low-D coordinate?
- *Deliverable*: Observable-informed coordinates z for each case, reconstruction error
- *Evidence*: Reconstruction quality vs POD-baseline
#### 4. Unified Mechanism Discussion
- Synthesize findings from all three analyses
- Key claims to support:
- Control operates by modifying pinball's existing wake (not generating new flows)
- Force-relevant correction is low-rank and target-aligned at natural scale
- Cross-scale illusion uses divergent correction paths
- Force and signature structures separate at zero lag but converge convectively
## Current Gaps by Pipeline
### SR/SINDy Gaps
- Illusion cross-diameter comparison not yet unified with CCD's correction-field framework
- Closed-loop validation of extracted control laws needs systematic comparison
### CCD Gaps
- Karman cloak analysis deferred (data ready, framework designed)
- Steady cloak needs closed-loop control to be meaningful
- Zone-restricted CCD not yet complete (in progress)
### OID Gaps
- Data pipeline not yet aligned with CCD's correction-field format
- No direct comparison of OID coordinates with CCD directions
- Requires full cross-analysis with existing CCD results
## Data Compatibility
All three pipelines ultimately read from the same data sources:
- `fields_aligned.npz` (96 aligned field snapshots)
- `controlled.npz` / `sensors.npz` (telemetry)
- `configs.py` (scene metadata)
The **correction-field framework** (`dq_ctl = q_ctl - q_blk`) is the standard analysis object across all three. Any analysis that uses raw `q_ctl` instead should be explicitly flagged as a cross-check.
## Recommendation
For the next phase of work:
1. **CCD** consolidates current results and adds zone-restricted analysis
2. **OID** should adopt CCD's data loading (`compute_correction_fields.py`) and correction-field protocol
3. **SR/SINDy** should align its cross-diameter comparison with CCD's correction-field O(dqctl,dqtar) results
4. A unifying figure comparing O(dqctl,dqtar) from CCD with SR control-law similarity across diameters would be powerful