Frank14f/DynamisLab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: project-wide cleanup — consolidate docs, remove obsolete code, update .gitignore

Browse Source 
- Remove obsolete docs (OID_handover, SR_analysis_results, ccd_* handover)
- Remove CCD legacy output_redux and old scripts
- Remove SR old sindy scripts and compare modules
- Update .gitignore to cover all analysis-generated outputs
- Retain all active code in OID/SR/CCD analysis directories

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Frank14f 2026-06-28 16:52:48 +08:00
parent c918ac0de4
commit 56e3c78a83
73 changed files with 1365 additions and 11072 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
.gitignore vendored
View File

@ -75,6 +75,17 @@ tensorboard/
*.h5 *.h5
*.hdf5 *.hdf5
*.npz *.npz
*.npy
# Analysis outputs (generated)
src/OID_analysis/data/derived/
src/OID_analysis/data/*/vorticity_*.png
src/OID_analysis/data/*/*/vorticity_*.png
src/OID_analysis/data/*/*/ddf_checkpoint.npy
src/OID_analysis/data/*/*/fifo_checkpoint.npy
src/CCD_analysis/data/
src/SR_analysis/data/*/*/fields.npz
outputs/
# Logs # Logs
*.log *.log

2
.gitmodules vendored
View File

@ -1,3 +1,5 @@
[submodule "CelerisLab"] [submodule "CelerisLab"]
path = CelerisLab path = CelerisLab
url = https://github.com/Frank14f/CelerisLab.git url = https://github.com/Frank14f/CelerisLab.git
ignore = dirty

81
docs/HANDOVER_SR_ANALYSIS.md
View File

@ -1,81 +0,0 @@
# SR Analysis Pipeline — Handover Notes (2026-06-15)
## 交接人 → 接手人
### 当前管线状态
| 模块 | 状态 | 说明 |
|------|:----:|------|
| Illusion 0.75L + 1L phase-state + abs | **已验证** | 闭环 0.974 / 0.958，无动作历史，可进 PySR |
| Illusion 1.5L | **边界 case** | bang-bang 机制，线性 SINDy 不适用 |
| Karman phase-state + abs | **0.699** | 优于 deriv 模式，但低于旧 v23 的 0.901 |
| Karman old v23 (a_lag) | **0.901** | 保留作基线对照 |
| Karman 泛化测试 | **已完成** | Re70/150/300/25 约 0.54-0.60 |
| PySR 符号回归 | **有 shell** | 需要修复 run_pysr.py 后重新跑 |
| Vortex 偏移扩展 | **未做** | 低优先级 |
### 核心文件改动（2026-06-14~15）
| 文件 | 改动类型 |
|------|----------|
| `utils/feature_builder.py` | **新增** PHASE_STATE_KEYS, ILLUSION_PHASE_KEYS, KARMAN_EXPANDED_KEYS, obs dynamics, error-state, mu_Cl_tot |
| `utils/sindy_fitter.py` | **新增** compute_action_deriv, get_feature_matrix_deriv(output_mode) |
| `sindy/run_all_v2.py` | **新增** --deriv, --phase, --karman-expand, --karman-mu, --output-mode, --augment-level CLI |
| `validate/run_closed_loop.py` | **新增** predict_v23_deriv, mode="abs", load_sindy_coefs返回mode |
| `validate/run_closed_loop_illusion.py` | **修改** 支持 predict_v23_deriv, 自动检测模式 |
| `validate/eval_rollout.py` | **新文件** 离线多步 rollout 评估 |
| `scripts/plot_sr_results.py` | **新文件** 结果可视化图表 |
| `docs/SR_analysis_results.md` | **新文件** 完整分析报告 |
| `docs/figures/SR_analysis/fig*.png` | **新文件** 6 张图表 |
### 关键设计决策（接手前必读）
1. **phase-state 特征** = `u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear` (6维)
2. **ILLUSION_PHASE_KEYS** = phase-state + `Cd_err, Cl_err, dCd_err/dt, dCl_err/dt` (10维)
3. **绝对动作输出** `output_mode="absolute"` 优于导数 `"deriv"`，无积分累积
4. **v23 结构**始终默认：front no-bias, rear shared-head
5. **时间一阶导**统一除以 `dt_c = SAMPLE_INTERVAL/2000`，跨场景可比较
6. `controlled.npz` 中新加了 `target_forces` 字段，illusion 场景必须有
7. **FIFO bias ≠ DRL action bias**，1U vs 2U 不要混淆
### 常用命令速查
```bash
# 拟合 + 验证 Illusion phase-state + absolute (完整流程)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes illusion_0.75L,illusion_1L --deriv --phase --output-mode absolute
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop_illusion.py \
--scene illusion_0.75L --device 0 --steps 320 \
--sindy-results src/SR_analysis/sindy/illusion/sindy_results_deriv.json
# 拟合 + 验证 Karman phase-state + absolute
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes karman_re100 --deriv --phase --output-mode absolute
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scene karman_re100 --device 0 --steps 200 --mode abs \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json
# 离线 rollout 评估
python3 src/SR_analysis/validate/eval_rollout.py \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json \
--scene karman_re100
# PySR (需要先修复滞后的 bug！)
conda run -n sr_env python src/SR_analysis/sindy/run_pysr.py --scene illusion_1L
# 画图
python3 scripts/plot_sr_results.py
```
### 目前最适合推进的方向
1. **Illusion PySR 符号回归**（0.75L + 1L separate → 公式比较）
2. **Karman 状态补强**（配合 CCD/OID 分析找出缺失的状态量，再回 SR）
3. **Karman 跨 Re 联合**（在 phase-state + mu 基础上做跨 Re 联合拟合 + 泛化）
### 注意重新运行 run_pysr.py
当前 `run_pysr.py` 有路径/导入问题，接手后需先确认：
- `env_sr` 或 `sr_env` 环境内的 PySR 可用性
- whitelist 特征与 `controlled.npz` 中的字段匹配（特别是 illusion 的 `target_forces`）
- 滞后构造正确（`a_prev[1:] = actions_phys[:-1]`）

136
docs/OID_handover.md
View File

@ -1,136 +0,0 @@
# OID Analysis Handover Notes
## For the incoming agent
---
## Quick Overview
You are taking over the **OID analysis line** of the DynamisLab fluidic pinball project. This is one of three parallel analysis pipelines:
- **SR/SINDy**: `obs -> act` white-box control law extraction (most mature)
- **CCD**: `structure -> force/signature` phase-aligned correlation decomposition (correction-field analysis in progress)
- **OID**: `Delta-q_ctl -> structure -> force/signature` full-time-series observable-related decomposition (this line)
OID has been fully implemented as an independent project under `src/OID_analysis/`. All 5 scenes (steady cloak, Karman cloak, illusion 0.75/1.0/1.5L) have been analyzed end-to-end through Phases 1-7.
---
## Start Here (in order)
1. **`docs/OID_analysis_results.md`** -- Full project report with 7 figures. Read this first. It explains OID concepts, all results, and all caveats.
2. **`src/OID_analysis/README.md`** -- Engineering entry point. How to run, directory structure, common pitfalls.
3. **`src/OID_analysis/OID_knowledge.md`** -- Confirmed facts, critical rules (15+ rules), current results, bug history.
4. **`src/OID_analysis/OID_notes.md`** -- Task tracking, open items.
5. **`docs/ccd_correction_field_report.md`** -- CCD report (sibling project, important cross-reference)
6. **`docs/SR_analysis_results.md`** -- SR report (control law white-box results)
---
## The Key Concept
OID (Observable-Inferred Decomposition) answers: "Which flow structures most affect a chosen **observable** (force, sensor error)?"
It does this by:
1. Starting with a POD basis (unified coordinate system, but ranked by energy not task relevance)
2. Computing cross-covariance between POD coefficients and the observable
3. SVD to find directions in POD space that best correlate with the observable
OID operates on **correction fields** Delta-q_ctl = q_ctl - q_blk (controlled field minus baseline zero-rotation field), NOT on raw controlled fields.
---
## The Flagship Result
Force-relevant and signature-relevant correction structures **systematically separate** across control tasks:
```
steady_cloak (+0.763) --> Karman (-0.034) --> illusion 0.75L (-0.082) --> illusion 1.0L (-0.495) --> illusion 1.5L (-0.932)
```
where the number is cosine similarity between force-OID mode 1 and signature-OID mode 1.
The monotonic trend from same-channel to strongly opposite is the project's most compelling new physical finding.
OID also consistently beats POD for predicting force and signature in all scenes.
---
## Key Files
| File | Purpose |
|------|---------|
| `src/OID_analysis/configs.py` | Scene definitions (12 scenes) |
| `src/OID_analysis/utils/analysis.py` | POD, OID, PCD, statistics (CPU, no GPU) |
| `src/OID_analysis/utils/cfd_interface.py` | Re-exports from CCD (GPU) |
| `src/OID_analysis/analysis/phase3_force_oid.py` | Force-OID implementation |
| `src/OID_analysis/analysis/phase4a_signature_oid.py` | Signature-OID implementation |
| `src/OID_analysis/analysis/phase4b_signature_pcd.py` | PCD whitened cross-correlation |
| `src/OID_analysis/analysis/robustness_analysis.py` | Rank/tau_c/window robustness |
| `src/OID_analysis/analysis/make_figures.py` | Generate all 7 figures |
| `src/OID_analysis/scripts/collect_illusion_qblk.py` | **Important**: illusion-position q_blk (separate from cloak) |
---
## Environments
```bash
# CFD data collection (GPU required, 2 GPUs available: 1 and 3)
conda run -n pycuda_3_10 python src/OID_analysis/scripts/...
# Analysis (CPU only)
conda run -n sr_env python3 src/OID_analysis/analysis/...
```
---
## Open Items (Priority Order)
### P0 - Should fix next
1. **Illusion 0.75L rank instability** (std=0.26 across r=6..16). Likely needs longer time series. Current 100 snapshots for POD may be insufficient.
2. **Karman future-signal R2~0** -- currently near zero. Consider reformulating the signature observable as phase-error instead of direct sensor error.
### P1 - Important but not blocking
3. **OID mode-to-field visualization** -- OID spatial modes are computed but not plotted. Would show whether force-sig separation maps to different physical regions (near-body vs downstream).
4. **causal-PCD** -- Need a separate action-PCD to get action-related z_act coordinates for the `obs -> z -> act` chain.
### P2 - Enhancement
5. **Cross-validation** across multiple independent rollouts
6. **Vortex scenes** extension (data collected in SR but not in OID)
7. **CCD cross-validation** of the force-sig separation trend using phase-aligned data
---
## Data to Keep / Not Commit
**Commit these** (small, reproducible):
- All `.py` files in `src/OID_analysis/`
- All `.md` files in `src/OID_analysis/`
- `docs/OID_analysis_results.md`
- JSON configs in `data/configs/`
**DO NOT commit** (large, regeneratable):
- All `.npz`, `.npy` in `data/`
- All `.png` in `data/derived/figures/`
- Check `.gitignore` for proper exclusion
---
## Context on SR and CCD
- **SR/SINDy** is the most mature line. Best result: Karman cross-Re unified backbone achieves 94.4% of PPO closed-loop performance. New phase-state features for illusion achieve 100.2% of PPO with zero action memory. The SR agent wrote a detailed README at `src/SR_analysis/README.md`.
- **CCD** is in correction-field transition. Round 5 (raw-field baseline) is frozen. Round 6 (correction-field) Phase 1-2 complete. The key CCD finding that cross-validates OID: zone-restricted analysis shows force-sig structures are almost orthogonal in the sensor zone at zero lag (O=0.01) but converge after convective delay (O=0.72-0.92).
- The **correction-field framework** (Delta-q_ctl) is the shared analysis object across all three lines. All analyses should use it.
---
## Final Notes
- The OID analysis code is self-contained. It re-uses `CCD_analysis.utils.cfd_interface` for GPU operations but has its own analysis utilities and configs.
- The `data_dir_for_scene()` function in `configs.py` is the single source of truth for all data paths. Never hardcode paths.
- The most likely next useful step is: (a) fix the 0.75L rank instability, (b) produce the mode-to-field plots, (c) feed OID coordinates into SR's SINDy framework for the `obs -> z -> act` test.

225
docs/SR_analysis_results.md
View File

@ -1,225 +0,0 @@
# SR Analysis: Phase-State SINDy Results Report
> Date: 2026-06-15
> Project: DynamisLab — Active hydrodynamic cloaking and illusion using DRL on a fluidic pinball.
> Analysis pipeline: SINDy (STLSQ) + feature engineering for interpretable control law extraction.
---
## Executive Summary
Three key findings from the SR analysis pipeline:
1. **Illusion 0.75L + 1L: New route validated.** Phase-state features + error-state + absolute action output achieves closed-loop similarity of **0.96+** (97% of PPO) with **zero action history features**. This proves that physically meaningful control laws can be extracted without relying on action memory.
2. **Illusion 1.5L: Regime shift identified.** The 1.5L target exhibits bang-bang/saturated control (alpha range [-8, 8], autocorrelation r=0.07). Linear SINDy is fundamentally inadequate (R2=0.12). This is a regime boundary, not a modelling failure.
3. **Karman: State representation still incomplete.** The same phase-state approach reaches 0.699 (vs 0.901 baseline with action history). The problem is not the output form (derivative vs absolute) but insufficient input state information for the Karman scene.
---
## 1. Methodology
### 1.1 Feature Architecture (Final)
Three feature levels were tested for fitting `alpha = f(state)` or `d(alpha)/dt = g(state)`:
| Level | Features | Dim | Description |
|-------|----------|:---:|-------------|
| Static | u_m, u_a, u_c, v_a, Cd_tot, Cd_rear, Cl_tot, Cl_diff | 8 | Current-step physics only, no memory |
| Phase-state | u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear (+error terms) | 6+4 | Oscillation phase + rate + drag feedback |
| Full-lag | Static + lag-1 of all 8 | 16 | Brute-force temporal context (baseline) |
### 1.2 Output Modes
Two output targets were compared:
- **Derivative mode**: predict d(alpha)/dt, then integrate `alpha(t) = alpha(t-1) + dt_c * dalpha/dt`
- **Absolute mode**: predict alpha directly, no integration needed
### 1.3 Evaluation Metrics
Models are evaluated on:
1. **One-step R2**: fit quality on training data
2. **Offline multi-step rollout**: 1/5/20/50 step recursive prediction on held-out PPO data
3. **CFD closed-loop**: full CFD environment with SINDy control law, DTW similarity vs target
---
## 2. Illusion Results
### 2.1 Three-Scene Summary
| Scene | S | Old v23 (a_lag) | New phase+error+abs | PPO baseline | % of PPO | Features | Action history? |
|------|:--:|:----------------:|:-------------------:|:------------:|:--------:|----------|:---------------:|
| 0.75L | 400 | 0.908 | **0.974** | 0.972 | 100.2% | 10-dim (ILLUSION_PHASE) | **No** |
| 1L | 600 | 0.962 | **0.958** | 0.973 | 98.5% | 10-dim (ILLUSION_PHASE) | **No** |
| 1.5L | 800 | 0.926 | **N/A** | 0.945 | — | — | Bang-bang regime |
**Key insight**: The 0.75L scene's new route **outperforms** the old v23 (0.974 vs 0.908), while 1L matches it within 1.5%. This definitively proves that "action history is not necessary."
### 2.2 Illusion 1L Front Coefficients (Phase-State, Absolute Output)
```
alpha_F = f(u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear, Cd_err, Cl_err, dCd_err/dt, dCl_err/dt)
R2 = 0.987
```
Key contributors (sorted by |coef|): Cd_rear > Cd_tot > Cd_err > Cl_tot > Cl_err > dCl_err/dt > dCd_err/dt
The formula is dominated by **drag-based feedback** (Cd_tot, Cd_rear, Cd_err), with the oscillation phase (u_a, Cl_tot) providing secondary modulation.
### 2.3 Illusion 1.5L — Regime Shift Evidence
| Metric | 1L | 1.5L | Ratio |
|--------|:--:|:----:|:-----:|
| Alpha range (front) | [-0.36, 1.11] | **[-8.0, 8.0]** | 10x |
| 1-step autocorrelation | 0.957 | **0.065** | — |
| d(alpha)/dt std | 0.12 | **6.07** | 50x |
| Linear SINDy R2 | 0.987 | **0.124** | — |
The 1.5L controller operates at saturation limits (maximum rotation speed), flipping between extremes. This is consistent with the frequency-doubling control strategy reported in the confirmation report.
---
## 3. Karman Results
### 3.1 Ablation Study (re100)
| Configuration | Feat | Output | R2 | Closed-loop | % of PPO |
|--------------|:----:|:-----:|:--:|:----------:|:--------:|
| old v23 (a_lag1 dominant) | 14+3 | alpha | 0.996 | **0.901** | 94.4% |
| **Phase-state -> abs (best new)** | **6** | **alpha** | **0.965** | **0.699** | 73.3% |
| Phase-state -> deriv | 6 | dalpha/dt | 0.837 | 0.656 | 68.8% |
| Phase-state + mu -> abs | 9 | alpha | 0.979 | 0.700 | 73.4% |
| Expanded 10-dim -> abs | 10 | alpha | 0.980 | **0.580** | 60.8% |
| Full-lag -> deriv | 16 | dalpha/dt | 0.939 | 0.619 | 64.9% |
| Static -> deriv | 8 | dalpha/dt | 0.321 | 0.745 | 78.1% |
**Key insight from ablation**:
1. **Phase-state + absolute output is the best new route** (0.699), but still well below the action-history baseline.
2. Adding extra static features (expanded 10-dim) **hurts** closed-loop despite higher R2 — classic covariate shift.
3. The static->deriv paradox: low R2 (0.321) but good closed-loop (0.745), because no-memory models are naturally robust to rollout divergence.
4. Mu modulation doesn't help at single-Re; its value will appear in cross-Re fitting.
### 3.2 Cross-Re Generalization (old v23 model)
| Re | Type | Closed-loop | Note |
|:--:|:----:|:----------:|------|
| 50 | Training | 0.582 | Low-frequency shedding |
| 100 | Training | **0.901** | Default / best |
| 200 | Training | 0.793 | Moderate degradation |
| 400 | Training | 0.664 | High-Re challenge |
| 25 | Unseen (subcritical) | 0.567 | Below Hopf bifurcation |
| 70 | Unseen | 0.577 | Between Re50 and Re100 |
| 150 | Unseen | 0.595 | Between Re100 and Re200 |
| 300 | Unseen | 0.541 | Outer extrapolation |
### 3.3 One-Step R2 vs Closed-Loop Paradox
A key phenomenon discovered during this work: **high one-step R2 does not predict good closed-loop performance** when temporal features (lags, derivatives) are present in the input. This is because:
- **Training**: features use ground-truth PPO observations
- **Deployment**: features use SINDY-controlled observations (distribution shift)
- Temporal features amplify this shift recursively
---
## 4. Roadmap to Next Steps
### 4.1 Illusion: Proceed to PySR
The 0.75L and 1L scenes are ready for symbolic regression. Recommended input:
```python
ILLUSION_PHASE_KEYS = [
"u_a", "du_a_dt", # oscillation phase
"Cl_tot", "dCl_tot_dt", # lift dynamics
"Cd_tot", "Cd_rear", # drag feedback
"Cd_err", "Cl_err", # force error
"dCd_err_dt", "dCl_err_dt", # error dynamics
]
```
Output: absolute alpha (non-dimensional action), no integration.
First do separate PySR for each scene, then compare formula structure for shared backbone.
### 4.2 Karman: CCD/OID for state completion
The 0.699 ceiling suggests missing state variables. Candidate directions:
- Recirculation zone length / reattachment point
- Wake centerline deflection
- POD mode coefficients for phase completion
- CCD modes correlated with control action
### 4.3 SR / CCD / OID Integration Framework
```
┌─────────────────────────────────────┐
│ Control Objective │
│ (stealth / illusion / erase) │
└──────────┬──────────────────────────┘
│
┌──────────▼──────────┐
│ DRL Policy │
│ (PPO + Sin act) │
└──────────┬──────────┘
│
┌────────────────┼────────────────┐
│ │ │
┌────────▼──────┐ ┌─────▼──────┐ ┌──────▼─────────┐
│ SINDy / SR │ │ CCD / OID │ │ Validation │
│ obs -> act │ │ obs -> z │ │ CFD closed- │
│ white-box │ │ structure │ │ loop + DTW │
│ control law │ │ analysis │ │ │
└────────┬──────┘ └─────┬──────┘ └──────┬─────────┘
│ │ │
└────────────────┼─────────────────┘
│
┌──────────▼──────────┐
│ Interpretable │
│ Control Mechanics │
│ obs -> z -> act │
│ -> structure -> │
│ -> signature │
└─────────────────────┘
```
---
## 5. Code Changes Summary
### Files Modified (8 core files)
| File | Changes |
|------|---------|
| `src/SR_analysis/utils/feature_builder.py` | PHYSICS_FEAT_KEYS, ILLUSION_ERR_KEYS, PHASE_STATE_KEYS, ILLUSION_PHASE_KEYS, KARMAN_EXPANDED_KEYS; obs dynamics (lag1 + derivative); error-state computation; 1D target_forces support |
| `src/SR_analysis/utils/sindy_fitter.py` | compute_action_deriv(); get_feature_matrix_deriv() with output_mode="deriv"|"absolute"; include_mu support |
| `src/SR_analysis/sindy/run_all_v2.py` | run_single_scene_deriv(); run_joint_karman_deriv(); --deriv, --phase, --karman-expand, --karman-mu, --output-mode, --augment-level CLI |
| `src/SR_analysis/validate/run_closed_loop.py` | predict_v23_deriv() with output_mode; mode="abs" branch; load_sindy_coefs returns "mode" |
| `src/SR_analysis/validate/run_closed_loop_illusion.py` | Support predict_v23_deriv; auto-detect SINDy mode from coefs |
| `src/SR_analysis/utils/__init__.py` | Export new constants and functions |
| `src/SR_analysis/sindy/wrap_joint.py` | Parameterized for karman/illusion |
| `src/SR_analysis/validate/eval_rollout.py` | **New**: offline multi-step rollout evaluation |
### Key Design Principles
1. **No action history in features** for all new routes (PHASE_STATE_KEYS, ILLUSION_PHASE_KEYS)
2. **Time-normalized derivatives** for cross-scene compatibility: `dx/dt = (x(t) - x(t-1)) / dt_c`
3. **Error-state** for Illusion: Cd_err, Cl_err encode current-to-target deviation
4. **Output flexibility**: both derivative and absolute modes supported
---
## 6. Figures
All figures in `docs/figures/SR_analysis/`:
- **fig1_illusion_comparison.png**: Illusion 3-scene comparison bar chart
- **fig2_karman_ablation.png**: Karman re100 ablation across 7 configurations
- **fig3_karman_generalization.png**: Karman cross-Re generalization (training vs unseen)
- **fig4_r2_vs_closedloop.png**: One-step R2 vs closed-loop paradox scatter
- **fig5_illusion_coefficients.png**: Illusion 1L front feature coefficients
- **fig6_roadmap.png**: Research progress roadmap

412
docs/ccd_correction_field_report.md
View File

@ -1,412 +0,0 @@
# CCD Analysis Report: Correction-Field Decomposition of Illusion Control
> **What this report is**: A self-contained summary of the CCD (Canonical Correlation Decomposition) analysis pipeline applied to the fluidic pinball illusion control problem. It assumes no prior knowledge of CCD or the project details — everything is explained from the ground up.
>
> **What this report is NOT**: A complete physics investigation. It is a progress report documenting what analysis was done, what was found, what it means, and where the open questions are.
---
## 1. The Problem in Plain Language
### 1.1 The physical system
Imagine three identical cylinders arranged in a triangle pointing upstream, placed in a channel with water flowing past them. Each cylinder can spin independently at a controlled speed. When the cylinders do NOT spin, the flow behind them forms a chaotic oscillating wake (a "von Karman vortex street").
The DRL controller can rotate the three cylinders at different speeds to change this wake. The goal of **illusion control** is: "make the flow field downstream of the three cylinders look like the flow field that would be produced by a single cylinder of a different size."
We test three target sizes: a cylinder of diameter 0.75 (smaller than the pinball cylinders), 1.0 (same size), and 1.5 (larger).
### 1.2 What the controller sees and does
- **Observations (input to controller)**: The forces on each cylinder (drag and lift) + the flow velocity measured at 3 points downstream
- **Actions (output of controller)**: 3 rotation speeds (one per cylinder), updated every 800 simulation timesteps
- **Reward (what the controller is trained to maximise)**: How closely the downstream sensors match the target cylinder's signal, plus how closely the total forces on the pinball match the target cylinder's forces
### 1.3 The key insight that changed everything
The naive approach is to ask: "does the controlled flow look like the target flow?" But this is not what the controller does. The controller works by **modifying the existing pinball wake**. A better question is: "what extra change does the controller add on top of the uncontrolled pinball wake, and does that change look like the change needed to transform the pinball wake into the target wake?"
This leads to the **correction-field framework**:
| Symbol | Meaning | How to think of it |
|--------|---------|-------------------|
| `q_in` | Clean channel flow (no pinball) | The baseline |
| `q_blk` | Pinball, no rotation | What the pinball does to the flow by its mere presence |
| `q_ctl` | Pinball with DRL control | The controlled flow |
| `q_tar` | The target cylinder alone | The flow we wish we had |
| `dq_blk = q_blk - q_in` | **Blockage field**: how the pinball disturbs the channel | Pinball's "mess" |
| `dq_ctl = q_ctl - q_blk` | **Correction field**: what control adds on top of the pinball | Controller's "fix" |
| `dq_tar = q_tar - q_blk` | **Target correction**: what change would turn pinball into target | The required "fix" |
The main question becomes: does `dq_ctl` (the actual fix) look like `dq_tar` (the required fix)?
---
## 2. What is CCD? (For the Non-Expert)
### 2.1 The core idea
**Proper Orthogonal Decomposition (POD)** finds the flow patterns that contain the most energy. It answers: "what are the dominant oscillating structures in this flow?"
**Canonical Correlation Decomposition (CCD)** finds the flow patterns that are most correlated with a specific quantity you care about (an "observable"). It answers: "what flow structures most determine the force on the cylinders?" or "what flow structures most determine the downstream sensor reading?"
The difference is crucial. Imagine a jet engine: the most energetic flow structures might be in the turbulent exhaust, but the structures that generate noise might be much weaker and completely different. POD would miss them because it ranks by energy, not by relevance to noise.
### 2.2 How CCD works (simplified)
1. **Take snapshots**: Record 96 velocity field snapshots of `dq_ctl` at evenly spaced times over 4 vortex shedding cycles
2. **Build a reference basis**: Use POD to find the main energy-containing structures in the TARGET correction field `dq_tar` — this gives us a coordinate system defined by what the target looks like
3. **Project into this basis**: Express the CONTROLLED correction field `dq_ctl` in terms of the target's structures
4. **Pick an observable**: Choose something we care about — the total lift force (`SigmaFy`), the cylinder rotation speeds (`action`), or the future sensor error (`signature`)
5. **Find the correlated patterns**: CCD finds the directions in flow-structure-space that best predict/correlate with the observable
6. **Measure compactness (m80)**: How many such directions do we need to capture 80% of the correlation? m80=1 means a single flow pattern explains most of the observable. m80=4 means we need more patterns.
7. **Measure overlap (O_k)**: Do two cases (e.g., target vs illusion) use the same flow patterns to generate the observable? O=1 means identical, O=0 means completely different.
### 2.3 Validation: how do we know CCD is meaningful?
We use **Leave-One-Cycle-Out (LOCO) cross-validation**:
- We have 4 shedding cycles of data
- Train CCD on 3 cycles, predict the observable on the held-out 1 cycle
- Compute R2 (how well the prediction matches reality)
- Repeat for each cycle as the held-out set
- If R2 > 0.4-0.5, the CCD patterns are stable and predictive
---
## 3. Data Quality and Preprocessing
Before any analysis, the raw flow fields must be phase-aligned — each snapshot corresponds to the same phase in the vortex shedding cycle across all cases. This is done by detecting the dominant shedding frequency, finding cycle boundaries, and extracting 24 evenly-spaced snapshots per cycle for 4 cycles = 96 snapshots total.
All cases pass quality gates:
- **Strict gate**: cycle-to-cycle period variation (CV_T) < 10%
- **Relaxed gate**: CV_T < 12%
| Case | Gate | Points/cycle | Interpolation factor | Strouhal |
|------|------|-------------|---------------------|----------|
| target_cylinder 0.75L | strict | 30.0 | 0.80 | 0.128 |
| target_cylinder 1.0L | strict | 24.8 | 0.97 | 0.133 |
| target_cylinder 1.5L | strict | 25.8 | 0.93 | 0.143 |
| illusion 0.75L | strict | 29.9 | 0.80 | — |
| illusion 1.0L | strict | 24.5 | 0.98 | — |
| illusion 1.5L | strict | 24.2 | 0.99 | — |
| pinball (uncontrolled) | strict | 21.4 | 1.12 | 0.113 |
The interpolation factor (rho) indicates how close each case is to having an integer number of snapshots per cycle. rho=1 is perfect; all cases have rho <= 1.12, meaning almost no interpolation artifacts.
---
## 4. The Five Analysis Lines
We answer five questions, each requiring a different observable for CCD:
| Line | Observable | Question | Symbol |
|------|-----------|----------|--------|
| **Force line (primary)** | Total lift force | Which correction structures most determine the lift? | `SigmaFy` |
| **Force line (secondary)** | Total drag force | Which correction structures most determine the drag? | `SigmaFx` |
| **Action line** | 3 cylinder rotation speeds | Which correction structures does the controller directly modulate? | `[Omega1, Omega2, Omega3]` |
| **Signature line** | Future sensor error | Which correction structures most determine whether downstream sensors will match the target? | `e_s(t+tau) = s_ctl(t+tau) - s_tar(t+tau)` |
The **signature line** has an extra dimension: time delay tau. We try tau=0 (instantaneous sensor error) and tau=tau_c (the time it takes for flow structures to convect from the pinball to the sensors, about 3-4 simulation steps).
---
## 5. Master Results
### 5.1 One-sentence summary per diameter
| Diameter | Summary |
|----------|---------|
| **1.0L** | The controller's correction direction is **nearly identical to the target's required correction** (O=0.913), and a single flow pattern captures 80% of the force-relevant correction. This is the "natural scale" case. |
| **0.75L** | The controller's correction only **partially aligns** with the target's required correction (O=0.564). The force and sensor-error structures are strongly separated in space at any instant, but converge after convective propagation. |
| **1.5L** | This is **not a failure but a special mechanism**. The controller achieves 94% sensor similarity using a qualitatively different strategy: the cylinder rotation commands have very weak correlation with the target-basis structures (action sigma1 = 0.28 vs 1.1-1.4), the correction energy is concentrated near the cylinders, and the shedding phase drifts over time. |
### 5.2 The master table
All values at r=6 (6 POD modes retained for the reference basis), unless noted.
| Metric | 0.75L | 1.0L | 1.5L |
|--------|-------|------|------|
| **O(dqctl, dqtar) — how similar are the corrections?** (1=identical, 0=orthogonal) | **0.564** | **0.913** | **0.667** |
| force_fy m80 — how many patterns needed for 80% of force? (lower = more concentrated) | 2 | 1 (at r=8/10) | 2 |
| action m80 — how many patterns for 80% of action correlation? | 2 | 3 | 3 |
| **action sigma1 — strength of action correlation** (higher = actions more tied to target-basis structures) | 1.39 | 1.13 | **0.28** |
| signature m80 (zero delay) | 3 | 3 | 2 |
| signature m80 (convective delay) | 2 | 3 | 2 |
| **O(force, sig) at zero lag** (1=force and sensor error use same structures) | **0.413** | **0.551** | — |
| **O(force, sig) at convective delay** | **0.806** | **0.768** | — |
| Phase drift (cycle-to-cycle period variation) | low | low | **high** |
| Body-wake KE / sensor-zone KE (higher = more correction energy near cylinders) | 0.73 | 1.17 | **2.58** |
### 5.3 Validation: can we trust these numbers?
**LOCO cross-validation R2 (m80 reconstruction, r=6):**
| Observable | 0.75L | 1.0L | Threshold | Verdict |
|-----------|-------|------|-----------|---------|
| force_fy (lift) | 0.65 +- 0.08 | 0.64 +- 0.02 | > 0.4 | PASS |
| force_fx (drag) | 0.38 +- 0.23 | 0.43 +- 0.11 | > 0.4 | WARNING |
| signature (zero lag) | 0.50 +- 0.09 | 0.49 +- 0.04 | > 0.4 | PASS |
| signature (convective delay) | 0.51 +- 0.09 | 0.53 +- 0.03 | > 0.4 | PASS |
The standard deviation across the 4 folds is small (0.02-0.09), meaning the patterns are stable across different data subsets. The **drag channel (force_fx) is unreliable** for detailed claims — its R2 is borderline and its variance is high. **All other channels pass validation.**
---
## 6. The Force vs Signature Separation: The Most Important Finding
### 6.1 Why this matters
One of the fundamental questions in flow control is: "are the flow structures that generate forces the same as the flow structures that determine what a downstream sensor sees?" If they are the SAME, then controlling the force automatically controls the sensor signal. If they are DIFFERENT, then the controller must manage two separate sets of structures.
**Our finding**: they are SEPARATED at any instant, but CONVERGE after the flow has time to convect downstream.
**Evidence — full-field CCD:**
| tau (convective delay in steps) | 0.75L O(force, sig) | 1.0L O(force, sig) |
|--|--|--|
| 0 (instantaneous) | **0.413** (separated) | **0.551** (partial) |
| ~3-4 (convective delay) | **0.806** (shared) | **0.768** (shared) |
At tau=0, the force-relevant and sensor-error-relevant structures share only 41-55% of their modal directions. After the flow convects downstream (tau=3-4), they share 77-81%. This makes physical sense: at any snapshot, the forces are determined by what is happening near the cylinders, while the sensor error is determined by what is further downstream. But after the near-body structures have had time to propagate downstream, they become the same thing.
### 6.2 Where does this separation happen spatially?
We divided the flow field into three zones and ran CCD separately in each:
| Zone | x-range (pixels) | What's there |
|------|-----------------|-------------|
| **near_body** | 350-500 | Around the cylinders (located at x=380-406) |
| **body_wake** | 500-700 | Just downstream of cylinders |
| **sensor_zone** | 580-650 | Where the velocity sensors measure the flow |
**0.75L — the separation is dramatic:**
| Zone | O(force, sig) at tau=0 | O(force, sig) at tau=tau_c |
|------|----------------------|---------------------------|
| near_body | 0.262 (separated) | 0.827 (shared) |
| body_wake | 0.269 (separated) | **0.917** (shared) |
| **sensor_zone** | **0.010 (NEARLY ORTHOGONAL)** | 0.722 (shared) |
In the sensor zone at zero lag, the force and signature structures are **effectively perpendicular** (O=0.01). This is the cleanest possible demonstration that force-relevant and sensor-error-relevant structures live in different spatial regions at any given instant. After the convective delay, the body_wake shows the strongest coupling (O=0.917), meaning the near-wake structures jointly determine future forces AND future sensor readings.
**1.0L — more uniform, less separation:**
| Zone | O(force, sig) at tau=0 | O(force, sig) at tau=tau_c |
|------|----------------------|---------------------------|
| near_body | 0.596 | 0.596 |
| body_wake | 0.509 | 0.483 |
| sensor_zone | 0.594 | **0.730** |
At the natural scale (1.0L), force and signature are more intrinsically linked across all zones. There is no zone with near-zero overlap. This makes sense: when the target shedding frequency matches the pinball's natural frequency, the same structures that produce forces are also those that the downstream sensors detect.
---
## 7. 1.5L Special Mechanism
The 1.5L case is not a failure (94.2% sensor similarity) but it operates differently:
1. **Action correlation is dramatically weaker**: sigma1 = 0.28 (vs 1.13-1.39 for other diameters). In the target's structural coordinate system, the cylinder rotation commands have very little explanatory power.
2. **Phase drift**: The shedding period varies significantly over time (CV_T across windows is high), unlike the stable periodic shedding of 0.75L and 1.0L.
3. **Correction energy concentrated near cylinders**: The KE ratio (body_wake / sensor_zone) = 2.58 (vs 0.73 for 0.75L, 1.17 for 1.0L). The controller is applying larger corrections near the cylinders, not just modifying the downstream wake.
4. **Signature coupling is stronger than force coupling**: The signature-line sigma1 (1.29) is greater than the force-line sigma1 (0.91). The correction field is more tightly tied to future sensor error than to instantaneous force.
---
## 8. Comparison: Correction Field vs Raw Field
Why go to the trouble of computing `dq_ctl = q_ctl - q_blk` instead of just working with `q_ctl` directly?
| Measure | Raw field (q_ctl) | Correction field (dq_ctl) | What it tells us |
|---------|------------------|--------------------------|-------------------|
| 1.0L O(target, illusion) | 0.919 | 0.913 | Similar — 1.0L is clean either way |
| 0.75L O(target, illusion) | 0.673 | **0.564** | Raw field was **contaminated** — ~16% of the apparent overlap was just baseline similarity |
| 1.0L force m80 | 2 | **1** | Correction field is more concentrated — the controller's ADDED structures are simpler than the full flow |
| LOCO R2 force_fy | 0.66-0.71 | 0.64-0.65 | Comparable — correction field doesn't degrade predictability |
**Conclusion**: Correction-field is the superior primary analysis object. The raw field can still be useful for historical comparison, but all mechanism claims should be based on correction-field analysis.
---
## 9. What We Learned About Steady Cloak
The steady cloak case (open-loop constant-speed rotation of the rear cylinders) was also analysed. The result: **it does not work well**. The RMS fluctuation suppression is essentially 0%, and the residual after cancellation is 81% of the original blockage. The downstream sensor region does better (13% residual) but that is mostly because the wake naturally recovers with distance.
This case is not suitable as a primary mechanism demonstration. A closed-loop steady cloak (using DRL) would be needed for meaningful analysis.
---
## 10. Limitations
1. **POD-reduced CCD**: All CCD results are constrained to the subspace spanned by the first 6-10 POD modes of the target correction field. If the controller uses structures that have very low energy in the target's natural basis, they will be truncated and invisible to CCD. For a "true" full-field CCD, significantly more data would be needed.
2. **The drag channel is unreliable**: force_fx (drag) fails LOCO validation (R2 ~0.4 with high variance). This is because the force reward in the DRL training matches drag statistics (mean, variance) but not the instantaneous waveform. Drag CCD results should only be used for O_k trend comparisons, not for mechanism claims.
3. **1.5L force-signature overlap not computed**: The force-vs-signature comparison could not be run for 1.5L because of technical issues with the cross-correlation computation. This is a gap that should be filled.
4. **Karman cloak not analysed**: The data for Karman cloak (vortex street incoming) has been collected but the analysis was deferred. The physical question is different (distortion compensation vs target generation) and the full pipeline is ready for when this becomes a priority.
---
## 11. Available Figures and How to Read Them
### Figure 1: Force sanity check — `sanity_force_{diam}L.png`
**Three files**: one per diameter.
**What it shows**: The raw force time series comparison between the target cylinder (red) and the controlled pinball illusion (blue). Four panels per figure:
- Top-left: Total drag force Fx over time
- Top-right: Total lift force Fy over time
- Bottom-left: Fx scatter plot (target vs illusion), with correlation r annotated
- Bottom-right: Fy scatter plot, with correlation r annotated
**How to read it**: The diagonal dashed line in the scatter plots indicates perfect tracking. For 1.0L, the Fy scatter (bottom-right) shows points clustering near the diagonal with r=0.82 — the controller tracks the lift waveform. For 1.5L, the Fy scatter shows r=-0.30 — the lift is negatively correlated, meaning the controller is doing something fundamentally different. The Fx scatter for all diameters shows r near zero — the controller matches mean drag but not the drag waveform.
**Look at**: The Fy scatter correlation coefficients. 0.75L: r=0.38 (weak positive), 1.0L: r=0.82 (strong positive), 1.5L: r=-0.30 (negative!).
---
### Figure 2: Correction field comparison — `corr_illusion_{diam}L_ctl_vs_tar.png`
**Three files**: one per diameter.
**What it shows**: A 2x2 panel comparing the controller's correction (`dq_ctl`, left column) with the target's required correction (`dq_tar`, right column).
- Top row: Mean streamwise velocity (ux) of the correction field
- Bottom row: RMS magnitude of the correction field
**How to read it**: Look at the spatial patterns in the mean ux panels (top row). If the left and right panels look similar in structure (red/blue pattern), the controller is adding a correction that resembles what the target requires. For 1.0L, they look nearly identical. For 0.75L, there are similarities but also clear differences in the wake region. The RMS panels (bottom row) show where the fluctuations are — bright regions indicate high unsteadiness in the correction.
**Look at**: How similar the top-left and top-right panels are. The more similar, the more the controller's correction "knows" what the target needs.
---
### Figure 3: Correction field maps — `corr_illusion_{diam}L_dq_ctl_(control_correction).png`
**Three files**: one per diameter, plus similar files for `dq_blk` and vorticity.
**What it shows**: Three panels of the `dq_ctl` correction field:
- Left: Mean streamwise velocity (ux)
- Centre: Mean cross-stream velocity (uy)
- Right: RMS magnitude
**How to read it**: Red in the ux panel means the controller is ACCELERATING the flow at that point; blue means DECELERATING. The RMS panel shows where the control is most unsteady. The pinball cylinder positions are at approximately x=380-406 (visible as blank regions).
**Look at**: The ux panel — where does the controller add positive (red) vs negative (blue) momentum? For 0.75L and 1.0L, there is a strong dipole pair in the wake. For 1.5L, the pattern is shifted and the amplitude is larger.
---
### Figure 4: CCD mode 1 — `ccd_mode1_fy_{diam}L_{target,illusion}.png`
**Four files**: 2 diameters x 2 cases (target and illusion).
**What it shows**: The first (most important) CCD mode for the force-fy line (lift), expressed as a velocity field. Left panel = ux component, right panel = uy component. Red = positive, blue = negative.
**How to read it**: This is the single flow pattern that is most correlated with the lift force. If the target and illusion panels look similar, it means the controller is using the same kind of flow pattern to generate lift as the target cylinder naturally uses.
**Look at**: Compare the 1.0L target mode with the 1.0L illusion mode — they should look very similar (consistent with O=0.913). Compare 0.75L target with 0.75L illusion — more differences expected (O=0.564).
---
### Figure 5: POD phase portraits — `pod_phase_portraits_target_basis.png`
**One file, three panels** (0.75L, 1.0L, 1.5L).
**What it shows**: The scatter of the first two POD coefficients (a1, a2) in the target-only basis. Red dots = target cylinder, blue dots = illusion (controlled), green dots = pinball (uncontrolled).
**How to read it**: Each dot represents one snapshot (96 per case). The spread of dots shows the "attractor" — the region of flow state space occupied by each case. If the blue dots overlap with the red dots, the illusion dynamics are similar to the target dynamics. If the blue dots are in a completely different region (like 1.5L), the controller is operating in a different dynamical regime.
**Look at**: For 1.0L, blue (illusion) should largely overlap with red (target) and be separated from green (pinball). For 0.75L, the separation is smaller. For 1.5L, the pattern may look different entirely.
---
### Figure 6: Overlap heatmap — `Ok_heatmap_fy_r6.png`
**One file, at r=6 POD rank**.
**What it shows**: A 3x3 heatmap with columns = diameters (0.75L, 1.0L, 1.5L) and rows = comparison pairs (target-illusion, target-pinball, illusion-pinball). Colour = O_1 (the modal overlap of the first CCD mode).
**How to read it**: Each cell tells you how similar two cases are in their force-relevant flow structures. Dark cells (values near 0.9) mean the two cases use nearly identical lift-generating structures. Light cells (values near 0.2) mean they use very different structures.
**Look at**: The top row (target-illusion overlap) across diameters. For 0.75L: ~0.67, for 1.0L: ~0.92, for 1.5L: ~0.62. The progression shows the controller's force strategy diverging from the target's as the target size moves away from the pinball's natural scale.
---
### Figure 7: Cross-diameter overlap — `cross_diameter_overlap_fy.png`
**One file**.
**What it shows**: A 3x3 heatmap showing how similar the illusion's force-CCD direction is between different diameters, when all are projected into the 1.0L target-only POD basis.
**How to read it**: Each cell shows O(diameter_i, diameter_j) — how aligned the force-relevant structures are between illusions at different target sizes. All values along the diagonal are 1.0 (a case is identical to itself). Off-diagonal values show cross-diameter similarity.
**Look at**: The O(0.75L, 1.0L) = ~0.85, O(0.75L, 1.5L) = ~0.96, O(1.0L, 1.5L) = ~0.92. Interestingly, the two "off-natural-scale" cases (0.75L and 1.5L) are MORE similar to each other in the 1.0L basis than either is to 1.0L. This suggests they use a similar "deviant" strategy.
---
### Figure 8: z_1 verification — `z1_verification_fy_{diam}L.png`
**Two files**: 0.75L and 1.0L.
**What it shows**: The temporal coefficient of the first CCD mode (z_1, blue) overlaid with the normalised total lift force (red). Top panel: raw z_1(t). Bottom panel: both signals normalised and overlaid.
**How to read it**: If the blue and red lines track each other well in the bottom panel, the CCD mode is successfully capturing the lift-related structures. This is a sanity check — it shows that CCD found something real.
**Look at**: The overlap between the blue dashed and red solid lines in the bottom panel. Good tracking = CCD is working correctly.
---
### Figure 9: 1.5L special diagnostics (3 files)
- **`15L_raw_timeseries.png`**: Raw sensor, force, and action time series for 1.5L. Shows sensor tracking (how well illusion = target for sensors), force comparison, and the DRL action signals.
- **`15L_windowed_periodicity.png`**: The cycle-to-cycle period variation (CV_T) over time for 1.5L. If CV_T exceeds the dashed lines, the shedding is not perfectly periodic. This confirms the "phase drift" behaviour.
- **`15L_overlap_summary.png`**: A bar chart comparing O(target, illusion) across diameters. The 1.5L bar is annotated as "special mechanism."
**How to read the periodicity figure**: The top panel shows CV_T over time — values below 0.10 (red dashed line) indicate stable periodic shedding. If values frequently exceed this, the shedding period is drifting. The middle panel shows the cycle period itself. The bottom panel shows the dominant frequency. Together, they reveal whether the flow is stably periodic or drifting.
---
### Figure 10: Steady cloak cancel test — `steady_cloak_cancel_test.png`
**One file**.
**What it shows**: Three panels comparing `dq_blk` (the blockage field — pinball's disturbance), `dq_ctl` (the control correction), and `dq_ctl + dq_blk` (the residual — what's left after control tries to cancel blockage).
**How to read it**: If the control perfectly cancels the blockage, the right panel (dq_ctl + dq_blk) should be near zero everywhere. Blue/red patterns in the right panel indicate incomplete cancellation. The presence of strong colour shows the control does not fully restore the flow.
**Look at**: The third panel — if it's mostly blank (near zero), the cancellation is working well. For this case, it is NOT blank, confirming the open-loop steady cloak does not effectively cancel the pinball disturbance.
---
## 12. Summary of Conclusions
1. **The correction-field framework is the correct way to analyse this problem**. It isolates what the controller actually changes, removing baseline similarity contamination.
2. **1.0L illusion is a low-rank, target-aligned correction**. O(dqctl, dqtar)=0.913, m80=1. When the target matches the pinball's natural scale, the controller modulates the existing shedding channel in a near-optimal way.
3. **Force and signature structures are spatially separated at zero lag but converge after convective delay**. This is confirmed by both full-field CCD (O=0.41-0.55 at tau=0, rising to 0.77-0.81 at tau=tau_c) and by zone-restricted CCD (sensor zone shows O=0.01 for 0.75L at tau=0).
4. **1.5L is a genuine special mechanism**, not a failure. It achieves 94% sensor similarity despite weak action coupling, strong phase drift, and a near-body-focused correction pattern.
5. **The drag channel (force_fx) is unreliable for mechanism claims**. It fails validation and should only be used for trend comparisons.
6. **The open-loop steady cloak is ineffective** (0% fluctuation suppression) and should not be a focus for mechanism analysis.
---
## 13. Data and Code Availability
All analysis scripts: `src/CCD_analysis/correction_analysis/*.py`
All results: `src/CCD_analysis/data/ccd/*.json`
All figures: `src/CCD_analysis/data/figures/*.png`
81 total figures, 8 JSON result files
Key result files:
- `ccd_results.json` — raw-field CCD (Round 5 baseline)
- `correction_ccd_results.json` — correction-field force/action CCD
- `correction_validation_results.json` — LOCO validation
- `signature_ccd_results.json` — signature-line CCD
- `15L_correction_results.json` — 1.5L analysis
- `zone_ccd_results.json` — zone-restricted CCD
- `steady_metrics.json` — steady cloak quantitative metrics

74
docs/ccd_handover.md
View File

@ -1,74 +0,0 @@
# CCD Direction Handover
## Agent Background
This agent worked on the CCD (Canonical Correlation Decomposition) analysis pipeline for the DynamisLab fluidic pinball project. The work spanned approximately 7-8 hours over 2026-06-14/15, covering Round 5 (raw-field baseline) and Round 6 (correction-field framework).
## Work Summary
### What was accomplished
1. **Data pipeline overhaul**: Replaced the old `resampled.npz` (interpolated) format with `fields_aligned.npz` (96 non-interpolated raw field snapshots) + `phase_plan.json`. Implemented `load_aligned_fields()` in `utils/resampling.py` as the unified data loader.
2. **Raw-field CCD baseline (Round 5)**: `ccd/run_ccd.py` and `ccd/validate.py` rewritten for the new data format. Target-only POD basis, per-force observable (SigmaFy primary), Q_delay=6, per-case z-score. 90 CCD entries, LOCO validation passed for force_fy (R2 0.66-0.71).
3. **Correction-field framework (Round 6)**: Shifted analysis object from `q_ctl` to `dq_ctl = q_ctl - q_blk` (the control correction field). Built `correction_analysis/compute_correction_fields.py` for unified q_in/q_blk/q_ctl/q_tar + dq_* field computation.
4. **Completed analyses in round 6**:
- Force/action CCD on dq_ctl (0.75L, 1.0L)
- Signature line CCD (0.75L, 1.0L) with tau scan (0, geom, corr)
- 1.5L force/action/signature CCD + phase drift diagnostics
- Steady cloak quantitative metrics
- Zone-restricted CCD (near_body, body_wake, sensor_zone) for 0.75L and 1.0L
- Snapshot POD speedup (SVD on 96x96 instead of 1310720x96)
5. **Karman reference data collected**: karman_q_in (vortex street without pinball) and karman_q_blk (pinball in vortex street, no control) — both 96 aligned frames.
6. **Documentation**:
- `docs/ccd_correction_field_report.md` — comprehensive 412-line report explaining everything from scratch, including 10-figure reading guide
- `docs/sr_ccd_oid_mapping.md` — cross-pipeline mapping (DRAFT - needs verification from SR and OID directions)
- `src/CCD_analysis/ccd_knowledge.md` — updated with final results
- `src/CCD_analysis/ccd_notes.md` — updated with completion status
### Key findings
1. **1.0L**: O(dqctl, dqtar)=0.913, force_fy m80=1 — the controller's correction nearly perfectly matches the target's required correction, and it's highly concentrated.
2. **Force vs Signature separation**: O(force,sig)=0.41-0.55 at tau=0 (separated), rising to 0.77-0.81 at tau=tau_c (shared). Zone-CCD shows 0.75L sensor_zone has O=0.01 at tau=0 (near orthogonal) and body_wake has O=0.917 at tau=tau_c.
3. **1.5L special mechanism**: O=0.667, action sigma1=0.28 (1/4 of others), strong phase drift, correction concentrated near-body.
### What is not done
1. **Karman cloak analysis** — data is ready (q_in, q_blk, q_ctl all have fields_aligned.npz), correction-field framework supports it, but analysis was deferred. Different physical question: distortion compensation vs target generation.
2. **1.5L force-vs-signature overlap** — 0.75L and 1.0L have O(force,sig) values, 1.5L has signature m80 but no overlap comparison.
3. **SR-CCD-OID mapping** — `docs/sr_ccd_oid_mapping.md` was written without reading SR and OID reports. Needs correction.
4. **Mixed-basis sensitivity** — deferred sensitivity check (currently target-only basis).
## Quick Start for Your First Commands
```bash
# Read the comprehensive report
less docs/ccd_correction_field_report.md
# Read the knowledge base
less src/CCD_analysis/ccd_knowledge.md
# Explore results
ls src/CCD_analysis/data/ccd/*.json
python3 -c "import json; r=json.load(open('src/CCD_analysis/data/ccd/correction_ccd_results.json')); print(f'{len(r)} entries'); [print(k) for k in list(r.keys())[:5]]"
# Check available figures
ls src/CCD_analysis/data/figures/*.png | wc -l
```
## Environment
- All CPU analysis: `conda run -n pycuda_3_10`
- GPU collection: same environment, devices 2 or 3
- LegacyCelerisLab (FlowField) needed for GPU scripts
- Python 3.10+, numpy, matplotlib, scipy (via conda)
- `sys.path.insert(0, 'src')` needed for imports

109
docs/sr_ccd_oid_mapping.md
View File

@ -1,109 +0,0 @@
# SR-CCD-OID Cross-Pipeline Mapping
## Purpose
This document maps the three analysis pipelines (SINDy-SR, CCD, OID) onto a unified chain. They are NOT competing approaches — they answer different questions at different positions along the control-to-signature pathway.
## Unified Control Analysis Chain
```
obs --[SR/SINDy]--> act --[CFD/physics]--> dq_ctl --[CCD/OID]--> force/signature
^ |
|_________________________________________________________________________|
closed loop
```
| Link | What happens | Which analysis |
|------|-------------|----------------|
| obs -> act | DRL policy maps sensor readings to control actions | **SR/SINDy** (white-box control law extraction) |
| act -> dq_ctl | Actions modify the flow field; the change relative to uncontrolled baseline is `dq_ctl` | CFD / data collection |
| dq_ctl -> force | Which correction structures most project to cylinder forces | **CCD** (force line), **OID** |
| dq_ctl -> signature | Which correction structures most determine future sensor mismatch | **CCD** (signature line), **OID** |
## Pipeline Comparison Table
| Aspect | SR / SINDy | CCD | OID / PCD |
|--------|-----------|-----|-----------|
| **Primary question** | How does the controller map observations to actions? | Which correction structures correlate most with force/action/signature? | What is the unified low-dimensional coordinate that captures observable-related structure? |
| **Input data** | Dimensionless obs and actions (time series) | `dq_ctl` fields (N snapshots x 2*NX*NY grid) + observable time series (force/action/sensor error) | POD coefficients of `dq_ctl` + observable time series |
| **Output** | Sparse symbolic control law (e.g. `a_F = 0.3*sin(u_s1)`) | CCD mode directions W, modal overlaps O_k, compactness m80, LOCO R2 | Low-dimensional coordinate z(t), observable reconstruction error |
| **Key method** | STLSQ threshold grid, G-equivariant constraints, SIN activation | POD-reduced CCD (Lyu23-inspired) | Observability Gramian / canonical correlation |
| **Current maturity** | Medium — cross-Re shared backbone found, G-equivariance validated | **Highest** — correction-field framework complete for illusion 0.75L/1.0L/1.5L with force/action/signature lines | Low-medium — framework defined, needs data alignment with CCD |
| **Validation** | Leave-one-Re-out cross-validation, closed-loop replay | LOCO (4-fold), blocked split, R2_m80 | pending alignment |
| **Key result** | Karman cloak cross-Re shared backbone exists (R2 > 0.9 for holdout 200) | 1.0L O(dqctl,dqtar)=0.913, m80=1; force/sig separated at tau=0, shared at tau_c | pending |
## Maturity by Scene
| Scene | SR/SINDy | CCD | OID |
|-------|----------|-----|-----|
| Karman cloak re50/100/200/400 | **Existing** (cross-Re backbone) | Data ready, analysis deferred | Not started |
| Illusion 0.75L | Existing | **Complete** (force/action/sig) | Partial |
| Illusion 1.0L | Existing | **Complete** (force/action/sig) | Partial |
| Illusion 1.5L | Existing | **Complete** (force/action/sig, special mechanism) | Not started |
| Steady cloak | Existing | Partial (quantitative metrics done) | Not started |
| Vortex cloak (lamb/taylor) | Existing | Not started | Not started |
## How They Assemble Into a Paper Chapter
### Chapter Structure Proposal
#### 1. Control Law Extraction (SR/SINDy)
- *Question*: What is the map from sensor observations to cylinder rotations?
- *Deliverable*: Symbolic control law for each scene, cross-scene comparison of feature usage
- *Evidence*: Leave-one-out validation, G-equivariance error < 10%
#### 2. Correction Field Analysis (CCD)
- *Question*: What flow structures does the controller actually modulate?
- *Deliverable*:
- Correction-field decomposition (`dq_ctl`)
- Force line: O(dqctl,dqtar) across diameters
- Action line: compactness m80
- Signature line: force-sig separation at zero lag, convergence at convective delay
- 1.5L special mechanism
- *Evidence*: LOCO validation R2 > 0.4 for all lines
#### 3. Low-Dimensional Coordinate (OID)
- *Question*: Can we describe controller-relevant structures in a unified low-D coordinate?
- *Deliverable*: Observable-informed coordinates z for each case, reconstruction error
- *Evidence*: Reconstruction quality vs POD-baseline
#### 4. Unified Mechanism Discussion
- Synthesize findings from all three analyses
- Key claims to support:
- Control operates by modifying pinball's existing wake (not generating new flows)
- Force-relevant correction is low-rank and target-aligned at natural scale
- Cross-scale illusion uses divergent correction paths
- Force and signature structures separate at zero lag but converge convectively
## Current Gaps by Pipeline
### SR/SINDy Gaps
- Illusion cross-diameter comparison not yet unified with CCD's correction-field framework
- Closed-loop validation of extracted control laws needs systematic comparison
### CCD Gaps
- Karman cloak analysis deferred (data ready, framework designed)
- Steady cloak needs closed-loop control to be meaningful
- Zone-restricted CCD not yet complete (in progress)
### OID Gaps
- Data pipeline not yet aligned with CCD's correction-field format
- No direct comparison of OID coordinates with CCD directions
- Requires full cross-analysis with existing CCD results
## Data Compatibility
All three pipelines ultimately read from the same data sources:
- `fields_aligned.npz` (96 aligned field snapshots)
- `controlled.npz` / `sensors.npz` (telemetry)
- `configs.py` (scene metadata)
The **correction-field framework** (`dq_ctl = q_ctl - q_blk`) is the standard analysis object across all three. Any analysis that uses raw `q_ctl` instead should be explicitly flagged as a cross-check.
## Recommendation
For the next phase of work:
1. **CCD** consolidates current results and adds zone-restricted analysis
2. **OID** should adopt CCD's data loading (`compute_correction_fields.py`) and correction-field protocol
3. **SR/SINDy** should align its cross-diameter comparison with CCD's correction-field O(dqctl,dqtar) results
4. A unifying figure comparing O(dqctl,dqtar) from CCD with SR control-law similarity across diameters would be powerful

176
src/CCD_analysis/README.md
View File

@ -1,170 +1,30 @@
# CCD_analysis: Observable-Correlated Decomposition for Fluidic Pinball Control # CCD_analysis
## Quick Start for New Agent Canonical Correlation Decomposition for fluidic pinball control analysis.
**Reading order:** ## Quick Start
1. This file (README.md) — scope, how to run, file structure
2. `ccd_knowledge.md` — confirmed facts, hard rules, current results
3. `ccd_notes.md` — what's done, what's not, future directions
4. `docs/ccd_correction_field_report.md` — comprehensive report (412 lines, explains everything from scratch)
--- 1. Read `ccd_knowledge.md` — 唯一知识库，包含理论、流程、结果
2. Run comparison: `conda run -n pycuda_3_10 python3 correction_analysis/compare_dqctl_scenes.py`
3. Run diagnostics: `conda run -n pycuda_3_10 python3 correction_analysis/diagnose_corrections.py`
## What This Pipeline Does ## Directory
CCD (Canonical Correlation Decomposition) finds flow structures most correlated with specific observables (force, action, sensor error), rather than by energy (POD). This pipeline works on the **fluidic pinball** — 3 rotating cylinders in a 2D channel — controlled by DRL (PPO).
The current analysis framework uses the **correction-field** approach: instead of analysing raw controlled fields `q_ctl`, we analyse the difference `dq_ctl = q_ctl - q_blk` (what the controller changes relative to the uncontrolled pinball).
Three analysis lines:
- **Force line**: which correction structures most determine cylinder forces (SigmaFy primary)
- **Action line**: which structures does the controller directly modulate (3 rotation speeds)
- **Signature line**: which structures most determine future sensor error (with tau delay scan)
---
## Directory Structure
``` ```
src/CCD_analysis/ correction_analysis/ — 所有当前分析代码
README.md <-- this file scripts/ — GPU 数据采集 + phase alignment
ccd_knowledge.md -- confirmed facts, hard rules, results utils/ — 核心算法（POD/CCD/场加载/平移）
ccd_notes.md -- method, what's done, what's not ccd/ — Round 5 旧基线（已冻结）
configs.py -- scene metadata, all parameters
utils/
resampling.py -- POD, CCD, field loading (CPU only)
cfd_interface.py -- LegacyCelerisLab wrapper (GPU needed)
__init__.py -- re-exports from resampling.py
scripts/
detect_period.py -- period detection + phase plan generation
replay_fields.py -- field replay for phase-aligned snapshots
collect_target_cylinder.py -- target cylinder data collection (GPU)
collect_illusion.py -- illusion PPO inference (GPU)
collect_pinball.py -- uncontrolled pinball (GPU)
collect_steady_cloak.py -- steady cloak (GPU)
collect_empty_channel.py -- empty channel (GPU)
collect_karman.py -- Karman cloak validation (GPU)
resample.py -- DEPRECATED (use detect_period + replay_fields)
visualize_ccd.py -- O_k, CCD modes, z_k, POD (Round 5 raw-field)
sanity_check_force.py -- raw force comparison target vs illusion
ccd/
run_ccd.py -- Round 5 raw-field CCD (FROZEN)
validate.py -- Round 5 validation (FROZEN)
correction_analysis/ <-- ALL CURRENT WORK HERE
compute_correction_fields.py -- build q_in/q_blk/q_ctl/q_tar + dq_*
diagnose_corrections.py -- 29 figures + zone metrics
decompose_corrections.py -- force/action CCD on dq_ctl
run_signature_line.py -- signature line CCD (tau scan)
run_15L_correction.py -- 1.5L force/action/signature
run_steady_metrics.py -- steady cloak quantitative metrics
run_zone_ccd.py -- zone-restricted CCD (3 zones)
process_legacy_steady.py -- load steady_cloak/target_channel old format
data/ data/
pinball/pinball/ -- uncontrolled pinball figures/ — 诊断图（无 colorbar）
steady_cloak/steady_cloak/ -- steady cloak (open-loop, old format) ccd/ — JSON 结果
target_channel/target_channel/ -- empty channel (old format) old_data/ — 归档废弃数据
target_cylinder/ -- target cylinders (0.75L, 1.0L, 1.5L)
illusion/ -- PPO controlled (0.75L, 1.0L, 1.5L)
karman/karman_re100/ -- Karman cloak validation
karman_target/karman_q_in/ -- vortex street without pinball
karman_blocked/karman_q_blk/ -- pinball in vortex street, no control
resampled/{scene}/ -- phase_plan.json per scene
ccd/ -- all JSON result files (8 files)
figures/ -- all PNG figures (80+)
old_data/ -- archived stale data
docs/
ccd_correction_field_report.md -- comprehensive report
sr_ccd_oid_mapping.md -- cross-pipeline mapping (DRAFT)
``` ```
--- ## Environment
## How to Regenerate Data ```
conda run -n pycuda_3_10
All commands from repo root. Environment: `conda run -n pycuda_3_10`.
### GPU Data Collection
```bash
# Target cylinders
python src/CCD_analysis/scripts/collect_target_cylinder.py --diameter 0.75 --device 2
python src/CCD_analysis/scripts/collect_target_cylinder.py --diameter 1.0 --device 2
python src/CCD_analysis/scripts/collect_target_cylinder.py --diameter 1.5 --device 2
# Illusion PPO inference (500 steps)
python src/CCD_analysis/scripts/collect_illusion.py --scene illusion_0.75L --device 2 --steps 500
python src/CCD_analysis/scripts/collect_illusion.py --scene illusion_1.0L --device 2 --steps 500
python src/CCD_analysis/scripts/collect_illusion.py --scene illusion_1.5L --device 2 --steps 500
# Baselines
python src/CCD_analysis/scripts/collect_pinball.py --device 2
python src/CCD_analysis/scripts/collect_steady_cloak.py --device 2
python src/CCD_analysis/scripts/collect_empty_channel.py --device 2
# Karman references (collected during Round 6)
python src/CCD_analysis/scripts/detect_period.py --scene karman_q_in
python src/CCD_analysis/scripts/detect_period.py --scene karman_q_blk
python src/CCD_analysis/scripts/replay_fields.py --scene karman_q_in --device 2
python src/CCD_analysis/scripts/replay_fields.py --scene karman_q_blk --device 2
``` ```
### Phase Alignment 详见 `ccd_knowledge.md`。
Run `detect_period.py` for each periodic scene, then `replay_fields.py` (GPU) to generate fields_aligned.npz.
### CPU Analysis (no GPU needed)
```bash
# Correction-field CCD pipeline
python correction_analysis/decompose_corrections.py # force/action on dq_ctl
python correction_analysis/run_signature_line.py # signature line
python correction_analysis/run_15L_correction.py # 1.5L special
python correction_analysis/run_zone_ccd.py # zone-restricted
python correction_analysis/run_steady_metrics.py # steady cloak
python correction_analysis/diagnose_corrections.py # figures + zone metrics
# Round 5 raw-field (FROZEN — not recommended for new analysis)
python ccd/run_ccd.py
python ccd/validate.py
python scripts/visualize_ccd.py
```
---
## Key Result Files
Write all results to `src/CCD_analysis/data/ccd/`:
| File | Contents | Source script |
|------|----------|--------------|
| `ccd_results.json` | Raw-field CCD (90 entries) | `ccd/run_ccd.py` |
| `validation_results.json` | Raw-field LOCO | `ccd/validate.py` |
| `correction_ccd_results.json` | Correction-field force/action (30 entries) | `decompose_corrections.py` |
| `correction_validation_results.json` | Correction-field LOCO | `decompose_corrections.py` |
| `signature_ccd_results.json` | Signature line (47 entries) | `run_signature_line.py` |
| `15L_correction_results.json` | 1.5L special (40 entries) | `run_15L_correction.py` |
| `zone_ccd_results.json` | Zone-restricted CCD (30 entries) | `run_zone_ccd.py` |
| `zone_metrics.json` | Per-zone KE/enstrophy | `diagnose_corrections.py` |
| `steady_metrics.json` | Steady cloak metrics | `run_steady_metrics.py` |
---
## Known Pitfalls (New Agent Must Read)
1. **Model naming convention**: `_2U` means S_DIM=14 (2 extra target force channels). NOT 2x velocity. u0 is ALWAYS 0.01. Round 1-3 were invalidated by this misinterpretation.
2. **action_bias vs preset_action**: action_bias=[0,-2,2] is for DRL action scaling. preset_action=[0,0,0,0,-1*U0,1*U0] is FIFO warmup. They are DIFFERENT.
3. **DO NOT use `nu_from_re()`** for illusion models. Only valid for standard u0=0.01, S_DIM=12 cases.
4. **Main analysis object is `dq_ctl = q_ctl - q_blk`**, not raw `q_ctl`. Use `correction_analysis/compute_correction_fields.py`.
5. **Karman q_in/q_blk vs q_re100**: karman_re100 is 72 frames (3 cycles, 18pts/cycle due to sampling). karman_q_in and karman_q_blk are 96 frames. When computing correction fields, N-mismatch is auto-trimmed.
6. **GPU state contamination**: Running PPO inference after other CFD on same GPU degrades similarity. Use a fresh GPU.
7. **Steady cloak has no forces**: `sensors.npz` only has sensor channels, no force data. Drag proxy uses momentum deficit.
8. **force_fx is unreliable**: R2 ~0.4, high variance. Only use for O_k trends, not mechanism claims.
9. **POD-reduced CCD limitation**: Results are constrained to the target-only POD subspace. Truncated structures cannot be recovered. Full-field CCD requires more data.

476
src/CCD_analysis/ccd_knowledge.md
View File

@ -1,138 +1,456 @@
# CCD 分析知识库 — 最终版本 (2026-06-22) # CCD Analysis Knowledge Base — 论文知识库 (2026-06-25)
## 工作阶段总览 > 统一文档。整合了 Round 5-6 的完整分析结果、correction-field 框架、Vortex/Cloak/Illusion 对比、几何对齐方案、以及所有操作流程。写论文时以本文档为准。
| 阶段 | 状态 | 说明 |
|------|------|------|
| Round 1-3 | **废弃** | 错误 u0=0.02, nu=0.008，所有结论无效 |
| Round 4 | **废弃** | 仍有 bug（模型选择、采集脚本），已归档至 old_data/ |
| **Round 5** | **完成 (frozen baseline)** | 正确数据 + fields_aligned.npz + target-only POD raw-field CCD |
| **Round 6** | **完成** | correction-field 框架: 数据层 + 差分诊断 + force/action/signature 三线 + 1.5L + 三区域分层 CCD |
### 核心结论: 哪个做完了、哪个没做
| 方向 | 状态 | 说明 |
|------|------|------|
| **Illusion 0.75L/1.0L correction-field** | **完整** | force/action/signature 三线 CCD、zone-CCD、LOCO 验证 |
| **Illusion 1.5L** | **完整** | force/action/signature CCD、phase drift、zone 指标（特殊机制 case） |
| **Steady cloak** | **定量化完成** | recirculation/RMS suppression/cancellation ratio，结论: open-loop 无效 |
| **Karman cloak** | **数据已齐，分析延后** | q_in / q_blk / q_ctl 三场已采集, phase plan 96 帧对齐 |
| **Signature line** | **已完成** (0.75L, 1.0L) | tau=0/geom/corr 扫描 + O(force,sig) + LOCO 验证 |
| **Zone-restricted CCD** | **已完成** | near_body/body_wake/sensor_zone 三层分别做 force+signature |
| **SR-CCD-OID mapping** | **草稿** | `docs/sr_ccd_oid_mapping.md` 已写，需要根据其他两方向的实际报告校正 |
--- ---
## 所有结果汇总 ## 1. 项目概览
### Round 5 — raw-field baseline (仅供参考，不再是主分析对象) ### 1.1 物理系统
**Protocol**: raw full field `q_ctl`, target-only POD, SigmaFy primary, Q_delay=6, per-case z-score **Fluidic Pinball**: 三个等间距圆柱（直径 D=20 格点，中心间距 0.75D）在二维通道内呈倒三角排列（间距 30°）。每个圆柱可独立旋转。DRL 控制器（PPO）每 SAMPLE_INTERVAL 步输出三个转速动作 [-1,1]^3，映射到物理表面速度。
| 指标 | 0.75L | 1.0L | 1.5L | **场景分类**:
|------|-------|------|------|
| O(target, illusion) mode1 | 0.673 | **0.919** | 0.621 |
| force_fy m80 | 2 | 2 | 2 |
| action m80 | 2 | 3 | 3 |
| LOCO force_fy R2_m80 | 0.71 +- 0.08 | 0.66 +- 0.03 | — |
| LOCO force_fx R2_m80 | 0.17 +- 0.49 | 0.23 +- 0.51 | — |
完整结果在 `ccd/ccd_results.json`, `ccd/validation_results.json`。 ```mermaid
flowchart LR
subgraph Cloak [Cloak：涡街伪装]
Steady[均匀来流 → 下游恢复均匀]
Karman[涡街来流 → 下游维持涡街]
Vortex[瞬态涡事件 → 涡不受畸变]
end
subgraph Illusion [Illusion：流场欺骗]
I075[0.75L: 伪装成更小圆柱]
I10[1.0L: 伪装成等大圆柱]
I15[1.5L: 伪装成更大圆柱]
end
```
**控制机制**（经 SR 和 correction-field 双重验证）:
- 后两圆柱（Top/Bottom）: 恒速反向旋转（cloak 约 ±0.313），补偿 pinball 后速度亏损
- 前端圆柱（Front）: 动态调节，控制升力（$\alpha_F = \Delta a_F/\Delta t - 14.952\mu C_{l,\text{tot}}$）
### 1.2 核心方法：Correction-field 框架
**基本思想**: 不直接分析受控流场 `q_ctl`，而是分析**控制施加的修正**：
| 符号 | 含义 | 计算方式 |
|------|------|----------|
| `q_in` | 均匀来流（干净通道） | 空通道采集 |
| `q_blk` | pinball 无控制 | 原始 pinball 涡街 |
| `q_ctl` | pinball + DRL 控制 | PPO 推理采集 |
| `q_tar` | 目标流场 | 目标圆柱/涡街参考 |
| `dq_blk = q_blk - q_in` | pinball 阻塞场 | pinball 对来流的畸变 |
| `dq_ctl = q_ctl - q_blk` | **控制修正场** | 控制器在 pinball 基础上加的改变 |
| `dq_tar = q_tar - q_blk` | **目标修正场** | 理论需达到的修正 |
**核心问题**: `dq_ctl` 是否等于 `dq_tar`？即控制做的"修正"是否就是理论上需要的"修正"？
### 1.3 关键发现
1. **Cloak 机制统一**：Steady/Karman/Vortex 三种场景的 dq_ctl 高度一致——补偿速度亏损（正 ux）+ 偶极子效应。控制策略相同，不论上游来流条件。
2. **Illusion 1.0L 与 Cloak 共享机制**：dq_ctl 结构定性一致。Illusion 本质是通过"Cloak 机制"让下游看起来像目标圆柱。
3. **Illusion 0.75L 效率降低**：O(dqctl,dqtar)=0.564，控制效率低于 1.0L
4. **Illusion 1.5L 特殊机制**：O(dqctl,dqtar)=0.667，action sigma1=0.28（其他 1.13-1.39），强相位漂移，修正集中在近体区
5. **Force/Signature 分离**：力相关结构和传感器相关结构在瞬时 tau=0 时分离（O=0.01-0.55），对流延迟后共享（O=0.72-0.81）
--- ---
### Round 6 — correction-field (主分析对象) ## 2. 方法细节
**Protocol**: `dq_ctl = q_ctl - q_blk`, target-only POD basis, SigmaFy primary, Q_delay=6 ### 2.1 物理参数
| 参数 | 值 | 说明 |
|------|-----|------|
| NX, NY | 1280, 512 | LBM 格点数 |
| L0 | 20 | 基准长度单位（1 圆柱直径） |
| U0 | 0.01 | 入口中心流速（抛物线分布） |
| nu | 0.004 | 默认运动粘度 |
| Re_code | 100 | 参考长度 2×D_cyl → Re_D=50 |
| D_cyl | L0=20 | 单一圆柱直径 |
| CENTER_Y | 255.5 | 通道中心 y 坐标 |
**Reynolds 数约定**：
- `re_code` 使用参考长度 D_REF = 2*D = 40（匹配模型文件命名）
- `Re_D = re_code / 2` 是真实物理雷诺数（使用单圆柱直径）
- 默认场景 `re_code=100` → `Re_D=50`
### 2.2 场景几何位置（2026-06-25 统一更新）
**所有场景已统一几何**：采集端直接在 `configs.py` 中设置统一坐标，无需后处理平移。
| 组 | 场景 | pinball 中心 | sensor x | 来源 configs |
|----|------|-------------|--------|-------------|
| **所有场景** | pinball, steady, karman, vortex, illusion_*, target_cylinder_* | **613 px** | **800 px** | configs.py UNIFIED 注释 |
- pinball: front x=30×L0=600, rear x=31.3×L0=626 → 中心 ≈ 613 px
- target cylinder: x=30.65×L0=613 px
- sensors: x=40×L0=800 px
### 2.3 几何对齐方案（已废弃，仅保留备用）
统一几何后不再需要后处理平移。`utils/field_translate.py` 保留作为可选工具，但不参与默认 pipeline。
<details>
<summary>旧方案（参考）</summary>
```python
SHIFT_ILLUSION_TO_CLOAK = +220 px # 旧 illusion → cloak
SHIFT_CLOAK_TO_ILLUSION = -220 px
```
</details>
---
## 3. 数据采集
### 3.1 GPU 采集脚本
| 脚本 | 功能 | 输出目录 |
|------|------|----------|
| `scripts/collect_target_cylinder.py` | 目标圆柱（单圆柱涡街） | `data/target_cylinder/{diam}L/` |
| `scripts/collect_pinball.py` | pinball 无控制基线 | `data/pinball/pinball/` |
| `scripts/collect_empty_channel.py` | 空通道 | `data/target_channel/target_channel/` |
| `scripts/collect_illusion.py` | Illusion PPO 推理 | `data/illusion/{scene}/` |
| `scripts/collect_karman.py` | Karman cloak PPO 推理 | `data/karman/karman_re100/` |
| `scripts/collect_karman_q_in.py` | 涡街无 pinball（Karman 参考） | `data/karman_target/karman_q_in/` |
| `scripts/collect_karman_q_blk.py` | 涡街+pinball 无控制（Karman 参考） | `data/karman_blocked/karman_q_blk/` |
| `scripts/collect_steady_cloak.py` | 稳态 cloak（开环恒速） | `data/steady_cloak/steady_cloak/` |
| `scripts/collect_vortex.py` | Vortex cloak（Taylor/Lamb） | `data/vortex_{type}/` |
### 3.2 采集流程（以 Karman 为例）
```
Phase 1: Target recording
FlowField → add_sensor(40*L0) ×3 → stabilize → add_vortex/cylinder
→ run(SI, zero) × F来O_LEN → save target.npz
Phase 2: Add pinball + Norm
restore → add_cylinder(pinball) ×3 → stabilize
→ run(SI, zero) × FIFO_LEN → compute norm
→ run(SI, bias_action) × FIFO_LEN → save ddf+fifo checkpoint
Phase 3: Controlled PPO inference
restore + warmup → for step in range(n_steps):
model.predict(obs) → action → run(SI, action_arr) → save telemetry + field
```
### 3.3 Norm 计算
```
force_norm_fact = 6 × max|forces|
sens_deviation = mean(sensors, axis=0)
sens_norm_fact[i] = 5 × max|sensors[:,i] - sens_deviation[i]|
obs = clip[forces/force_norm, (sens - deviation)/sens_norm], to [-1, 1]
```
**注意**：Norm 是在 scene-specific 采集时计算的，不同场景的 norm 值不同。推理时必须使用对应场景的 norm。
### 3.4 Phase Alignment 流程
```
detect_period.py:
sensors[:, 3] → FFT → dominant frequency + period
zero-crossing detection → cycle boundaries (CV_T)
select best 4-cycle window → map 4×24=96 uniform phase points
→ save phase_plan.json
replay_fields.py:
load phase_plan + ddf_checkpoint + actions
replay PPO → at each phase_plan step_index → save velocity field
→ save fields_aligned.npz + replay_verify.json
```
### 3.5 数据状态
| 场景 | scene_id | 帧数 | 格式 |
|------|----------|------|------|
| pinball | pinball | 96 | fields_aligned.npz |
| target_cylinder_{0.75,1.0,1.5}L | target_cylinder | 96 | fields_aligned.npz |
| illusion_{0.75,1.0,1.5}L | illusion | 96 | fields_aligned.npz |
| karman_re100 | karman | 72 (3周期) | fields_aligned.npz |
| karman_q_in | karman_target | 96 | fields_aligned.npz |
| karman_q_blk | karman_blocked | 96 | fields_aligned.npz |
| steady_cloak | steady_cloak | 500 | fields.npz (旧格式) |
| target_channel | target_channel | 100 | fields.npz (旧格式) |
| vortex_{lamb/taylor}/target/uncontrolled | 各自 scene_id | 150 | fields.npz (瞬态) |
---
## 4. CCD 方法
### 4.1 算法原理（Lyu23）
CCD（Canonical Correlation Decomposition）通过 CCA 在流场和可观测量之间找到相关性最大的方向：
```python
# Reduced CCD in POD coefficient space
def compute_reduced_ccd(pod_coeffs, observable, Q_delay=6):
# 1. Construct lagged observable matrix P (with symmetric delay window)
# 2. Standardize P and A (POD coefficients)
# 3. Cross-correlation matrix: C = P @ A^T / (N * sqrt(Q))
# 4. SVD(C) → R, sigma, W
# W = CCD directions in POD space
# sigma = correlation strength (sorted descending)
# z = CCD temporal coefficients
```
**物理意义**：
- sigma[0] = 第一 CCD 模式的相关系数
- m80 = 需要多少模式来捕获 80% 的总相关性（compactness 指标）
- O_k = 两个 case 之间第 k 模的重叠（内积绝对值）
### 4.2 三条分析线
| 线 | Observable | 问题 | 物理意义 |
|----|-----------|------|----------|
| **Force** (主) | SigmaFy = sum(Fy_i) | 哪些修正结构决定升力？ | 力相关流场结构 |
| **Force** (次) | SigmaFx | 哪些修正结构决定阻力？ | 不可靠（R2~0.4） |
| **Action** | [omega1, omega2, omega3] | 控制器直接调制哪些结构？ | 动作相关流场结构 |
| **Signature** | e(t+tau) = s_ctl(t+tau) - s_tar(t+tau) | 哪些结构决定未来传感器误差？ | 传感器相关流场结构 |
### 4.3 验证方法
**LOCO (Leave-One-Cycle-Out)**:
- 4 个涡街周期，留 1 做测试，3 做训练
- 训练 CCD → 预测可观测量 → 计算 R2
- 通过阈值：R2_m80 > 0.4
| Observable | 0.75L R2_m80 | 1.0L R2_m80 | 结论 |
|-----------|-------------|-------------|------|
| force_fy | 0.65 ± 0.08 | 0.64 ± 0.02 | PASS |
| force_fx | 0.38 ± 0.23 | 0.43 ± 0.11 | WARNING |
| signature tau=0 | 0.50 ± 0.09 | 0.49 ± 0.04 | PASS |
| signature tau=tau_c | 0.51 ± 0.09 | 0.53 ± 0.03 | PASS |
### 4.4 Zone-Restricted CCD
统一几何后所有场景共用一套 zone 定义（定义见 `diagnose_corrections.py` 的 `define_zones_karman()`）：
| 区域 | x 范围 (像素) | 包含 |
|------|-------------|------|
| near_body | 580-720 | pinball 周围 |
| body_wake | 720-850 | 近尾流 |
| sensor_zone | 780-850 | 传感器区域 |
---
## 5. 结果
### 5.1 Correction-field CCD 主表
| 指标 | 0.75L | 1.0L | 1.5L | | 指标 | 0.75L | 1.0L | 1.5L |
|------|-------|------|------| |------|-------|------|------|
| **O(dqctl, dqtar) mode1** | **0.564** | **0.913** | **0.667** | | **O(dqctl, dqtar) mode1** | **0.564** | **0.913** | **0.667** |
| dqctl force_fy m80 | 2 | **1** (r=8/10) | 2 | | force_fy m80 | 2 | **1** (r=8/10) | 2 |
| dqctl action sigma1 | 1.39 | 1.13 | **0.28** | | action sigma1 | 1.39 | 1.13 | **0.28** |
| **O(force, sig) at tau=0** | **0.413** | **0.551** | — | | O(force, sig) tau=0 | **0.413** | **0.551** | — |
| **O(force, sig) at tau=tau_c** | **0.806** | **0.768** | — | | O(force, sig) tau=tau_c | **0.806** | **0.768** | — |
| Phase drift | low | low | **high** | | Phase drift | low | low | **high** |
| Body-wake/sensor KE ratio | 0.73 | 1.17 | **2.58** | | Body-wake/sensor KE ratio | 0.73 | 1.17 | **2.58** |
**LOCO 验证 (r=6, R2_m80)**: ### 5.2 三区域 Force-Signature Overlap
| Observable | 0.75L | 1.0L | **0.75L** — sensor_zone 在 tau=0 时近乎正交：
|-----------|-------|------|
| force_fy | 0.65 +- 0.08 (PASS) | 0.64 +- 0.02 (PASS) |
| force_fx | 0.38 +- 0.23 (WARNING) | 0.43 +- 0.11 (WARNING) |
| signature tau=0 | 0.50 +- 0.09 (PASS) | 0.49 +- 0.04 (PASS) |
| signature tau=tau_c | 0.51 +- 0.09 (PASS) | 0.53 +- 0.03 (PASS) |
**三区域分层 CCD (zone-restricted, r=6 force_fy)**:
0.75L:
| Zone | O(force, sig) tau=0 | O(force, sig) tau=tau_c | | Zone | O(force, sig) tau=0 | O(force, sig) tau=tau_c |
|------|--------------------|------------------------| |------|--------------------|------------------------|
| near_body | 0.262 | 0.827 | | near_body | 0.262 | 0.827 |
| body_wake | 0.269 | **0.917** | | body_wake | 0.269 | **0.917** |
| sensor_zone | **0.010** | 0.722 | | sensor_zone | **0.010** | 0.722 |
1.0L: **1.0L** — 更均匀，没有近零区域：
| Zone | O(force, sig) tau=0 | O(force, sig) tau=tau_c | | Zone | O(force, sig) tau=0 | O(force, sig) tau=tau_c |
|------|--------------------|------------------------| |------|--------------------|------------------------|
| near_body | 0.596 | 0.596 | | near_body | 0.596 | 0.596 |
| body_wake | 0.509 | 0.483 | | body_wake | 0.509 | 0.483 |
| sensor_zone | 0.594 | **0.730** | | sensor_zone | 0.594 | **0.730** |
完整结果在 `ccd/correction_ccd_results.json`, `ccd/signature_ccd_results.json`, `ccd/15L_correction_results.json`, `ccd/zone_ccd_results.json`, `ccd/zone_metrics.json`。 ### 5.3 Cloak 全景对比（Steady / Karman / Vortex dq_ctl）
完整报告: `docs/ccd_correction_field_report.md` (412 行, 含 10 张图的详细解读指南) 所有 cloak 场景的 dq_ctl 展现了**一致的物理机制**：
1. **速度亏损补偿**：pinball 后方正 ux（红色），控制加速尾流
2. **偶极子效应**：圆柱附近旋转产生的偶极子模式
3. **涡量结构**：三种场景涡量分布高度相似
量化指标：
| 场景 | dq_ctl RMS (crop) | 性质 |
|------|------------------|------|
| steady_cloak | 0.196 | 稳态，开环 |
| karman_re100 | 0.397 | 周期，PPO 闭环 |
| vortex_lamb | 0.157 | 瞬态，PPO 闭环 |
| vortex_taylor | 0.191 | 瞬态，PPO 闭环 |
**结论**：不论上游条件如何（稳态/周期涡街/瞬态涡），控制策略的基本物理机制一致——"通过后两圆柱旋转补偿 pinball 阻塞引起的速度亏损，前端圆柱调节升力"。这与 SR 分析的结论完全吻合。
### 5.4 Steady Cloak 定量化
| 指标 | 值 |
|------|-----|
| 全局波动抑制 | ≈0% |
| 残留/阻塞比 | 81% |
| 传感器区残留 | 13% |
**结论**：开环恒速旋转几乎无法抑制波动。需要闭环 DRL 控制。
### 5.5 Action-CCD Mode 1 可视化
Action-CCD 找出了控制器直接调制的主要结构。对 Cloak 场景，action-CCD mode 1 ≈ dq_ctl mean field，确认了"控制调制的结构 = correction-field 的主成分"的直觉。
--- ---
## 关键可保留结论 ## 6. Correction-field 诊断图
1. **Correction-field 优于 raw-field 作为主分析对象**。1.0L 更集中(m80=1 vs 2)、0.75L 假高被扣(0.564 vs 0.673)、LOCO 未崩。 ### 6.1 图例说明
2. **1.0L 的 force-relevant correction 与 target 高度对齐** (O=0.913, m80=1)。这是目前最干净的结果。 所有位于 `data/figures/` 下（无 colorbar，干净布局，裁剪到 x=300-1100）：
3. **Force 和 signature 结构在零延迟时分离，在对流延迟后共享**。全域 O=0.41-0.55 升至 0.77-0.81。三区域 CC 显示 0.75L sensor_zone 在 tau=0 时近乎正交(O=0.01)。 | 图 | 内容 | 列数 |
|----|------|------|
| `corr_comparison_all_scenes.png` | 7 场景全景（4 cloak + 3 illusion） | 7 × 4 行 |
| `corr_illusion_comparison_dqctl.png` | Illusion 三直径对比 | 3 × 4 行 |
| `corr_cloak_comparison_dqctl.png` | Cloak 四场景对比 | 4 × 4 行 |
| `corr_illusion_{diam}L_dq_ctl/blk/tar_*.png` | 各场景单独 dq 图 | 3/1 panels |
| `corr_illusion_{diam}L_ctl_vs_tar.png` | dq_ctl vs dq_tar 对比 | 2×2 |
| `action_ccd_mode1_{scene}.png` | Action-CCD mode1 | 1×3 |
4. **1.5L 是特殊机制 case**，不是失败。O=0.667, action sigma1=0.28(1/4 of others), phase drift 显著, correction 集中在近体区。 ### 6.2 全景对比图的读法
5. **力匹配是统计量匹配，不是波形跟踪**。Cd 均值匹配好但瞬时相关 r~0。force_fx 不适合 waveform-level 断言。 每行 = 一个物理量（ux_mean / uy_mean / RMS / vorticity）
每列 = 一个场景
颜色统一（同一行在所有列间共享 vmax）
**核心观察**：
- cloak 四列间 dq_ctl 高度一致 → 控制策略不依赖上游条件
- illusion_1.0L 与 cloak 定性一致 → 1.0L illusion=cloak 机制
- illusion_0.75L 偏弱但定性相似 → 控制效率降低
- illusion_1.5L 结构突变 → 完全不同的策略
--- ---
## 数据状态 ## 7. 操作流程
所有 10 个场景均有 96 帧 phase plan。场景与对应 `scene_id`: ### 7.1 环境
| 场景 | scene_id | fields_aligned? | ```bash
|------|----------|----------------| conda run -n pycuda_3_10
| pinball | pinball | YES | # Python 3.10+, numpy, matplotlib, LegacyCelerisLab, stable-baselines3
| target_cylinder_{0.75,1.0,1.5}L | target_cylinder | YES | # CUDA device 2 (采集用)
| illusion_{0.75,1.0,1.5}L | illusion | YES | ```
| karman_re100 | karman | YES (72帧, 3cycles x 24pts, SI=800 采样密度低) |
| karman_q_in | karman_target | YES (96帧) | ### 7.2 快速重新生成所有图
| karman_q_blk | karman_blocked | YES (96帧) |
| steady_cloak | steady_cloak | NO (旧 fields.npz, 500帧) | ```bash
| target_channel | target_channel | NO (旧 fields.npz, 100帧) | # 1. Correction-field pipeline（CPU）
conda run -n pycuda_3_10 python3 correction_analysis/diagnose_corrections.py
# 2. 对比图（CPU）
conda run -n pycuda_3_10 python3 correction_analysis/compare_dqctl_scenes.py
```
### 7.3 从零开始完整流程
```bash
# Step 1: GPU 数据采集（每次 > ~4min）
# （已有数据则可跳过）
# Step 2: Phase alignment（CPU）
python3 scripts/detect_period.py --scene {scene_name}
# Step 3: Field replay（GPU）
conda run -n pycuda_3_10 python3 scripts/replay_fields.py --scene {scene_name} --device 2
# Step 4: Correction-field 分析（CPU）
conda run -n pycuda_3_10 python3 correction_analysis/compute_correction_fields.py
conda run -n pycuda_3_10 python3 correction_analysis/diagnose_corrections.py
# Step 5: 对比图（CPU）
conda run -n pycuda_3_10 python3 correction_analysis/compare_dqctl_scenes.py
```
### 7.4 添加新场景
1. 在 `configs.py` 中添加场景注册
2. 在 `compute_correction_fields.py` 的 `_SCENE_MAP` 中添加映射
3. 在 `_resolve_source()` 中添加加载器分发
4. 编写 GPU 采集脚本（参考 `collect_vortex.py`）
5. 在 `diagnose_corrections.py` 的 `SCENE_TYPES` 中添加场景
6. 运行采集 → detect_period → replay_fields → diagnose_corrections
--- ---
## 硬规则（新增和继承） ## 8. 代码结构
- Illusion 只用 o14 模型（S_DIM=14） ```
- `_2U` 表示 S_DIM=14，不是 2x velocity src/CCD_analysis/
- 每个 illusion 直径必须匹配自己的 target cylinder ccd_knowledge.md <-- 本文档（唯一知识库）
- `action_bias` 与 FIFO warmup 的 `preset_action` 不能混淆 (bias=[0,-2,2], preset=[0,0,0,0,-1*U0,1*U0]) configs.py -- 场景元数据
- 默认 u0=0.01, nu=0.004 utils/
- 主分析对象 = `dq_ctl = q_ctl - q_blk`，不是 `q_ctl` resampling.py -- POD, CCD, 场加载
- 不要对 steady cloak 使用 phase-based CCD field_translate.py -- 场平移对齐
- Karman 的分析框架与 illusion 不同（distortion compensation vs target generation） load_vortex_fields.py -- 瞬态 vortex 场加载
cfd_interface.py -- LegacyCelerisLab 封装 (GPU)
scripts/
detect_period.py -- 周期检测 → phase_plan.json
replay_fields.py -- 场回放 → fields_aligned.npz
collect_*.py -- GPU 数据采集
correction_analysis/
compute_correction_fields.py -- correction-field 计算
diagnose_corrections.py -- 诊断图生成
compare_dqctl_scenes.py -- 多场景对比图
decompose_corrections.py -- 旧 force/action CCD
run_signature_line.py -- 旧 signature CCD
run_15L_correction.py -- 旧 1.5L 分析
run_zone_ccd.py -- 旧 zone CCD
run_steady_metrics.py -- 旧 steady cloak 度量
process_legacy_steady.py -- 旧格式加载
data/
{scene_id}/{scene_name}/ -- 各场景数据
resampled/{scene}/ -- phase_plan.json
ccd/ -- JSON 结果文件
figures/ -- PNG 图（无 colorbar 版本）
old_data/ -- 归档废弃数据
```
--- ---
## 不能写成正式机制结论的话 ## 9. 硬规则
1. **Illusion 只用 o14 模型**（S_DIM=14，`_2U` 系列）。不使用 o12 模型。
2. **`_2U` 表示 S_DIM=14**（2 个额外的目标力通道），不是 2× 速度。U0 始终是 0.01。
3. **action_bias ≠ preset_action**。bias 是 DRL action scaling（illusions=[0,-2,2]），preset 是 FIFO warmup 用的动作数组。
4. **默认物理参数**：u0=0.01, nu=0.004（所有模型一致）。
5. **主分析对象 = dq_ctl**（不是 q_ctl）。
6. **Steady cloak 不要使用 phase-based CCD**——它是稳态问题。
7. **Karman 的物理问题不同**（incoming-street preservation vs target generation），不要硬套 illusion 模板。
8. **所有场景已在采集时统一几何**（pinball 中心 613 px，sensor x=800 px）；`field_translate.py` 仅作备用工具，不参与默认 pipeline。
9. **旧数据格式**（`fields.npz`）：(N, NX, NY)；**新对齐格式**（`fields_aligned.npz`）：(N, NX, NY)；**加载后统一转 (N, NY, NX)**。
---
## 10. 不能写进论文的结论（内部参考，勿引用）
以下陈述看似合理但没有被充分证据支持，不得写进正式论文：
- "Illusion 已证明使用完全不同于 target 的物理机制。" - "Illusion 已证明使用完全不同于 target 的物理机制。"
- "低 overlap 已足以证明 force 通道与 target 正交。" - "低 overlap 已足以证明 force 通道与 target 正交。"
- "CCD 已经直接识别了壁面涡量生成机制。" - "CCD 已经直接识别了壁面涡量生成机制。"
- "far wake 模态就是瞬时力的主载体。" - "far wake 模态就是瞬时力的主载体。"
---
## 11. 未完成工作（2026-06-28 更新）
| 方向 | 状态 | 说明 |
|------|------|------|
| Karman cloak CCD 分析 | 数据已齐，分析延后 | 问题定义不同（distortion compensation） |
| 1.5L force-sig overlap | 待重跑 | SI=800 数据已有；SI=200 高频采集待做 |
| SR-CCD-OID 映射 | 草稿 | 归档在 `data/old_data/sr_ccd_oid_mapping.md`，需根据各方向最终报告校正 |
| 1.5L 高频采样 | 待做 | 子步采集（SI=200），解析高频控制结构 |
| 统一几何后 CCD 重跑 | 待做 | `decompose_corrections.py` 需加 1.5L 后重跑；`correction_ccd_results.json` 仍为 6 月 15 日旧版 |
| 项目目录清理 | 已完成 (2026-06-28) | 移除根级 `old_data/`、`old_scripts/`、`output/`、`steady/`；废弃脚本归档至 `data/old_data/`；文档更新至统一几何 |

72
src/CCD_analysis/ccd_notes.md
View File

@ -1,72 +0,0 @@
# CCD 分析方法与工作规划 — 最终版本
## 工作阶段总览
### Round 5 (raw-field baseline) — 完成
见 `ccd_knowledge.md` 中的 Round 5 章节。所有 `ccd/` 目录下脚本已冻结不修改。
### Round 6 (correction-field framework) — 全部完成
**产出列表**:
| 文件 | 说明 |
|------|------|
| `correction_analysis/process_legacy_steady.py` | 加载 steady_cloak/target_channel 旧格式 |
| `correction_analysis/compute_correction_fields.py` | q_in/q_blk/q_ctl/q_tar + dq_* 差分场计算 |
| `correction_analysis/diagnose_corrections.py` | 29张差分场图 + 三层区域 KE/enstrophy 指标 |
| `correction_analysis/decompose_corrections.py` | dq_ctl 上 force/action CCD (0.75L, 1.0L) |
| `correction_analysis/run_signature_line.py` | signature line CCD (0.75L, 1.0L, tau扫描) |
| `correction_analysis/run_15L_correction.py` | 1.5L force/action/signature CCD |
| `correction_analysis/run_steady_metrics.py` | steady cloak 定量化 |
| `correction_analysis/run_zone_ccd.py` | 三区域分层 CCD (0.75L, 1.0L) |
| `docs/ccd_correction_field_report.md` | 综合报告 (412行, 含10图解读) |
| `docs/sr_ccd_oid_mapping.md` | 三线映射草稿 |
**数据采集 (GPU)**:
- karman_q_in (涡街无pinball) 和 karman_q_blk (涡街+pinball无控制) — 各 96帧 aligned 场
- karman_re100 — 72帧 (3 cycles, 每周期18点, rho=1.33)
---
## 已完成 vs 未完成
### 已完成
- Illusion 三直径 correction-field force/action/signature CCD
- O(dqctl,dqtar) 跨直径对比
- O(force,sig) 模态重叠 (全域 + 三区域)
- LOCO 验证 (除 1.5L)
- 1.5L special mechanism 分析
- Zone-restricted CCD (near_body, body_wake, sensor_zone)
- Steady cloak 定量化
- Snapshot POD 加速 (`utils/resampling.py: compute_pod`)
- Karman 参考场采集
### 未完成/延后
- Karman cloak 周期 correction analysis (数据已齐，框架已设计，分析延后)
- 1.5L force vs signature overlap (0.75L/1.0L 已完成，1.5L 有 signature m80 但缺 O 值)
- Steady cloak 需要 closed-loop DRL 数据才能做有意义分析
- OID/PCD/whitening 等更高级版本
---
## 当前建议的继续方向
1. **Karman cloak 分析** — 已有 `q_in/q_blk/q_ctl` 三场和 `correction_analysis/compute_correction_fields.py` 支持，可平移 illusion 的 force/action/signature 框架。注意 Karman 的物理问题不同 (incoming-street preservation vs target generation)。
2. **1.5L force-sig overlap** — 补齐后可与 0.75L/1.0L 比较。已在 `correction_analysis/run_15L_correction.py` 中有 signature line 框架，补 O 值即可。
3. **SR-CCD-OID 映射校正** — `docs/sr_ccd_oid_mapping.md` 需要根据 SR 和 OID 两方向的实际现状校正。
---
## 成功标准回顾
这套分析框架若要算成功，至少应支持以下判断:
- [x] 控制首先调制的是 pinball 已存在基线上的 **correction field**，而不是直接"生成目标全流场"
- [x] force-relevant structures 与 signature-relevant structures 可以不同（证据: O(force,sig) tau=0 = 0.01-0.55）
- [ ] force 与 signature 若由不同结构族主导 → 机制分层（部分证据: zone-CCD 显示 0.75L sensor_zone O=0.01，但 tau_c 后共享）
- [x] 1.5L 显示不同于 0.75L/1.0L 的修正策略（证据: action sigma1=0.28, O=0.667, phase drift）

51
src/CCD_analysis/configs.py
View File

@ -165,9 +165,9 @@ for key, mn, diam, si in _ILLUSION_2U:
"model_subdir": "250525", "model_subdir": "250525",
"n_objects_env": 6, "n_objects_env": 6,
"obs_slice": (0, 12), "obs_slice": (0, 12),
"sensor_x": 30.0, "sensor_x": 40.0, # UNIFIED: was 30.0
"pinball_front_x": 19.0, "pinball_front_x": 30.0, # UNIFIED: was 19.0
"pinball_rear_x": 20.3, "pinball_rear_x": 31.3, # UNIFIED: was 20.3
"target_type": "periodic", "target_type": "periodic",
"s_dim": 14, # CRITICAL: all 2U are 14-dim "s_dim": 14, # CRITICAL: all 2U are 14-dim
"u0": U0, # 0.01 (NOT 0.02) "u0": U0, # 0.01 (NOT 0.02)
@ -190,8 +190,8 @@ for diam, si in [(0.75, 400), (1.0, 600), (1.5, 800)]:
"model_name": None, "model_name": None,
"n_objects_env": 4, # 1 cylinder + 3 sensors "n_objects_env": 4, # 1 cylinder + 3 sensors
"obs_slice": (0, 8), # cylinder force(2) + sensor(6) "obs_slice": (0, 8), # cylinder force(2) + sensor(6)
"sensor_x": 30.0, "sensor_x": 40.0, # UNIFIED: was 30.0
"cylinder_x": 20.0, "cylinder_x": 30.65, # UNIFIED: was 20.0 (pinball center)
"target_type": "periodic", "target_type": "periodic",
"s_dim": None, "s_dim": None,
"u0": U0, # 0.01, matching illusion "u0": U0, # 0.01, matching illusion
@ -217,6 +217,47 @@ SCENES["target_channel"] = {
"nu": nu_from_re(100), "nu": nu_from_re(100),
} }
# -- Vortex cloak scenes (transient, Taylor monopole & Lamb dipole) ---------
# Taylor: monopole vortex, strength=0.03*U0, r=2*L0=40
# Lamb: dipole vortex, strength=0.5*U0, r=2*L0=40
# Both: MAX_STEPS=150, action_scale=4, action_bias=(0,-4,4), s_dim=12
# Geometry: pinball at (30, 31.3)xL0, sensors at 40*xL0
# Vortex at 10*xL0 (target) or 15*xL0 (pinball phase)
_VORTEX_SCENES = [
("vortex_lamb", "vortex_lamb", "lamb", 0.50),
("vortex_taylor", "vortex_taylor", "taylor", 0.03),
("vortex_uncontrolled_lamb", None, "lamb", 0.50),
("vortex_uncontrolled_taylor", None, "taylor", 0.03),
("vortex_target_lamb", None, "lamb", 0.50),
("vortex_target_taylor", None, "taylor", 0.03),
]
for key, mn, vtype, vstrength in _VORTEX_SCENES:
is_controlled = mn is not None
SCENES[key] = {
"scene_id": key,
"re_code": 100,
"has_disturbance": False,
"sample_interval": 800,
"vortex_type": vtype,
"vortex_strength": vstrength,
"conv_len": 30,
"action_scale": 4.0 if is_controlled else None,
"action_bias": (0.0, -4.0, 4.0) if is_controlled else None,
"source": "PPO_inference" if is_controlled else "open_loop",
"model_name": mn,
"model_subdir": "old",
"n_objects_env": 6,
"obs_slice": (0, 12),
"sensor_x": 40.0,
"pinball_front_x": 30.0,
"pinball_rear_x": 31.3,
"target_type": "transient",
"max_steps": 150,
"s_dim": 12 if is_controlled else None,
"u0": U0,
"nu": nu_from_re(100),
}
# -- Utility helpers --------------------------------------------------------- # -- Utility helpers ---------------------------------------------------------

116
src/CCD_analysis/correction_analysis/compute_correction_fields.py
View File

@ -8,9 +8,10 @@ Each scene type maps to different data sources:
| illusion_1.0L | target_channel* | pinball | illusion_1.0L | target_cylinder_1.0L | | illusion_1.0L | target_channel* | pinball | illusion_1.0L | target_cylinder_1.0L |
| illusion_1.5L | target_channel* | pinball | illusion_1.5L | target_cylinder_1.5L | | illusion_1.5L | target_channel* | pinball | illusion_1.5L | target_cylinder_1.5L |
| steady_cloak | target_channel* | pinball | steady_cloak* | None (target=q_in) | | steady_cloak | target_channel* | pinball | steady_cloak* | None (target=q_in) |
| karman_re100 | karman_q_in (TBD) | TBD | karman_re100 | karman_q_in (TBD) | | karman_re100 | karman_q_in | karman_q_blk | karman_re100 | karman_q_in |
(*) loaded via load_legacy_steady() (*) loaded via load_legacy_steady(). Vortex scenes use load_vortex_fields().
All scenes now use UNIFIED geometry (pinball center at 613px, sensors at 800px).
""" """
from __future__ import annotations from __future__ import annotations
@ -21,65 +22,118 @@ from typing import Any, Optional
import numpy as np import numpy as np
from CCD_analysis.configs import NX, NY from CCD_analysis.configs import DATA_DIR, NX, NY
from CCD_analysis.utils.resampling import ( from CCD_analysis.utils.resampling import (
load_aligned_fields, load_aligned_fields,
build_field_matrix as _build_field_matrix, build_field_matrix as _build_field_matrix,
) )
from CCD_analysis.correction_analysis.process_legacy_steady import load_legacy_steady from CCD_analysis.correction_analysis.process_legacy_steady import load_legacy_steady
from CCD_analysis.utils.load_vortex_fields import load_vortex_fields
# ---------------------------------------------------------------------------
# Placeholder for future karman reference fields
# ---------------------------------------------------------------------------
_KARMAN_PLACEHOLDER = None
"""Temporary placeholder for karman_q_in / karman_q_blk data.
Will be replaced with proper data loading once collected.
"""
def _load_karman_placeholder(scene_label: str) -> Optional[dict]:
"""Return None placeholder for karman reference fields."""
print(f" [PLACEHOLDER] karman reference '{scene_label}' -- returning None")
return None
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Source mapping # Source mapping
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def _resolve_source(name: str) -> dict: def _resolve_source(name: str) -> Optional[dict]:
"""Load a named data source, dispatching to the correct loader. """Load a named data source, dispatching to the correct loader.
Parameters Parameters
---------- ----------
name : str name : str
Scene or special name: Scene or special name:
- 'target_channel' -> load_legacy_steady - 'target_channel', 'steady_cloak' -> load_legacy_steady
- 'steady_cloak' -> load_legacy_steady - 'vortex_*' -> load_vortex_fields (transient)
- 'pinball' -> load_aligned_fields - 'karman_re100' -> _load_karman_re100 (handles 72 vs 96 mismatch)
- 'illusion_*' -> load_aligned_fields - all others -> load_aligned_fields
- 'target_cylinder_*'-> load_aligned_fields
- 'karman_re100' -> load_aligned_fields
- 'karman_q_in' -> _load_karman_placeholder
- 'karman_q_blk' -> _load_karman_placeholder
Returns Returns
------- -------
dict or None dict or None
""" """
_LEGACY_SCENES = {"target_channel", "steady_cloak"} _LEGACY_SCENES = {"target_channel", "steady_cloak"}
_VORTEX_SCENES = {
"vortex_lamb", "vortex_taylor",
"vortex_uncontrolled_lamb", "vortex_uncontrolled_taylor",
"vortex_target_lamb", "vortex_target_taylor",
}
if name in _LEGACY_SCENES: if name in _LEGACY_SCENES:
return load_legacy_steady(name) return load_legacy_steady(name)
elif name in _VORTEX_SCENES:
return load_vortex_fields(name)
elif name == "karman_re100":
return _load_karman_re100()
else: else:
# karman_q_in, karman_q_blk, and all others use aligned loader
return load_aligned_fields(name) return load_aligned_fields(name)
# ---------------------------------------------------------------------------
# Karman re100 special loader (handles 72 vs 96 frame mismatch)
# ---------------------------------------------------------------------------
def _load_karman_re100() -> dict:
"""Load karman_re100 aligned fields, handling the 72 vs 96 frame mismatch.
karman_re100's fields_aligned.npz has 72 snapshots (3 cycles x 24 pts)
but the phase_plan.json lists 96 step_indices (4 cycles x 24 pts).
This loader truncates step_indices to match.
"""
import json as _json
scene_name = "karman_re100"
from CCD_analysis.configs import SCENES as _SCENES
cfg = _SCENES[scene_name]
scene_id = cfg["scene_id"]
data_dir = os.path.join(DATA_DIR, scene_id, scene_name)
# Load fields_aligned.npz
fa_path = os.path.join(data_dir, "fields_aligned.npz")
fd = np.load(fa_path)
ux_raw = fd["ux"]
uy_raw = fd["uy"]
N = ux_raw.shape[0] # 72
fd.close()
# Transpose (N, NX, NY) -> (N, NY, NX)
ux = np.ascontiguousarray(ux_raw.transpose(0, 2, 1))
uy = np.ascontiguousarray(uy_raw.transpose(0, 2, 1))
# Load phase_plan, truncate to N
plan_path = os.path.join(DATA_DIR, "resampled", scene_name, "phase_plan.json")
with open(plan_path) as f:
plan = _json.load(f)
step_indices = list(plan["step_indices"][:N])
# Load telemetry
tele_path = os.path.join(data_dir, "controlled.npz")
td = np.load(tele_path)
result = {
"ux": ux,
"uy": uy,
"forces": td["forces"][step_indices] if "forces" in td else None,
"actions": td["actions"][step_indices] if "actions" in td else None,
"sensors": td["sensors"][step_indices] if "sensors" in td else None,
"meta": {
"scene": scene_name,
"scene_id": scene_id,
"gate": plan.get("gate", "unknown"),
"CV_T": plan.get("CV_T"),
"f_dom": plan.get("f_dom"),
"N_raw_per_cycle": plan.get("N_raw_per_cycle"),
"rho_interp": plan.get("rho_interp"),
"sample_interval": cfg.get("sample_interval"),
"note": "step_indices truncated from 96 to 72 (3 cycles, not 4)",
},
"step_indices": step_indices,
}
td.close()
return result
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Correction computation # Correction computation
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@ -91,6 +145,8 @@ _SCENE_MAP = {
"illusion_1.5L": ("target_channel", "pinball", "illusion_1.5L", "target_cylinder_1.5L"), "illusion_1.5L": ("target_channel", "pinball", "illusion_1.5L", "target_cylinder_1.5L"),
"steady_cloak": ("target_channel", "pinball", "steady_cloak", None), "steady_cloak": ("target_channel", "pinball", "steady_cloak", None),
"karman_re100": ("karman_q_in", "karman_q_blk", "karman_re100", "karman_q_in"), "karman_re100": ("karman_q_in", "karman_q_blk", "karman_re100", "karman_q_in"),
"vortex_lamb": ("target_channel", "vortex_uncontrolled_lamb", "vortex_lamb", "vortex_target_lamb"),
"vortex_taylor": ("target_channel", "vortex_uncontrolled_taylor", "vortex_taylor", "vortex_target_taylor"),
} }

46
src/CCD_analysis/correction_analysis/diagnose_corrections.py
View File

@ -34,13 +34,15 @@ from CCD_analysis.correction_analysis.compute_correction_fields import (
FIG_DIR = os.path.join(DATA_DIR, "figures") FIG_DIR = os.path.join(DATA_DIR, "figures")
os.makedirs(FIG_DIR, exist_ok=True) os.makedirs(FIG_DIR, exist_ok=True)
# Scene types to process (karman will be added when data is ready) # Scene types to process
SCENE_TYPES = [ SCENE_TYPES = [
"illusion_0.75L", "illusion_0.75L",
"illusion_1.0L", "illusion_1.0L",
"illusion_1.5L", "illusion_1.5L",
"steady_cloak", "steady_cloak",
"karman_re100", "karman_re100",
"vortex_lamb",
"vortex_taylor",
] ]
@ -206,23 +208,20 @@ def plot_mean_rms(dq: dict, label: str, prefix: str, zones: Optional[dict] = Non
# Mean ux # Mean ux
vmax = max(abs(ux_m).max(), 1e-12) vmax = max(abs(ux_m).max(), 1e-12)
im0 = axes[0].imshow(ux_m, cmap="RdBu_r", vmin=-vmax, vmax=vmax, axes[0].imshow(ux_m, cmap="RdBu_r", vmin=-vmax, vmax=vmax,
origin="lower", aspect="equal", extent=extent) origin="lower", aspect="equal", extent=extent)
axes[0].set_title(f"{label}: mean ux") axes[0].set_title(f"{label}: mean ux")
plt.colorbar(im0, ax=axes[0], fraction=0.046)
# Mean uy # Mean uy
vmax = max(abs(uy_m).max(), 1e-12) vmax = max(abs(uy_m).max(), 1e-12)
im1 = axes[1].imshow(uy_m, cmap="RdBu_r", vmin=-vmax, vmax=vmax, axes[1].imshow(uy_m, cmap="RdBu_r", vmin=-vmax, vmax=vmax,
origin="lower", aspect="equal", extent=extent) origin="lower", aspect="equal", extent=extent)
axes[1].set_title(f"{label}: mean uy") axes[1].set_title(f"{label}: mean uy")
plt.colorbar(im1, ax=axes[1], fraction=0.046)
# RMS magnitude # RMS magnitude
im2 = axes[2].imshow(rms, cmap="viridis", origin="lower", axes[2].imshow(rms, cmap="viridis", origin="lower",
aspect="equal", extent=extent) aspect="equal", extent=extent)
axes[2].set_title(f"{label}: RMS magnitude") axes[2].set_title(f"{label}: RMS magnitude")
plt.colorbar(im2, ax=axes[2], fraction=0.046)
# Overlay zone boundaries if provided # Overlay zone boundaries if provided
if zones is not None: if zones is not None:
@ -253,11 +252,10 @@ def plot_vorticity(dq: dict, label: str, prefix: str):
fig, ax = plt.subplots(figsize=(10, 4)) fig, ax = plt.subplots(figsize=(10, 4))
vmax = max(np.percentile(abs(vor), 99), 1e-12) vmax = max(np.percentile(abs(vor), 99), 1e-12)
im = ax.imshow(vor, cmap="RdBu_r", vmin=-vmax, vmax=vmax, ax.imshow(vor, cmap="RdBu_r", vmin=-vmax, vmax=vmax,
origin="lower", aspect="equal", origin="lower", aspect="equal",
extent=(0, NX - 1, 0, NY - 1)) extent=(0, NX - 1, 0, NY - 1))
ax.set_title(f"{label}: mean vorticity") ax.set_title(f"{label}: mean vorticity")
plt.colorbar(im, ax=ax, fraction=0.046, label=r"$\omega_z$")
plt.tight_layout() plt.tight_layout()
path = os.path.join(FIG_DIR, f"{prefix}_vorticity_{label.replace(' ', '_')}.png") path = os.path.join(FIG_DIR, f"{prefix}_vorticity_{label.replace(' ', '_')}.png")
fig.savefig(path, dpi=150) fig.savefig(path, dpi=150)
@ -286,7 +284,7 @@ def run():
try: try:
corr = compute_correction(scene_type) corr = compute_correction(scene_type)
except (FileNotFoundError, KeyError) as e: except (FileNotFoundError, KeyError, AssertionError, ValueError) as e:
print(f" SKIP: {e}", flush=True) print(f" SKIP: {e}", flush=True)
continue continue
@ -296,7 +294,8 @@ def run():
# Determine which zones to use # Determine which zones to use
is_illusion = "illusion" in scene_type is_illusion = "illusion" in scene_type
zones = zones_ill if is_illusion else zones_ill # zones_karman for later is_karman = "karman" in scene_type or "vortex" in scene_type
zones = zones_ill if is_illusion else (zones_karman if is_karman else zones_ill)
for dq_key, dq_label in [ for dq_key, dq_label in [
("dq_blk", "dq_blk (pinball blockage)"), ("dq_blk", "dq_blk (pinball blockage)"),
@ -314,8 +313,8 @@ def run():
metrics = zone_metrics(dq, zones, dq_label) metrics = zone_metrics(dq, zones, dq_label)
all_metrics[f"{scene_type}_{dq_key}"] = metrics all_metrics[f"{scene_type}_{dq_key}"] = metrics
# For illusion, also plot dq_tar if available # For illusion/karman/vortex, also plot dq_tar if available
if dq_key == "dq_ctl" and is_illusion: if dq_key == "dq_ctl" and (is_illusion or is_karman):
dq_tar = corr.get("dq_tar") dq_tar = corr.get("dq_tar")
if dq_tar is not None: if dq_tar is not None:
plot_mean_rms(dq_tar, "dq_tar (target correction)", prefix, zones) plot_mean_rms(dq_tar, "dq_tar (target correction)", prefix, zones)
@ -330,30 +329,26 @@ def run():
ux_tar, _ = mean_field(dq_tar["ux"], dq_tar["uy"]) ux_tar, _ = mean_field(dq_tar["ux"], dq_tar["uy"])
vmax = max(abs(ux_ctl).max(), abs(ux_tar).max(), 1e-12) vmax = max(abs(ux_ctl).max(), abs(ux_tar).max(), 1e-12)
im = axes[0, 0].imshow(ux_ctl, cmap="RdBu_r", vmin=-vmax, vmax=vmax, axes[0, 0].imshow(ux_ctl, cmap="RdBu_r", vmin=-vmax, vmax=vmax,
origin="lower", aspect="equal", extent=extent) origin="lower", aspect="equal", extent=extent)
axes[0, 0].set_title("dq_ctl mean ux") axes[0, 0].set_title("dq_ctl mean ux")
plt.colorbar(im, ax=axes[0, 0], fraction=0.046)
im = axes[0, 1].imshow(ux_tar, cmap="RdBu_r", vmin=-vmax, vmax=vmax, axes[0, 1].imshow(ux_tar, cmap="RdBu_r", vmin=-vmax, vmax=vmax,
origin="lower", aspect="equal", extent=extent) origin="lower", aspect="equal", extent=extent)
axes[0, 1].set_title("dq_tar mean ux") axes[0, 1].set_title("dq_tar mean ux")
plt.colorbar(im, ax=axes[0, 1], fraction=0.046)
# Row 1: RMS # Row 1: RMS
rms_ctl = rms_field(dq["ux"], dq["uy"]) rms_ctl = rms_field(dq["ux"], dq["uy"])
rms_tar = rms_field(dq_tar["ux"], dq_tar["uy"]) rms_tar = rms_field(dq_tar["ux"], dq_tar["uy"])
rmax = max(rms_ctl.max(), rms_tar.max(), 1e-12) rmax = max(rms_ctl.max(), rms_tar.max(), 1e-12)
im = axes[1, 0].imshow(rms_ctl, cmap="viridis", vmin=0, vmax=rmax, axes[1, 0].imshow(rms_ctl, cmap="viridis", vmin=0, vmax=rmax,
origin="lower", aspect="equal", extent=extent) origin="lower", aspect="equal", extent=extent)
axes[1, 0].set_title("dq_ctl RMS") axes[1, 0].set_title("dq_ctl RMS")
plt.colorbar(im, ax=axes[1, 0], fraction=0.046)
im = axes[1, 1].imshow(rms_tar, cmap="viridis", vmin=0, vmax=rmax, axes[1, 1].imshow(rms_tar, cmap="viridis", vmin=0, vmax=rmax,
origin="lower", aspect="equal", extent=extent) origin="lower", aspect="equal", extent=extent)
axes[1, 1].set_title("dq_tar RMS") axes[1, 1].set_title("dq_tar RMS")
plt.colorbar(im, ax=axes[1, 1], fraction=0.046)
plt.suptitle(f"{scene_type}: dq_ctl vs dq_tar comparison") plt.suptitle(f"{scene_type}: dq_ctl vs dq_tar comparison")
plt.tight_layout() plt.tight_layout()
@ -375,20 +370,17 @@ def run():
extent = (0, NX - 1, 0, NY - 1) extent = (0, NX - 1, 0, NY - 1)
vmax = max(abs(ux_b).max(), abs(ux_c).max(), abs(ux_cancel).max(), 1e-12) vmax = max(abs(ux_b).max(), abs(ux_c).max(), abs(ux_cancel).max(), 1e-12)
im = axes[0].imshow(ux_b, cmap="RdBu_r", vmin=-vmax, vmax=vmax, axes[0].imshow(ux_b, cmap="RdBu_r", vmin=-vmax, vmax=vmax,
origin="lower", aspect="equal", extent=extent) origin="lower", aspect="equal", extent=extent)
axes[0].set_title("dq_blk mean ux (blockage)") axes[0].set_title("dq_blk mean ux (blockage)")
plt.colorbar(im, ax=axes[0], fraction=0.046)
im = axes[1].imshow(ux_c, cmap="RdBu_r", vmin=-vmax, vmax=vmax, axes[1].imshow(ux_c, cmap="RdBu_r", vmin=-vmax, vmax=vmax,
origin="lower", aspect="equal", extent=extent) origin="lower", aspect="equal", extent=extent)
axes[1].set_title("dq_ctl mean ux (correction)") axes[1].set_title("dq_ctl mean ux (correction)")
plt.colorbar(im, ax=axes[1], fraction=0.046)
im = axes[2].imshow(ux_cancel, cmap="RdBu_r", vmin=-vmax, vmax=vmax, axes[2].imshow(ux_cancel, cmap="RdBu_r", vmin=-vmax, vmax=vmax,
origin="lower", aspect="equal", extent=extent) origin="lower", aspect="equal", extent=extent)
axes[2].set_title("dq_ctl + dq_blk (cancel test)") axes[2].set_title("dq_ctl + dq_blk (cancel test)")
plt.colorbar(im, ax=axes[2], fraction=0.046)
plt.suptitle(f"Steady cloak: cancellation test") plt.suptitle(f"Steady cloak: cancellation test")
plt.tight_layout() plt.tight_layout()

419
src/CCD_analysis/old_scripts/analyze_15L.py
View File

@ -1,419 +0,0 @@
"""1.5L auxiliary analysis: raw diagnostics, spectrum, windowed periodicity, POD.
Usage:
python3 src/CCD_analysis/scripts/analyze_15L.py
Output:
src/CCD_analysis/data/figures/15L_*.png
"""
from __future__ import annotations
import json
import os
import sys
from collections import deque
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
_SRC = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from CCD_analysis.configs import DATA_DIR, SCENES
from CCD_analysis.utils.resampling import (
detect_dominant_frequency, detect_cycle_stability, phase_resample,
compute_pod, cumulative_energy, compute_reduced_ccd,
)
FIG_DIR = os.path.join(DATA_DIR, "figures")
os.makedirs(FIG_DIR, exist_ok=True)
# Scene config — read from configs.py, not hardcoded
_SCENE = SCENES["illusion_1.5L"]
SI = _SCENE["sample_interval"] # 800 for 1.5L
CONV_LEN = _SCENE.get("conv_len", 36) # Illusion uses 36
FIFO_LEN = 150
def load_controlled(name):
p = os.path.join(DATA_DIR, "illusion", name, "controlled.npz")
d = np.load(p)
return d["sensors"], d["forces"], d["actions"]
def load_target(name):
p = os.path.join(DATA_DIR, "target_cylinder", name, "sensors.npz")
d = np.load(p)
return d["sensors"], d["forces"]
def load_pinball_sensors():
p = os.path.join(DATA_DIR, "pinball", "pinball", "sensors.npz")
d = np.load(p)
return d["sensors"]
def load_resampled_fields(name):
p = os.path.join(DATA_DIR, "resampled", name, "resampled.npz")
d = np.load(p)
return d["ux"], d["uy"]
# ============================================================
# Task 3.1: Raw time-series diagnostics
# ============================================================
def task_31():
print("=== Task 3.1: Raw time-series diagnostics ===", flush=True)
sens_i, forc_i, act_i = load_controlled("illusion_1.5L")
sens_t, forc_t = load_target("target_cylinder_1.5L")
n_plot = min(400, len(sens_i))
t = np.arange(n_plot) * SI / 1000 # time in T0 units (1 T0 = 1000 steps)
fig, axes = plt.subplots(3, 1, figsize=(14, 10))
# Sensors
ax = axes[0]
for ch in range(6):
ax.plot(t, sens_i[:n_plot, ch], label=f"ill_s{ch}", alpha=0.7)
ax.plot(t, sens_t[:n_plot, 3], "k--", label="target_s1_v", linewidth=2)
ax.set_ylabel("Velocity (lattice)")
ax.set_title("1.5L Sensors: Illusion vs Target")
ax.legend(fontsize=7, ncol=3)
ax.grid(True, alpha=0.3)
# Forces
ax = axes[1]
for ch in range(6):
ax.plot(t, forc_i[:n_plot, ch], label=f"ill_F{ch}", alpha=0.7)
ax.plot(t, forc_t[:n_plot, 0], "k--", label="target_Fx", linewidth=2)
ax.plot(t, forc_t[:n_plot, 1], "k:", label="target_Fy", linewidth=2)
ax.set_ylabel("Force (lattice)")
ax.set_title("1.5L Forces")
ax.legend(fontsize=7, ncol=3)
ax.grid(True, alpha=0.3)
# Actions
ax = axes[2]
for ch in range(3):
ax.plot(t, act_i[:n_plot, ch], label=f"Omega_{ch}")
ax.set_xlabel("Time (T0 units)")
ax.set_ylabel("Omega (normalised)")
ax.set_title("1.5L Actions (DRL output, [-1, 1])")
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
path = os.path.join(FIG_DIR, "15L_raw_timeseries.png")
fig.savefig(path, dpi=150)
plt.close(fig)
print(f" Saved: {path}", flush=True)
# ============================================================
# Task 3.2: Spectrum analysis
# ============================================================
def task_32():
print("=== Task 3.2: Spectrum analysis ===", flush=True)
sens_i, forc_i, act_i = load_controlled("illusion_1.5L")
sens_t, forc_t = load_target("target_cylinder_1.5L")
sens_p = load_pinball_sensors()
fig, axes = plt.subplots(2, 2, figsize=(14, 8))
def add_spectrum(signal, ax, label, color, ls="-"):
y = signal - np.mean(signal)
n = len(y)
window = np.hanning(n)
spec = np.abs(np.fft.rfft(y * window)) ** 2
freqs = np.fft.rfftfreq(n, d=SI)
ax.plot(freqs[1:], spec[1:], label=label, color=color, ls=ls, alpha=0.8)
# Sensor v component (channel 1 = uy of center sensor)
# In controlled.npz: sensors[:, 1] = sensor1_uy (center sensor v)
ax = axes[0, 0]
add_spectrum(sens_t[:500, 1], ax, "Target", "red")
add_spectrum(sens_i[:500, 1], ax, "Illusion 1.5L", "blue")
add_spectrum(sens_p[:500, 1], ax, "Pinball (uncontrolled)", "green")
ax.set_xlim(0, 0.005)
ax.set_xlabel("Frequency (1/step)")
ax.set_ylabel("Power")
ax.set_title("Spectrum: sensor v (center)")
ax.legend()
ax.grid(True, alpha=0.3)
# Actions spectrum
ax = axes[0, 1]
for ch in range(3):
add_spectrum(act_i[:500, ch], ax, f"Action {ch}", f"C{ch+1}")
ax.set_xlim(0, 0.005)
ax.set_xlabel("Frequency (1/step)")
ax.set_ylabel("Power")
ax.set_title("1.5L Action spectrum")
ax.legend()
ax.grid(True, alpha=0.3)
# Force spectra
ax = axes[1, 0]
add_spectrum(forc_t[:500, 0], ax, "Target Fx", "red")
add_spectrum(forc_i[:500, 0] + forc_i[:500, 2] + forc_i[:500, 4], ax, "Illusion total Fx", "blue")
ax.set_xlim(0, 0.005)
ax.set_xlabel("Frequency (1/step)")
ax.set_ylabel("Power")
ax.set_title("Force Fx spectrum")
ax.legend()
ax.grid(True, alpha=0.3)
# Strouhal comparison
ax = axes[1, 1]
for name, sig, c in [
("Target", sens_t[:500, 1], "red"),
("Illusion", sens_i[:500, 1], "blue"),
("Pinball", sens_p[:500, 1], "green"),
]:
f_dom, T_dom, _ = detect_dominant_frequency(sig, SI)
St = f_dom * (1.5 * 20) / 0.01 # D=1.5*L0, U0=0.01
ax.bar(name, St, color=c, alpha=0.6, label=f"St={St:.3f}")
ax.set_ylabel("Strouhal number")
ax.set_title("Dominant Strouhal comparison (1.5L ref)")
ax.grid(True, alpha=0.3)
plt.tight_layout()
path = os.path.join(FIG_DIR, "15L_spectrum.png")
fig.savefig(path, dpi=150)
plt.close(fig)
print(f" Saved: {path}", flush=True)
# ============================================================
# Task 3.3: Windowed periodicity
# ============================================================
def task_33():
print("=== Task 3.3: Windowed periodicity ===", flush=True)
sens_i, forc_i, act_i = load_controlled("illusion_1.5L")
signal = sens_i[:, 1] # center sensor v
window = 200
stride = 20
n_windows = (len(signal) - window) // stride
cv_vals, T_vals, f_vals, t_centers = [], [], [], []
for w in range(n_windows):
seg = signal[w * stride:w * stride + window]
cv_T, mean_T, _ = detect_cycle_stability(seg, SI)
f_dom, T_dom, _ = detect_dominant_frequency(seg, SI)
cv_vals.append(cv_T)
T_vals.append(mean_T)
f_vals.append(f_dom)
t_centers.append((w * stride + window // 2) * SI / 1000)
fig, axes = plt.subplots(3, 1, figsize=(14, 8), sharex=True)
ax = axes[0]
ax.plot(t_centers, cv_vals, "o-", markersize=3)
ax.axhline(0.10, color="r", ls="--", label="strict gate")
ax.axhline(0.12, color="orange", ls="--", label="relaxed gate")
ax.set_ylabel("CV_T")
ax.set_title("1.5L Windowed cycle stability (window=200 steps)")
ax.legend()
ax.grid(True, alpha=0.3)
ax = axes[1]
ax.plot(t_centers, T_vals, "o-", markersize=3, color="green")
ax.set_ylabel("Mean period (steps)")
ax.axhline(800 * 24.2, color="gray", ls=":", label="expected") # N_raw*SI
ax.legend()
ax.grid(True, alpha=0.3)
ax = axes[2]
ax.plot(t_centers, f_vals, "o-", markersize=3, color="purple")
ax.set_xlabel("Time (T0 units)")
ax.set_ylabel("Freq (1/step)")
ax.grid(True, alpha=0.3)
plt.tight_layout()
path = os.path.join(FIG_DIR, "15L_windowed_periodicity.png")
fig.savefig(path, dpi=150)
plt.close(fig)
print(f" Saved: {path}", flush=True)
# Find stable windows
stable_windows = [(t, cv, T) for t, cv, T in zip(t_centers, cv_vals, T_vals) if cv < 0.10]
print(f" Stable windows (CV_T<0.10): {len(stable_windows)}/{n_windows}", flush=True)
return stable_windows
# ============================================================
# Task 3.4: POD projection attractor comparison
# ============================================================
def task_34():
print("=== Task 3.4: POD attractor comparison ===", flush=True)
# Load resampled data for all diameters
data = {}
for diam in [0.75, 1.0, 1.5]:
for kind in ["illusion", "target_cylinder"]:
name = f"{kind}_{diam}L"
d = load_resampled_fields(name)
if d[0] is not None:
data[name] = d
# Build 1.5L POD basis (target + illusion)
name_t = "target_cylinder_1.5L"
name_i = "illusion_1.5L"
ux_t, uy_t = data[name_t]
ux_i, uy_i = data[name_i]
snaps = []
for ux, uy in [(ux_t, uy_t), (ux_i, uy_i)]:
for c in range(ux.shape[0]):
for p in range(ux.shape[1]):
snaps.append(np.concatenate([ux[c, p].ravel(), uy[c, p].ravel()]))
Q = np.column_stack(snaps)
mf, modes, sv, coeffs = compute_pod(Q)
fig, axes = plt.subplots(1, 3, figsize=(15, 4))
for idx, (diam, color) in enumerate([(0.75, "blue"), (1.0, "green"), (1.5, "red")]):
name = f"illusion_{diam}L"
if name not in data:
continue
ux, uy = data[name]
proj_snaps = []
for c in range(ux.shape[0]):
for p in range(ux.shape[1]):
proj_snaps.append(np.concatenate([ux[c, p].ravel(), uy[c, p].ravel()]))
Qp = np.column_stack(proj_snaps)
a = modes[:, :6].T @ (Qp - mf[:, None])
ax = axes[idx]
ax.plot(a[0, :], a[1, :], ".", color=color, markersize=2, alpha=0.5)
ax.plot(a[0, :96], a[1, :96], "-", color=color, alpha=0.3, linewidth=0.5)
ax.set_xlabel("a1")
ax.set_ylabel("a2")
ax.set_title(f"Illusion {diam}L in 1.5L POD basis")
ax.grid(True, alpha=0.3)
ax.set_aspect("equal")
plt.tight_layout()
path = os.path.join(FIG_DIR, "15L_pod_attractor_comparison.png")
fig.savefig(path, dpi=150)
plt.close(fig)
print(f" Saved: {path}", flush=True)
# ============================================================
# Task 3.5: Short-window CCD (if stable windows found)
# ============================================================
def task_35(stable_windows_info):
print("=== Task 3.5: Short-window CCD ===", flush=True)
if len(stable_windows_info) < 3:
print(" Not enough stable windows, skipping short-window CCD", flush=True)
return
sens_i, forc_i, act_i = load_controlled("illusion_1.5L")
sens_t, forc_t = load_target("target_cylinder_1.5L")
# We need fields for CCD — load from resampled
ux_i, uy_i = load_resampled_fields("illusion_1.5L")
ux_t, uy_t = load_resampled_fields("target_cylinder_1.5L")
# Build reference POD (all 4 cycles of target + illusion)
snaps = []
for ux, uy in [(ux_t, uy_t), (ux_i, uy_i)]:
for c in range(ux.shape[0]):
for p in range(ux.shape[1]):
snaps.append(np.concatenate([ux[c, p].ravel(), uy[c, p].ravel()]))
Q = np.column_stack(snaps)
mf, modes, sv, coeffs = compute_pod(Q)
modes_r = modes[:, :6]
# Use 4-cycle resampled data for CCD (as in standard pipeline)
frc_i = np.load(os.path.join(DATA_DIR, "resampled", "illusion_1.5L", "resampled.npz"))["forces"]
frc_t = np.load(os.path.join(DATA_DIR, "resampled", "target_cylinder_1.5L", "resampled.npz"))["forces"]
flat_i = frc_i.reshape(-1, frc_i.shape[-1]).T # (6, 96)
flat_t = frc_t.reshape(-1, frc_t.shape[-1]).T
# Force observable
y_i = np.vstack([flat_i[0] + flat_i[2] + flat_i[4],
flat_i[1] + flat_i[3] + flat_i[5]])
y_t = np.vstack([flat_t[0], flat_t[1]])
# Project illusion fields
proj = []
for c in range(ux_i.shape[0]):
for p in range(ux_i.shape[1]):
proj.append(np.concatenate([ux_i[c, p].ravel(), uy_i[c, p].ravel()]))
Qi = np.column_stack(proj)
a_i = modes_r.T @ (Qi - mf[:, None])
W_i, sig_i, _, z_i, _, _ = compute_reduced_ccd(a_i, y_i[:, :a_i.shape[1]], Q_delay=12)
# Compare with target
proj_t = []
for c in range(ux_t.shape[0]):
for p in range(ux_t.shape[1]):
proj_t.append(np.concatenate([ux_t[c, p].ravel(), uy_t[c, p].ravel()]))
Qt = np.column_stack(proj_t)
a_t = modes_r.T @ (Qt - mf[:, None])
W_t, sig_t, _, z_t, _, _ = compute_reduced_ccd(a_t, y_t[:, :a_t.shape[1]], Q_delay=12)
# Overlap
n = min(W_i.shape[1], W_t.shape[1], 5)
ov = [float(abs(
W_i[:, k] / (np.linalg.norm(W_i[:, k]) + 1e-12) @
W_t[:, k] / (np.linalg.norm(W_t[:, k]) + 1e-12)
)) for k in range(n)]
print(f" 1.5L short-window force-CCD O(target, illusion): O1={ov[0]:.4f}, O2={ov[1]:.4f}", flush=True)
fig, axes = plt.subplots(1, 2, figsize=(12, 4))
# z_1(t) comparison
ax = axes[0]
ax.plot(z_i[0, :], label="Illusion z_1")
ax.plot(z_t[0, :], "--", label="Target z_1")
ax.set_xlabel("Flat sample index")
ax.set_ylabel("CCD temporal coeff z_1")
ax.set_title("1.5L Force-CCD: z_1(t)")
ax.legend()
ax.grid(True, alpha=0.3)
# Sigma decay
ax = axes[1]
ax.semilogy(sig_i, "o-", label="Illusion", markersize=4)
ax.semilogy(sig_t, "s-", label="Target", markersize=4)
ax.set_xlabel("Mode index")
ax.set_ylabel("Singular value")
ax.set_title("1.5L Force-CCD: singular value decay")
ax.legend()
ax.grid(True, alpha=0.3)
path = os.path.join(FIG_DIR, "15L_short_window_ccd.png")
fig.savefig(path, dpi=150)
plt.close(fig)
print(f" Saved: {path}", flush=True)
# ============================================================
# Main
# ============================================================
def main():
print("=" * 60, flush=True)
print("1.5L Auxiliary Analysis", flush=True)
print("=" * 60, flush=True)
task_31()
task_32()
stable_info = task_33()
task_34()
task_35(stable_info)
print("\nDone. All figures saved to", FIG_DIR, flush=True)
if __name__ == "__main__":
main()

13
src/CCD_analysis/output_redux/illusion/meta.json
View File

@ -1,13 +0,0 @@
{
"case": "illusion_1L",
"model": "/home/frank14f/DynamisLab/models/250525/d1a3o14_250525_imit_1L_2U_600S.zip",
"U0": 0.02,
"viscosity": 0.008,
"sample_interval": 600,
"n_infer_steps": 500,
"mean_reward_last100": 0.50427141107983,
"mean_similarity_last100": 0.8369960935939864,
"force_norm_fact": 0.05429763346910477,
"validation_passed": false,
"n_obj": 6
}

25
src/CCD_analysis/output_redux/illusion/norm.json
View File

@ -1,25 +0,0 @@
{
"force_norm_fact": 0.05429763346910477,
"sens_deviation": [
1.893601894378662,
-0.2520896792411804,
1.3097574710845947,
-0.04255330562591553,
1.897708535194397,
0.2153952717781067
],
"sens_norm_fact": [
4.2874250411987305,
5.249192714691162,
1.472514271736145,
7.114207744598389,
4.274000644683838,
5.054762363433838
],
"action_scale": 8.0,
"action_bias": [
0.0,
-2.0,
2.0
]
}

194
src/CCD_analysis/output_redux/illusion/target_harmonics.json
View File

@ -1,194 +0,0 @@
[
{
"dc": 0.02266536364952723,
"amps": [
0.00038379516744314115,
9.514000233530361e-05,
8.349139917453543e-05,
7.792522654399734e-05,
7.433183588746336e-05
],
"freqs": [
0.16,
0.16666666666666669,
0.13333333333333333,
0.26666666666666666,
0.15333333333333335
],
"phases": [
-1.1237148062087887,
1.9611885389112236,
1.6703234092960584,
-0.3613027596994091,
-1.1120847569223562
]
},
{
"dc": 5.844893375372825e-05,
"amps": [
0.008381073411089025,
0.0009710627254597952,
0.0008309252043062449,
0.000440363650103523,
0.0004196318731941633
],
"freqs": [
0.08,
0.08666666666666667,
0.07333333333333333,
0.06666666666666667,
0.09333333333333334
],
"phases": [
-0.20583713241972612,
2.9062320022605928,
-0.19244834395419047,
-0.06340420123785095,
2.797172757014809
]
},
{
"dc": 2.023785818417867,
"amps": [
0.6206119106073832,
0.08359220819570973,
0.0772933966332921,
0.05856013170212723,
0.044579995576176853
],
"freqs": [
0.08,
0.24000000000000002,
0.08666666666666667,
0.07333333333333333,
0.16
],
"phases": [
-1.2902033342248966,
2.247158738027268,
1.8698785846127643,
-1.3230065080948876,
-2.7643028382054635
]
},
{
"dc": -0.07650716103613377,
"amps": [
0.8881293616425711,
0.19361219712346026,
0.11776456319197573,
0.1056433212084406,
0.08416293765029953
],
"freqs": [
0.08,
0.16,
0.24000000000000002,
0.08666666666666667,
0.07333333333333333
],
"phases": [
-3.1077828808645815,
0.9383438720400492,
0.0795160787541243,
0.03419287750174611,
-3.1125200854751167
]
},
{
"dc": 1.8414087001482646,
"amps": [
0.1497365430682559,
0.040132780833895966,
0.026889484355317024,
0.02298740258772072,
0.01741055707849075
],
"freqs": [
0.16,
0.16666666666666669,
0.15333333333333335,
0.32,
0.17333333333333334
],
"phases": [
-0.27142943807213477,
2.865319464251447,
-0.27348869296327505,
-3.041810616039961,
2.803592539777442
]
},
{
"dc": -0.008232745975255966,
"amps": [
1.5054029642098454,
0.30809832902221,
0.17612766731044907,
0.14518758298142478,
0.13409758532624938
],
"freqs": [
0.08,
0.24000000000000002,
0.08666666666666667,
0.07333333333333333,
0.24666666666666667
],
"phases": [
-3.035701468779973,
0.44692729545584287,
0.096649357931439,
-3.035027928495052,
-2.7069943130770233
]
},
{
"dc": 2.0234219272931417,
"amps": [
0.6202491421096321,
0.0839598074044769,
0.0773465705446304,
0.05861231122571406,
0.04506359085306284
],
"freqs": [
0.08,
0.24000000000000002,
0.08666666666666667,
0.07333333333333333,
0.16
],
"phases": [
1.8537647038281104,
-0.8723353223902255,
-1.2906675657274453,
1.8392270400405824,
-2.377576985448929
]
},
{
"dc": 0.06527064591646195,
"amps": [
0.8933136072736296,
0.1854083343143704,
0.1218220099415223,
0.10078312046223797,
0.09011504483851066
],
"freqs": [
0.08,
0.16,
0.24000000000000002,
0.08666666666666667,
0.07333333333333333
],
"phases": [
-3.1049896733668625,
-2.145674517803205,
0.14371115551840652,
0.007693943504592867,
-3.0961205975207657
]
}
]

11
src/CCD_analysis/output_redux/illusion/validation_report.json
View File

@ -1,11 +0,0 @@
{
"case": "illusion",
"output_dir": "/home/frank14f/DynamisLab/src/CCD_analysis/output_redux/illusion",
"checks": {
"reward": false,
"similarity": false,
"vorticity_png": true
},
"all_pass": false,
"extra": {}
}

BIN
src/CCD_analysis/output_redux/illusion/vorticity_controlled.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 207 KiB

BIN
src/CCD_analysis/output_redux/illusion/vorticity_target.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 212 KiB

BIN
src/CCD_analysis/output_redux/illusion/vorticity_uncontrolled.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 241 KiB

13
src/CCD_analysis/output_redux/karman/meta.json
View File

@ -1,13 +0,0 @@
{
"case": "karman_cloak",
"model": "/home/frank14f/DynamisLab/models/old/d1a3o12_re100.zip",
"viscosity": 0.004,
"U0": 0.01,
"sample_interval": 800,
"n_infer_steps": 200,
"mean_reward_last100": 0.6534655690193176,
"overall_similarity": 0.9503773416909905,
"force_norm_fact": 0.019220656715333462,
"validation_passed": true,
"n_obj": 7
}

24
src/CCD_analysis/output_redux/karman/norm.json
View File

@ -1,24 +0,0 @@
{
"force_norm_fact": 0.019220656715333462,
"sens_deviation": [
0.7905515432357788,
-0.11469581723213196,
0.24619626998901367,
0.01082652248442173,
0.8161152601242065,
0.12503524124622345
],
"sens_norm_fact": [
3.1592841148376465,
3.0264341831207275,
1.8399379253387451,
3.451172351837158,
3.055002450942993,
2.9197099208831787
],
"action_bias": [
0.0,
-4.0,
4.0
]
}

15
src/CCD_analysis/output_redux/karman/validation_report.json
View File

@ -1,15 +0,0 @@
{
"case": "karman",
"output_dir": "/home/frank14f/DynamisLab/src/CCD_analysis/output_redux/karman",
"checks": {
"overall_similarity": true,
"mean_similarity_last100": true,
"vorticity_png": true
},
"all_pass": true,
"extra": {
"overall_similarity": 0.9503773416909905,
"mean_reward_last100": 0.6534655690193176,
"mean_sim_last100": 0.9482672810554504
}
}

BIN
src/CCD_analysis/output_redux/karman/vorticity_controlled.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 212 KiB

BIN
src/CCD_analysis/output_redux/karman/vorticity_target.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 233 KiB

BIN
src/CCD_analysis/output_redux/karman/vorticity_uncontrolled.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 265 KiB

11
src/CCD_analysis/output_redux/pinball/meta.json
View File

@ -1,11 +0,0 @@
{
"case": "pinball",
"U0": 0.01,
"viscosity": 0.004,
"n_steps": 200,
"sample_interval": 800,
"n_obj": 6,
"f_dom": 5.6250000000000005e-05,
"T_dom_steps": 17777.777777777777,
"St": 0.11250000000000002
}

BIN
src/CCD_analysis/output_redux/pinball/vorticity.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 203 KiB

211
src/CCD_analysis/output_redux/review_report.json
View File

@ -1,211 +0,0 @@
{
"pinball": {
"meta": {
"case": "pinball",
"U0": 0.01,
"viscosity": 0.004,
"n_steps": 200,
"sample_interval": 800,
"n_obj": 6,
"f_dom": 5.6250000000000005e-05,
"T_dom_steps": 17777.777777777777,
"St": 0.11250000000000002
},
"validation": {},
"files": {
"sensors.npz": {
"exists": true,
"size_mb": 0.01
},
"fields.npz": {
"exists": true,
"size_mb": 890.44
},
"vorticity.png": {
"exists": true,
"size_mb": 0.2
},
"meta.json": {
"exists": true,
"size_mb": 0.0
}
}
},
"steady_cloak": {
"meta": {
"case": "steady_cloak",
"U0": 0.01,
"viscosity": 0.004,
"omega_front": 0.0,
"omega_bottom": 0.051,
"omega_top": -0.051,
"rear_omega_scale": 5.1,
"n_samples": 30,
"sample_interval": 800,
"sensor_mean": [
1.1140856742858887,
-0.01482248492538929,
1.1645309925079346,
3.380884905368475e-08,
1.1140861511230469,
0.014822724275290966
],
"sensor_std": 0.0003435599210206419
},
"validation": {},
"files": {
"sensors.npz": {
"exists": true,
"size_mb": 0.0
},
"fields.npz": {
"exists": true,
"size_mb": 128.28
},
"vorticity.png": {
"exists": true,
"size_mb": 0.09
},
"meta.json": {
"exists": true,
"size_mb": 0.0
}
}
},
"karman": {
"meta": {
"case": "karman_cloak",
"model": "/home/frank14f/DynamisLab/models/old/d1a3o12_re100.zip",
"viscosity": 0.004,
"U0": 0.01,
"sample_interval": 800,
"n_infer_steps": 200,
"mean_reward_last100": 0.6534655690193176,
"overall_similarity": 0.9503773416909905,
"force_norm_fact": 0.019220656715333462,
"validation_passed": true,
"n_obj": 7
},
"validation": {
"case": "karman",
"output_dir": "/home/frank14f/DynamisLab/src/CCD_analysis/output_redux/karman",
"checks": {
"overall_similarity": true,
"mean_similarity_last100": true,
"vorticity_png": true
},
"all_pass": true,
"extra": {
"overall_similarity": 0.9503773416909905,
"mean_reward_last100": 0.6534655690193176,
"mean_sim_last100": 0.9482672810554504
}
},
"files": {
"vorticity_target.png": {
"exists": true,
"size_mb": 0.23
},
"vorticity_controlled.png": {
"exists": true,
"size_mb": 0.21
},
"vorticity_uncontrolled.png": {
"exists": true,
"size_mb": 0.26
},
"meta.json": {
"exists": true,
"size_mb": 0.0
},
"validation_report.json": {
"exists": true,
"size_mb": 0.0
},
"norm.json": {
"exists": true,
"size_mb": 0.0
},
"target.npz": {
"exists": true,
"size_mb": 0.0
},
"save_states.npz": {
"exists": true,
"size_mb": 0.01
},
"controlled.npz": {
"exists": true,
"size_mb": 0.02
},
"open_loop_fields.npz": {
"exists": true,
"size_mb": 900.46
}
}
},
"illusion": {
"meta": {
"case": "illusion_1L",
"model": "/home/frank14f/DynamisLab/models/250525/d1a3o14_250525_imit_1L_2U_600S.zip",
"U0": 0.02,
"viscosity": 0.008,
"sample_interval": 600,
"n_infer_steps": 500,
"mean_reward_last100": 0.50427141107983,
"mean_similarity_last100": 0.8369960935939864,
"force_norm_fact": 0.05429763346910477,
"validation_passed": false,
"n_obj": 6
},
"validation": {
"case": "illusion",
"output_dir": "/home/frank14f/DynamisLab/src/CCD_analysis/output_redux/illusion",
"checks": {
"reward": false,
"similarity": false,
"vorticity_png": true
},
"all_pass": false,
"extra": {}
},
"files": {
"vorticity_target.png": {
"exists": true,
"size_mb": 0.21
},
"vorticity_controlled.png": {
"exists": true,
"size_mb": 0.2
},
"vorticity_uncontrolled.png": {
"exists": true,
"size_mb": 0.24
},
"meta.json": {
"exists": true,
"size_mb": 0.0
},
"validation_report.json": {
"exists": true,
"size_mb": 0.0
},
"norm.json": {
"exists": true,
"size_mb": 0.0
},
"target.npz": {
"exists": true,
"size_mb": 0.0
},
"save_states.npz": {
"exists": true,
"size_mb": 0.01
},
"controlled.npz": {
"exists": true,
"size_mb": 0.03
}
}
}
}

20
src/CCD_analysis/output_redux/steady_cloak/meta.json
View File

@ -1,20 +0,0 @@
{
"case": "steady_cloak",
"U0": 0.01,
"viscosity": 0.004,
"omega_front": 0.0,
"omega_bottom": 0.051,
"omega_top": -0.051,
"rear_omega_scale": 5.1,
"n_samples": 30,
"sample_interval": 800,
"sensor_mean": [
1.1140856742858887,
-0.01482248492538929,
1.1645309925079346,
3.380884905368475e-08,
1.1140861511230469,
0.014822724275290966
],
"sensor_std": 0.0003435599210206419
}

BIN
src/CCD_analysis/output_redux/steady_cloak/vorticity.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 91 KiB

101
src/CCD_analysis/scripts/cfg.py
View File

@ -1,101 +0,0 @@
# CCD_analysis/scripts/cfg.py
# RELIABILITY: HIGH. Paths and constants only, no CFD dependency.
"""Configuration constants for CCD analysis pipeline."""
import os
# -- Paths -------------------------------------------------------------------
_PROJ_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
ANALYSIS_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
CONFIG_DIR = os.path.join(ANALYSIS_DIR, "configs")
OUTPUT_DIR = os.path.join(ANALYSIS_DIR, "output")
MODEL_DIR = os.path.join(_PROJ_ROOT, "models")
LEGACY_CFD_DIR = os.path.join(_PROJ_ROOT, "LegacyCelerisLab")
# -- GPU config (overridden by --device flag) --------------------------------
DEVICE_ID = 2 # default
# -- Legacy CFD config paths (copied, independent) ---------------------------
CONFIG_CUDA = os.path.join(CONFIG_DIR, "config_cuda.json")
CONFIG_FLOWFIELD_BASE = os.path.join(CONFIG_DIR, "config_flowfield.json")
# -- Physics constants -------------------------------------------------------
U0 = 0.01 # inlet centre velocity (lattice units)
D_CYL = 20.0 # single cylinder diameter (lattice units)
D_REF = 40.0 # reference length = 2*D (used for code "Re")
L0 = 20.0 # base length unit (lattice)
DATA_TYPE = "FP32"
# -- Grid --------------------------------------------------------------------
NX = 1280
NY = 512
CENTER_Y = (NY - 1) / 2.0 # 255.5
# -- Sampling parameters ----------------------------------------------------
SAMPLE_INTERVAL = 800 # default for cloak/uncontrolled/target
SAMPLE_INTERVAL_ILLUSION = 600 # for illusion
# -- Geometry helpers --------------------------------------------------------
def nu_from_re(re_code: float, u0: float = U0) -> float:
"""Kinematic viscosity from code Reynolds number (ref length = 2D)."""
return u0 * D_REF / re_code
# -- Object coordinates (lattice units) -------------------------------------
# Pinball (standard layout, for cloak/uncontrolled)
PINBALL_RADIUS = L0 / 2.0
FRONT_CENTER = (30.0 * L0, CENTER_Y) # (600, 255.5)
BOTTOM_CENTER = (31.3 * L0, CENTER_Y - 0.75 * L0) # (626, 240.5)
TOP_CENTER = (31.3 * L0, CENTER_Y + 0.75 * L0) # (626, 270.5)
# Pinball (illusion layout — different positions)
ILLUSION_FRONT = (19.0 * L0, CENTER_Y) # (380, 255.5)
ILLUSION_BOTTOM = (20.3 * L0, CENTER_Y + 0.75 * L0) # (406, 270.5)
ILLUSION_TOP = (20.3 * L0, CENTER_Y - 0.75 * L0) # (406, 240.5)
# Sensors
SENSOR_RADIUS = L0 / 4.0 # 5
SENSOR_CENTERS_CLOAK = [ # x=40*L0 for cloak/uncontrolled
(40.0 * L0, CENTER_Y + 2.0 * L0),
(40.0 * L0, CENTER_Y),
(40.0 * L0, CENTER_Y - 2.0 * L0),
]
SENSOR_CENTERS_ILLUSION = [ # x=30*L0 for illusion
(30.0 * L0, CENTER_Y + 2.0 * L0),
(30.0 * L0, CENTER_Y),
(30.0 * L0, CENTER_Y - 2.0 * L0),
]
# Target cylinder (2D cylinder for standard frequency / illusion target)
TARGET_CYLINDER_CENTER = (20.0 * L0, CENTER_Y) # (400, 255.5)
TARGET_CYLINDER_RADIUS = 1.0 * L0 # 20
# -- Model paths -------------------------------------------------------------
MODEL_CLOAK_RE100 = os.path.join(MODEL_DIR, "old", "d1a3o12_re100.zip")
MODEL_CLOAK_250326 = os.path.join(MODEL_DIR, "250326", "d1a3o12_250326.zip")
MODEL_ILLUSION_1L = os.path.join(
MODEL_DIR, "250525", "d1a3o14_250525_imit_1L_2U_600S.zip"
)
# -- Action parameters -------------------------------------------------------
ACTION_SCALE_CLOAK = 8.0
ACTION_BIAS_CLOAK = (0.0, -4.0, 4.0)
ACTION_SCALE_ILLUSION = 8.0
ACTION_BIAS_ILLUSION = (0.0, -2.0, 2.0)
# -- DRL parameters ----------------------------------------------------------
S_DIM_CLOAK = 12
S_DIM_ILLUSION = 14
A_DIM = 3
FIFO_LEN = 150
CONV_LEN = 30
STABILIZE_STEPS = int(4 * NX / U0)
# -- Phase resampling --------------------------------------------------------
N_TARGET_CYCLES = 4 # number of cycles to extract
N_PTS_PER_CYCLE = 24 # phase points per cycle
TOTAL_PHASE_FRAMES = N_TARGET_CYCLES * N_PTS_PER_CYCLE # 96
# -- CCD parameters ----------------------------------------------------------
CCD_Q_DEFAULT = 12 # delay window size (half-cycle)
CCD_R_CANDIDATES = [6, 8, 10] # POD truncation candidates

12
src/CCD_analysis/scripts/collect_illusion.py
View File

@ -65,10 +65,10 @@ def run_single(scene_name: str, device_id: int, n_steps: int) -> dict:
print("=== Target recording ===") print("=== Target recording ===")
ff_tgt = FlowField(field_cfg, cuda_cfg, device_id=device_id) ff_tgt = FlowField(field_cfg, cuda_cfg, device_id=device_id)
tgt_radius = cfg["target_diameter"] * L0 tgt_radius = cfg["target_diameter"] * L0
ff_tgt.add_cylinder((20.0 * L0, CENTER_Y, 0.0), tgt_radius) ff_tgt.add_cylinder((30.65 * L0, CENTER_Y, 0.0), tgt_radius) # UNIFIED: was 20.0
print(f" target cylinder: diameter={cfg['target_diameter']}L, radius={tgt_radius}", flush=True) print(f" target cylinder: diameter={cfg['target_diameter']}L, radius={tgt_radius}", flush=True)
for y_off in [2.0, 0.0, -2.0]: for y_off in [2.0, 0.0, -2.0]:
ff_tgt.add_sensor((30.0 * L0, CENTER_Y + y_off * L0, 0.0), L0 / 4.0) ff_tgt.add_sensor((40.0 * L0, CENTER_Y + y_off * L0, 0.0), L0 / 4.0) # UNIFIED: was 30.0
n_tgt = 4 n_tgt = 4
ff_tgt.run(int(4 * 1280 / u0), np.zeros(n_tgt, dtype=DATA_TYPE)) ff_tgt.run(int(4 * 1280 / u0), np.zeros(n_tgt, dtype=DATA_TYPE))
@ -91,10 +91,10 @@ def run_single(scene_name: str, device_id: int, n_steps: int) -> dict:
print("=== Pinball env + norm ===") print("=== Pinball env + norm ===")
ff = FlowField(field_cfg, cuda_cfg, device_id=device_id) ff = FlowField(field_cfg, cuda_cfg, device_id=device_id)
for y_off in [2.0, 0.0, -2.0]: for y_off in [2.0, 0.0, -2.0]:
ff.add_sensor((30.0 * L0, CENTER_Y + y_off * L0, 0.0), L0 / 4.0) ff.add_sensor((40.0 * L0, CENTER_Y + y_off * L0, 0.0), L0 / 4.0) # UNIFIED: was 30.0
ff.add_cylinder((19.0 * L0, CENTER_Y, 0.0), L0 / 2.0) ff.add_cylinder((30.0 * L0, CENTER_Y, 0.0), L0 / 2.0) # UNIFIED: was 19.0
ff.add_cylinder((20.3 * L0, CENTER_Y + 0.75 * L0, 0.0), L0 / 2.0) ff.add_cylinder((31.3 * L0, CENTER_Y + 0.75 * L0, 0.0), L0 / 2.0) # UNIFIED: was 20.3
ff.add_cylinder((20.3 * L0, CENTER_Y - 0.75 * L0, 0.0), L0 / 2.0) ff.add_cylinder((31.3 * L0, CENTER_Y - 0.75 * L0, 0.0), L0 / 2.0) # UNIFIED: was 20.3
n_env = 6 n_env = 6
ff.run(int(4 * 1280 / u0), np.zeros(n_env, dtype=DATA_TYPE)) ff.run(int(4 * 1280 / u0), np.zeros(n_env, dtype=DATA_TYPE))

150
src/CCD_analysis/steady/run_steady.py
View File

@ -1,150 +0,0 @@
"""Steady cloak metrics (mean flow error, recirculation zone, fluctuation suppression).
Uses empty channel reference from target_channel for E_mean computation.
Usage:
python steady/run_steady.py
"""
from __future__ import annotations
import json
import os
import sys
import numpy as np
_SRC = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from CCD_analysis.configs import DATA_DIR
def load_fields(d: str):
p = os.path.join(d, "fields.npz")
if not os.path.isfile(p):
return None
f = np.load(p)
return f["ux"].astype(np.float64), f["uy"].astype(np.float64)
def main():
print("=== Steady Cloak Metrics ===\n", flush=True)
steady_dir = os.path.join(DATA_DIR, "steady_cloak", "steady_cloak")
pinball_dir = os.path.join(DATA_DIR, "pinball", "pinball")
empty_chan_dir = os.path.join(DATA_DIR, "target_channel", "target_channel")
sc = load_fields(steady_dir)
if sc is None:
print("ERROR: no steady cloak fields. Run scripts/collect_steady_cloak.py first.", flush=True)
return 1
ec = load_fields(empty_chan_dir)
if ec is None:
print("ERROR: no empty channel fields. Run scripts/collect_empty_channel.py first.", flush=True)
return 1
ux_s, uy_s = sc
ux_e, uy_e = ec
# Time means
ux_s_mean = np.mean(ux_s, axis=0)
uy_s_mean = np.mean(uy_s, axis=0)
ux_e_mean = np.mean(ux_e, axis=0)
uy_e_mean = np.mean(uy_e, axis=0)
# RMS fluctuations
ux_s_rms = np.sqrt(np.mean((ux_s - ux_s_mean) ** 2, axis=0))
uy_s_rms = np.sqrt(np.mean((uy_s - uy_s_mean) ** 2, axis=0))
total_rms = float(np.sqrt(np.mean(ux_s_rms**2 + uy_s_rms**2)))
print(f"1. Total RMS (steady cloak): {total_rms:.6f}", flush=True)
# === E_mean: mean flow error vs empty channel ===
# Use total velocity norm to avoid division by near-zero uy
vel_norm = np.sqrt(np.mean(ux_e_mean**2 + uy_e_mean**2)) + 1e-12
diff_ux = ux_s_mean - ux_e_mean
diff_uy = uy_s_mean - uy_e_mean
E_mean_ux = float(np.sqrt(np.mean(diff_ux**2)) / vel_norm)
E_mean_uy = float(np.sqrt(np.mean(diff_uy**2)) / vel_norm)
E_mean_total = float(np.sqrt(np.mean(diff_ux**2 + diff_uy**2)) / vel_norm)
print(f"2. E_mean (vs empty channel, normalized by |U_e|): ux={E_mean_ux:.4f}, uy={E_mean_uy:.4f}, total={E_mean_total:.4f}", flush=True)
# === Recirculation zone ===
ny, nx = ux_s_mean.shape
cline = ux_s_mean[ny // 2, :]
neg = np.where(cline[400:-50] < 0)[0]
if len(neg) > 0:
L_r = float(neg[-1])
print(f"3. Recirculation length L_r: {L_r:.0f} lattice units", flush=True)
else:
L_r = 0.0
print(f"3. Recirculation length L_r: 0 (no reverse flow)", flush=True)
# Recirculation area: count pixels with ux < 0 in the mean field downstream of pinball
recirc_mask = (ux_s_mean < 0) & (np.arange(nx) > 400)
A_r = int(np.sum(recirc_mask))
print(f" Recirculation area A_r: {A_r} pixels", flush=True)
# === Sensor mean restoration ===
def load_sensor_mean(path):
sp = os.path.join(path, "sensors.npz")
if not os.path.isfile(sp):
return None
sd = np.load(sp)
return np.mean(sd["sensors"], axis=0)
sens_s = load_sensor_mean(steady_dir)
sens_e = load_sensor_mean(empty_chan_dir)
if sens_s is not None and sens_e is not None:
diff_sens = sens_s[:6] - sens_e[:6]
print(f"4. Sensor mean error (||s_cloak - s_channel||): {np.linalg.norm(diff_sens):.4f}", flush=True)
print(f" Per-component error: {[f'{v:.4f}' for v in diff_sens]}", flush=True)
else:
print("4. Sensor comparison: unavailable", flush=True)
# === Fluctuation suppression vs uncontrolled pinball ===
pb = load_fields(pinball_dir)
fluc_suppression = None
if pb is not None:
ux_p, uy_p = pb
ux_p_mean = np.mean(ux_p, axis=0)
uy_p_mean = np.mean(uy_p, axis=0)
ux_p_rms = np.sqrt(np.mean((ux_p - ux_p_mean) ** 2, axis=0))
uy_p_rms = np.sqrt(np.mean((uy_p - uy_p_mean) ** 2, axis=0))
pb_total_rms = float(np.sqrt(np.mean(ux_p_rms**2 + uy_p_rms**2)))
print(f"5. Pinball total RMS: {pb_total_rms:.6f}", flush=True)
if pb_total_rms > 1e-12:
fluc_suppression = (1 - total_rms / pb_total_rms) * 100
print(f" Fluctuation suppression: {fluc_suppression:.1f}%", flush=True)
# === Control cost ===
sens_path = os.path.join(steady_dir, "sensors.npz")
if os.path.isfile(sens_path):
sd = np.load(sens_path)
forces = sd.get("forces")
if forces is not None and forces.shape[1] >= 6:
omega_rms = float(np.sqrt(np.mean(forces ** 2)))
print(f"6. RMS forces (control effort proxy): {omega_rms:.6f}", flush=True)
results = {
"total_rms": total_rms,
"E_mean_ux": E_mean_ux,
"E_mean_uy": E_mean_uy,
"E_mean_total": E_mean_total,
"L_r": L_r,
"A_r": A_r,
"fluctuation_suppression_pct": fluc_suppression,
}
out_dir = os.path.join(DATA_DIR, "steady")
os.makedirs(out_dir, exist_ok=True)
with open(os.path.join(out_dir, "steady_metrics.json"), "w") as f:
json.dump(results, f, indent=2)
print(f"\nSaved to {out_dir}/steady_metrics.json", flush=True)
return 0
if __name__ == "__main__":
sys.exit(main())

287
src/SR_analysis/README.md
View File

@ -1,20 +1,78 @@
# SR_analysis: Unified SINDy-SR Analysis Pipeline # SR_analysis: Symbolic Regression Analysis Pipeline
## Overview Extracts interpretable control laws (`obs -> act`) from DRL-trained policies for the
fluidic pinball. Uses **PySR symbolic regression** on dimensionless physical features with
G-equivariant structural constraints (v23: front no-bias, rear shared-head).
This directory consolidates the SINDy-and-symbolic-regression analysis pipeline ## Current Results (2026-06-25)
for the DynamisLab fluidic pinball project. It replaces the old
`src/analysis_crossre/` and `src/analysis_cloak/` directories with a unified
structure.
The pipeline fits **sparse interpretable control laws** (`obs -> act`) for all ### Karman Cloak — Cross-Re Unified Formula
cloak and illusion scenes, using dimensionless physical features,
G-equivariant structural constraints, and STLSQ threshold grids.
For background, see: | Scene | Front Formula | Top Formula | CFD Closed-Loop |
- `sindy_sr_notes.md` -- execution plan and task tracking |-------|--------------|-------------|:---------------:|
- `sindy_sr_knowledge.md` -- confirmed facts and known pitfalls | Joint (Re50-400) | `daF_dt - 14.952*mu*Cl_tot` | `alpha_T = 3.414` (const) | **0.847 avg** |
- `../../docs/SR_analysis_results.md` -- comprehensive results report | Re50 independent | PySR per-Re best | — | **0.895** |
| Re100 independent | PySR per-Re best | — | **0.888** |
| Re200 independent | PySR per-Re best | — | **0.916** |
| Re400 (SI=400 opt) | Joint formula | Joint formula | **0.819** |
### Illusion
| Scene | Front Formula | CFD Closed-Loop | % of PPO |
|-------|--------------|:---------------:|:--------:|
| 0.75L | `-0.169*(Cl_tot + dCl_tot_dt) - 1.240` | **0.979** | 100.7% |
| 1L | `(du_a_dt + u_a + 26.5)*0.0123` | **0.957** | 98.4% |
| **Joint (0.75L+1L)** | `target_Cd - 5.428 + 0.0098*(du_a_dt + u_a)` | **0.978 / 0.970** | — |
| 1.5L | High-freq periodic modulation (not SR-amenable) | — | — |
**Key finding**: 0.75L and 1L formulas have fundamentally different skeletons (Cl_tot vs u_a
dominant). Joint formula still achieves excellent CFD results on both although the underlying
mechanisms differ.
### Illusion Generalization (Joint Formula, No PPO)
| Diameter | Similarity | Notes |
|:--------:|:----------:|-------|
| 0.5L | 0.854 | Signal weak, noise-dominated |
| 0.6L | **0.939** | Generalizes well |
| 0.8L | **0.908** | Generalizes well |
| 1.2L | 0.849 | Begins to degrade |
| 1.5L | N/A | High-frequency regime, different mechanism |
| 2.0L | 0.676 | Degraded, near 1.5L regime |
Valid range: 0.6L-1.0L (similarity > 0.90).
### Vortex Cloak (Generalization)
Karman joint formula tested on vortex scenes (no retraining):
| Scene | Karman Joint Formula | PPO Baseline |
|-------|:-------------------:|:------------:|
| vortex_lamb | **0.949** | 0.942 |
| vortex_taylor | **0.905** | 0.916 |
---
## Pipeline Overview
```
controlled.npz (PPO rollout)
|
v
compute_features() --> dimensionless physics features (ILLUSION_PHASE_KEYS, etc.)
|
v
PySR symbolic regression --> sparse interpretable formulas
|
v
CFD closed-loop validation --> final similarity score
```
### Key Design Decisions
1. **Feature levels**: Static (8-dim) -> Phase-state (6-dim) -> Illusion-phase (10-dim)
2. **Output target**: Non-dimensional alpha, not physical omega
3. **v23 structure**: Front no-bias, rear shared-head (Bottom = -Top(Gx))
4. **Final judge**: CFD closed-loop similarity, not one-step R2
--- ---
@ -22,164 +80,105 @@ For background, see:
``` ```
SR_analysis/ SR_analysis/
configs.py # Unified scene metadata (all 10+ scenes) configs.py # Scene metadata (Karman, Illusion, Vortex)
configs/ configs/legacy/ # Legacy CFD configs (config_cuda.json, config_flowfield.json)
legacy/ # Legacy CFD configs
utils/ utils/
__init__.py # Selective exports (no pycuda dependency) __init__.py # Exports (no pycuda dependency)
feature_builder.py # Dimensionless features + G-operator + phase-state features feature_builder.py # Dimensionless features, G-operator, phase-state features
sindy_fitter.py # STLSQ + feature matrices + derivative/absolute modes sindy_fitter.py # STLSQ fitting + feature matrices
cfd_interface.py # LegacyCelerisLab wrapper (requires pycuda_3_10) cfd_interface.py # LegacyCelerisLab wrapper (requires pycuda_3_10)
g_operator.py # Equivariance diagnostics g_operator.py # Equivariance diagnostics
data/ data/ # Inference output data (controlled.npz, target.npz)
karman/ # Karman cloak: karman_re50/100/200/400 karman/ karman_re50..400/
steady/ # Steady cloak illusion/ illusion_0.75L,1L,1.5L/
illusion/ # Illusion: illusion_0.75L/1L/1.5L vortex/ vortex_lamb,taylor/
vortex/ # Vortex cloak
scripts/ scripts/
infer_karman.py # Inference: LegacyCFD + PPO -> controlled.npz infer_karman.py # PPO inference -> controlled.npz
infer_illusion.py # Inference for illusion scenes infer_illusion.py # PPO inference -> controlled.npz
infer_vortex.py # Inference for vortex scenes infer_vortex.py # PPO inference -> controlled.npz
gen_illusion_target.py # Target data generation for generalization scenes
visualize_ppo_illusion.py# PPO visualization with vorticity
sindy/ sindy/
run_all_v2.py # Unified SINDy fitting (supports --deriv, --phase, --output-mode etc.) run_pysr.py # PySR symbolic regression (niter=40)
run_pysr.py # Restricted PySR symbolic regression run_pysr_deep.py # Karman deep PySR (niter=120, Re independent + joint)
wrap_joint.py # Joint model -> wrapped format for validator run_pysr_deep_illusion.py# Illusion deep+joint PySR (niter=120)
compare_v2.py # Cross-scene comparison report
karman/illusion/vortex/ # SINDy output JSONs
validate/ validate/
run_closed_loop.py # Karman closed-loop validator (v23/deriv/abs modes) run_closed_loop.py # Karman closed-loop validator
run_closed_loop_illusion.py # Illusion closed-loop validator run_closed_loop_illusion.py # Illusion closed-loop validator
run_closed_loop_vortex.py # Vortex closed-loop validator
run_closed_loop_re400_si.py # Karman re400 short-SI validator
predict_pysr.py # PySR formula sympy.lambdify wrapper
eval_rollout.py # Offline multi-step rollout evaluation eval_rollout.py # Offline multi-step rollout evaluation
results/ # Validation result JSONs launch_pysr_validation.py # Batch CFD validation launcher
compare/ batch_illusion_generalization.sh# Batch generalization CFD validation
support_overlap.py # Support set comparison results/ # 136 JSON files — canonical + intermediate
shared_core.py # Shared core detection results/README.md # Result file reference table
results/archive/ # Archived intermediate search attempts
``` ```
--- ---
## Key Design Decisions ## Usage
### 1. Scene Metadata Driven ### PySR Symbolic Regression (conda: sr_env)
All scene parameters defined once in `configs.py`.
### 2. Feature Levels
| Level | Features | Dim | Description |
|-------|----------|:---:|-------------|
| Static | u_m, u_a, u_c, v_a, Cd_tot, Cd_rear, Cl_tot, Cl_diff | 8 | Current-step only |
| **Phase-state** | u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear | **6** | Oscillation phase + rate |
| Illusion-phase | Phase-state + Cd_err, Cl_err, dCd_err/dt, dCl_err/dt | **10** | Phase + error-state |
| Karman-expanded | Phase-state + u_m, u_c, v_a, Cl_diff | **10** | Phase + supplementary |
| Full-lag | Static + lag-1 | 16 | Full temporal context |
### 3. Output Modes
- **deriv**: predict `d(alpha)/dt`, then `alpha(t) = alpha(t-1) + dt_c * dalpha/dt`
- **absolute**: predict `alpha(t)` directly (no integration drift)
### 4. G-Equivariant Structure (v23)
```
Front(t) = f_front(x(t)) # no bias, odd under G
Top(t) = f_rear(x(t)) # with bias
Bottom(t) = -f_rear(G[x(t)]) # shared-head
```
---
## Current Best Results (2026-06-15)
### Illusion — New Route: Phase-state + Error-state + Absolute Action
| Scene | Closed-loop | % of PPO | Action history? | Features |
|-------|:----------:|:--------:|:---------------:|----------|
| 0.75L | **0.974** | 100.2% | **No** | ILLUSION_PHASE (10dim) |
| 1L | **0.958** | 98.5% | **No** | ILLUSION_PHASE (10dim) |
| 1.5L | N/A | — | **No** | Bang-bang regime |
### Karman re100 — Ablation
| Config | Feat | Output | R2 | Closed-loop | Note |
|--------|:----:|:-----:|:--:|:----------:|------|
| old v23 (a_lag) | 14+3 | alpha | 0.996 | **0.901** | Baseline |
| **Phase->abs** | **6** | **alpha** | **0.965** | **0.699** | Best new route |
| Phase->deriv | 6 | dalpha/dt | 0.837 | 0.656 | |
| Phase+mu->abs | 9 | alpha | 0.979 | 0.700 | mu helps cross-Re |
| Expanded->abs | 10 | alpha | 0.980 | 0.580 | Overfitting |
---
## Commands
All from repo root (`/home/frank14f/DynamisLab`).
### SINDy Fitting
```bash ```bash
# Illusion phase-state + absolute (recommended for 0.75L/1L) # Illusion
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \ conda run -n sr_env python src/SR_analysis/sindy/run_pysr_deep_illusion.py --individual
--scenes illusion_0.75L,illusion_1L --deriv --phase --output-mode absolute
# Karman phase-state + absolute # Karman deep (cross-Re independent + joint)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \ conda run -n sr_env python src/SR_analysis/sindy/run_pysr_deep.py --both
--scenes karman_re100 --deriv --phase --output-mode absolute
# Karman expanded (10 dim)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes karman_re100 --deriv --karman-expand --output-mode absolute
# Karman with mu modulation
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes karman_re100 --deriv --karman-mu --output-mode absolute
# Old-style (v2, with action history)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes karman_re50,karman_re100 --joint
``` ```
### Closed-loop Validation ### CFD Closed-Loop Validation (conda: pycuda_3_10, GPU 1 or 2)
```bash ```bash
# Karman with absolute action # Illusion PySR formula
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scene karman_re100 --device 0 --steps 200 --mode abs \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json
# Karman old v23
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scene karman_re100 --device 0 --steps 200 --mode v23 \
--sindy-results src/SR_analysis/sindy/karman/sindy_joint_wrapped.json
# Illusion with absolute action
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop_illusion.py \ conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop_illusion.py \
--scene illusion_1L --device 0 --steps 320 \ --scene illusion_1L --device 2 --steps 320 --mode pysr \
--sindy-results src/SR_analysis/sindy/illusion/sindy_results_deriv.json --pysr-front validate/results/pysr_illusion_1L_front.json \
--pysr-top validate/results/pysr_illusion_1L_top.json
# Karman joint formula
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scene karman_re100 --device 2 --steps 200 --mode pysr \
--pysr-front validate/results/karman_joint_deep_front.json \
--pysr-top validate/results/karman_joint_deep_top.json
# Vortex (generalization test)
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop_vortex.py \
--scene vortex_lamb --device 2 --steps 150 --mode pysr \
--pysr-front validate/results/karman_joint_deep_front.json \
--pysr-top validate/results/karman_joint_deep_top.json
``` ```
### PySR Symbolic Regression ### PPO Inference (generate controlled.npz)
```bash ```bash
conda run -n sr_env python src/SR_analysis/sindy/run_pysr.py --scene illusion_1L conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_karman.py --re 100 --device 2
``` conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_illusion.py --diameter 1.0 --device 2
conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_vortex.py --type lamb --device 2
### Offline Rollout Evaluation
```bash
python3 src/SR_analysis/validate/eval_rollout.py \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json \
--scene karman_re100
``` ```
--- ---
## Important Reminders ## Critical Reminders
- `controlled.npz` actions are **normalized [-1,1]** — must convert via `(norm * scale + bias) * u0` - **actions.npz are normalized [-1,1]**, not physical omega. Convert: `(action * scale + bias) * u0`
- **FIFO bias ≠ DRL action bias** for Illusion: FIFO=[0, -0.01, 0.01], decode=[0, -0.02, 0.02] - **PySR needs `sensors_raw`/`forces_raw`** passed to `compute_features()` or derivative features are zero
- "2U" in model name = S_DIM=14 (not 2x velocity), u0 always 0.01 - **Output target must be alpha** (non-dim): `Y = actions_phys / u0`
- SAMPLE_INTERVAL: 0.75L=400, 1L=600, 1.5L/Karman=800 - **One-step R2 high != closed-loop good** -- always validate in CFD
- Closed-loop steps auto-set: S=400→320, S=600→214, S=800→160 - **Controls must propagate**: steps >= NX/u0/SI (S=400->320, S=600->214, S=800->160)
- One-step R² high ≠ closed-loop good — always validate - **FIFO bias != DRL action bias** for Illusion: FIFO=[0,-U0,U0], decode=[0,-2,2]*U0
- For phase-state features, always pass `sensors_raw`/`forces_raw` to enable derivative computation - **Joint formula must be manually reviewed** for spurious terms (e.g. `daB_dt` is constant=0 at deployment)
---
## Key Documentation
| File | Content |
|------|---------|
| `src/SR_analysis/sindy_sr_knowledge.md` | Background knowledge, bug history, known pitfalls (for coder reference) |
| `src/SR_analysis/sindy_sr_notes.md` | Task list, phase breakdown, current status |
| `docs/SR_analysis_report.md` | **Single consolidated report** — all formulas, results, methodology, structural analysis |
| `docs/illusion_joint_formula_analysis.md` | Illusion joint formula deep dive — physical interpretation, generalization curve |

152
src/SR_analysis/compare/shared_core.py
View File

@ -1,152 +0,0 @@
"""Shared core detection across scenes.
Finds features that are active across ALL scenes in a group (e.g. all Karman Re,
all Illusion diameters) and identifies the cross-scene shared core.
Usage:
python compare/shared_core.py --sindy-results sindy/karman/sindy_results.json \\
--scenes karman_re50 karman_re100 karman_re200 karman_re400
python compare/shared_core.py \\
--sindy-results sindy/results.json \\
--scenes karman_re100 illusion_1L vortex_lamb steady
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from typing import Dict, List, Tuple
import numpy as np
_REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
if _REPO not in sys.path:
sys.path.insert(0, _REPO)
_SRC = os.path.join(_REPO, "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from SR_analysis.utils.sindy_fitter import get_active_support
RELATIVE_THRESHOLD = 0.02
def feat_group(name: str) -> str:
if name == "bias":
return "bias"
if name in ("u_m", "u_a", "u_c", "v_a", "sin_ua", "cos_ua"):
return "sensor"
if name.startswith("Cd") or name.startswith("Cl"):
return "force"
if "lag1" in name:
return "memory_lag"
if name.startswith("da"):
return "memory_delta"
if name == "mu" or name.startswith("mu_"):
return "mu_mod"
return "other"
def detect_core(scene_data: Dict[str, dict], channels: List[str],
threshold: float) -> dict:
"""Find features active in ALL scenes for each channel."""
scene_names = list(scene_data.keys())
results = {}
for ch_name in channels:
fn_key = f"feature_names_{'front' if ch_name == 'front' else 'rear'}"
# Collect active sets per scene
active_per_scene = {}
for sn in scene_names:
fn = scene_data[sn][fn_key]
coef = scene_data[sn][ch_name]["best_coef"]
active = get_active_support(np.array(coef, dtype=np.float64)[:len(fn)],
fn, threshold)
active_per_scene[sn] = set(active.keys())
# Intersection = shared core
core_keys = set.intersection(*active_per_scene.values()) if active_per_scene else set()
# Union for scene-specific detection
all_keys = set.union(*active_per_scene.values()) if active_per_scene else set()
scene_specific = {}
for sn in scene_names:
others = set.union(*[v for k, v in active_per_scene.items() if k != sn])
diff = active_per_scene[sn] - others
if diff:
scene_specific[sn] = sorted(diff)
# Coef means for core features
core_coefs = {}
for k in sorted(core_keys):
vals = []
for sn in scene_names:
fn = scene_data[sn][fn_key]
coef = scene_data[sn][ch_name]["best_coef"]
if k in fn:
idx = fn.index(k)
vals.append(float(coef[idx]))
core_coefs[k] = {
"mean": float(np.mean(vals)),
"std": float(np.std(vals)),
"per_scene": {sn: vals[i] for i, sn in enumerate(scene_names)},
}
results[ch_name] = {
"n_scenes": len(scene_names),
"n_core": len(core_keys),
"core_features": {k: {"group": feat_group(k), "coef": v}
for k, v in core_coefs.items()},
"scene_specific": {sn: sorted(v) for sn, v in scene_specific.items()},
}
return results
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--sindy-results", type=str, required=True)
ap.add_argument("--scenes", type=str, nargs="+", required=True)
ap.add_argument("--threshold", type=float, default=RELATIVE_THRESHOLD)
ap.add_argument("--out", type=str, default=None)
args = ap.parse_args()
with open(args.sindy_results) as f:
all_data = json.load(f)
per = all_data.get("per_scene", {})
scene_data = {sn: per[sn] for sn in args.scenes if sn in per}
if len(scene_data) < 2:
print(f"Need >=2 scenes. Found: {list(scene_data.keys())}")
return 1
print(f"Shared Core Detection: {len(scene_data)} scenes")
for sn in scene_data:
print(f" {sn}")
print(f" threshold={args.threshold}")
results = detect_core(scene_data, ["front", "top", "bottom"], args.threshold)
for ch_name, ch_data in results.items():
print(f"\n--- {ch_name} ---")
print(f" Core features: {ch_data['n_core']}")
for k, v in ch_data["core_features"].items():
c = v["coef"]
print(f" {k:20s} mean={c['mean']:+.6f} std={c['std']:.6f} [{v['group']}]")
for sn, keys in ch_data["scene_specific"].items():
if keys:
print(f" {sn} specific: {', '.join(keys)}")
if args.out:
output = {"scenes": args.scenes, "threshold": args.threshold,
"channels": results}
os.makedirs(os.path.dirname(args.out), exist_ok=True)
with open(args.out, "w") as f:
json.dump(output, f, indent=2)
print(f"\nSaved: {args.out}")
if __name__ == "__main__":
main()

158
src/SR_analysis/compare/support_overlap.py
View File

@ -1,158 +0,0 @@
"""Cross-scene support overlap analysis.
Compares SINDy support sets across scenes at a given relative threshold.
Usage:
python compare/support_overlap.py --sindy-results sindy/karman/sindy_results.json \\
--scenes karman_re100 karman_re200 illusion_1L vortex_lamb
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from typing import Dict, List, Tuple
import numpy as np
_REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
if _REPO not in sys.path:
sys.path.insert(0, _REPO)
_SRC = os.path.join(_REPO, "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from SR_analysis.utils.sindy_fitter import get_active_support
RELATIVE_THRESHOLD = 0.02 # default: 2% of max coefficient
def load_sindy_scenes(sindy_path: str, scenes: List[str]) -> dict:
"""Load sindy results for the given scene names."""
with open(sindy_path) as f:
data = json.load(f)
result = {}
for sn in scenes:
per = data["per_scene"].get(sn)
if per is None:
print(f"WARNING: {sn} not found in {sindy_path}")
continue
result[sn] = per
return result
def feat_group(name: str) -> str:
"""Classify a feature into a group."""
if name == "bias":
return "bias"
if name in ("u_m", "u_a", "u_c", "v_a", "sin_ua", "cos_ua"):
return "sensor"
if name.startswith("Cd") or name.startswith("Cl"):
return "force"
if "lag1" in name:
return "memory_lag"
if name.startswith("da"):
return "memory_delta"
if name == "mu" or name.startswith("mu_"):
return "mu_mod"
return "other"
def classify(
a_active: Dict[str, float],
b_active: Dict[str, float],
) -> Tuple[List[Tuple[str, float, float]], List[Tuple[str, float]], List[Tuple[str, float]]]:
"""Classify features as shared, A-only, B-only.
Returns (shared, A_only, B_only) where shared has feature name + both coeffs.
"""
a_keys = set(a_active.keys())
b_keys = set(b_active.keys())
shared = sorted(a_keys & b_keys)
a_only = sorted(a_keys - b_keys)
b_only = sorted(b_keys - a_keys)
shared_out = [(k, a_active[k], b_active[k]) for k in shared]
a_out = [(k, a_active[k]) for k in a_only]
b_out = [(k, b_active[k]) for k in b_only]
return shared_out, a_out, b_out
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--sindy-results", type=str, required=True)
ap.add_argument("--scenes", type=str, nargs="+", required=True,
help="Scene names to compare")
ap.add_argument("--threshold", type=float, default=RELATIVE_THRESHOLD)
ap.add_argument("--channels", type=str, nargs="+",
default=["front", "top", "bottom"],
help="Which channels to compare")
ap.add_argument("--out", type=str, default=None)
args = ap.parse_args()
data = load_sindy_scenes(args.sindy_results, args.scenes)
if len(data) < 2:
print("Need at least 2 scenes to compare")
return 1
scene_names = list(data.keys())
scene_a, scene_b = scene_names[0], scene_names[1]
print(f"Support Overlap: {scene_a} vs {scene_b} (th={args.threshold})")
print("=" * 60)
all_results = {}
for ch_name in args.channels:
# Map "front" -> "feature_names_front", etc
fn_key = f"feature_names_{'front' if ch_name == 'front' else 'rear'}"
fn_a = data[scene_a][fn_key]
fn_b = data[scene_b][fn_key]
fn_min = min(len(fn_a), len(fn_b))
fn_a_trim = fn_a[:fn_min]
fn_b_trim = fn_b[:fn_min]
ch_a = get_active_support(np.array(data[scene_a][ch_name]["best_coef"])[:fn_min],
fn_a_trim, args.threshold)
ch_b = get_active_support(np.array(data[scene_b][ch_name]["best_coef"])[:fn_min],
fn_b_trim, args.threshold)
shared, a_only, b_only = classify(ch_a, ch_b)
print(f"\n--- {ch_name} ---")
print(f" {scene_a} nz={len(ch_a)} {scene_b} nz={len(ch_b)} Shared={len(shared)}")
for name, ca, cb in shared:
print(f" {name:20s} A={ca:+9.6f} B={cb:+9.6f} [{feat_group(name)}]")
for name, ca in a_only:
print(f" {scene_a[:10]:>10s} {name:20s} A={ca:+9.6f} [{feat_group(name)}]")
for name, cb in b_only:
print(f" {scene_b[:10]:>10s} {name:20s} B={cb:+9.6f} [{feat_group(name)}]")
all_results[ch_name] = {
"scene_a_nz": len(ch_a),
"scene_b_nz": len(ch_b),
"shared_nz": len(shared),
"shared": [{"name": n, "coef_a": ca, "coef_b": cb} for n, ca, cb in shared],
f"{scene_a}_only": [{"name": n, "coef": ca} for n, ca in a_only],
f"{scene_b}_only": [{"name": n, "coef": cb} for n, cb in b_only],
}
if args.out:
output = {"scene_a": scene_a, "scene_b": scene_b,
"threshold": args.threshold,
"channels": all_results}
os.makedirs(os.path.dirname(args.out), exist_ok=True)
with open(args.out, "w") as f:
json.dump(output, f, indent=2)
print(f"\nSaved: {args.out}")
if __name__ == "__main__":
main()

2
src/SR_analysis/configs.py
View File

@ -213,6 +213,8 @@ for vtype, mn, strength in _SCENES_VORTEX:
# -- Illusion generalization (unseen diameters, no PPO model) ---------------- # -- Illusion generalization (unseen diameters, no PPO model) ----------------
for diam, mn, si in [ for diam, mn, si in [
(0.5, None, 400), (0.5, None, 400),
(0.6, None, 400), # added 2026-06-25 for generalization test
(0.8, None, 600), # added 2026-06-25 for generalization test
(1.2, None, 600), (1.2, None, 600),
(2.0, None, 800), (2.0, None, 800),
]: ]:

7
src/SR_analysis/data/illusion/illusion_0.75L/config.json
View File

@ -5,17 +5,18 @@
"mu": 0.02, "mu": 0.02,
"nu": 0.004, "nu": 0.004,
"has_disturbance": false, "has_disturbance": false,
"sample_interval": 600, "sample_interval": 400,
"conv_len": 36,
"action_scale": 8.0, "action_scale": 8.0,
"action_bias": "(0.0, -2.0, 2.0)", "action_bias": "(0.0, -2.0, 2.0)",
"source": "PPO_inference", "source": "PPO_inference",
"model_name": "d1a3o12_250525_imit_075L_1U", "model_name": "d1a3o14_250525_imit_075L_2U_400S",
"n_objects_env": 6, "n_objects_env": 6,
"obs_slice": "(0, 12)", "obs_slice": "(0, 12)",
"sensor_x": 30.0, "sensor_x": 30.0,
"pinball_front_x": 19.0, "pinball_front_x": 19.0,
"pinball_rear_x": 20.3, "pinball_rear_x": 20.3,
"target_type": "periodic", "target_type": "periodic",
"s_dim": 12, "s_dim": 14,
"u0": 0.01 "u0": 0.01
} }

26
src/SR_analysis/data/illusion/illusion_0.75L/norm.json
View File

@ -1,20 +1,20 @@
{ {
"force_norm_fact": 0.013476977124810219, "force_norm_fact": 0.013459767680615187,
"sens_deviation": [ "sens_deviation": [
0.962617814540863, 0.9466423392295837,
-0.12039308249950409, -0.1479361653327942,
0.6415857672691345, 0.6496055126190186,
0.011103342287242413, -0.06913633644580841,
0.9339056611061096, 0.9421071410179138,
0.11935960501432419 0.08593743294477463
], ],
"sens_norm_fact": [ "sens_norm_fact": [
2.0483264923095703, 2.1083030700683594,
2.5809006690979004, 2.7172951698303223,
0.7443606853485107, 0.7220026850700378,
3.4969263076782227, 3.7818100452423096,
2.1811583042144775, 2.1336052417755127,
2.5745153427124023 2.4038033485412598
], ],
"action_bias": [ "action_bias": [
0.0, 0.0,

2
src/SR_analysis/data/illusion/illusion_0.75L/result.json
View File

@ -1,5 +1,5 @@
{ {
"scene": "illusion_0.75L", "scene": "illusion_0.75L",
"controlled": true, "controlled": true,
"similarity": 0.18393706196948187 "similarity": 0.980430435808052
} }

7
src/SR_analysis/data/illusion/illusion_1.5L/config.json
View File

@ -3,9 +3,10 @@
"target_diameter": 1.5, "target_diameter": 1.5,
"re_code": 100, "re_code": 100,
"mu": 0.02, "mu": 0.02,
"nu": 0.008, "nu": 0.004,
"has_disturbance": false, "has_disturbance": false,
"sample_interval": 600, "sample_interval": 800,
"conv_len": 36,
"action_scale": 8.0, "action_scale": 8.0,
"action_bias": "(0.0, -2.0, 2.0)", "action_bias": "(0.0, -2.0, 2.0)",
"source": "PPO_inference", "source": "PPO_inference",
@ -17,5 +18,5 @@
"pinball_rear_x": 20.3, "pinball_rear_x": 20.3,
"target_type": "periodic", "target_type": "periodic",
"s_dim": 14, "s_dim": 14,
"u0": 0.02 "u0": 0.01
} }

26
src/SR_analysis/data/illusion/illusion_1.5L/norm.json
View File

@ -1,20 +1,20 @@
{ {
"force_norm_fact": 0.054184265434741974, "force_norm_fact": 0.013502256944775581,
"sens_deviation": [ "sens_deviation": [
1.9146838188171387, 0.9501011371612549,
-0.23843951523303986, -0.1168946698307991,
1.305143117904663, 0.6514688730239868,
0.0009254463366232812, 0.0026063816621899605,
1.8709181547164917, 0.9466588497161865,
0.2255263477563858 0.1172625869512558
], ],
"sens_norm_fact": [ "sens_norm_fact": [
4.165492057800293, 2.11934757232666,
5.171088695526123, 2.5621178150177,
1.5217405557632446, 0.7280007004737854,
6.904999732971191, 3.4596621990203857,
4.384937763214111, 2.1354496479034424,
5.106513023376465 2.5682904720306396
], ],
"action_bias": [ "action_bias": [
0.0, 0.0,

4
src/SR_analysis/data/illusion/illusion_1.5L/result.json
View File

@ -1,5 +1,5 @@
{ {
"scene": "illusion_15L", "scene": "illusion_1.5L",
"controlled": true, "controlled": true,
"similarity": 0.30179914651024675 "similarity": 0.9453180853373244
} }

5
src/SR_analysis/data/illusion/illusion_1L/config.json
View File

@ -6,16 +6,17 @@
"nu": 0.004, "nu": 0.004,
"has_disturbance": false, "has_disturbance": false,
"sample_interval": 600, "sample_interval": 600,
"conv_len": 36,
"action_scale": 8.0, "action_scale": 8.0,
"action_bias": "(0.0, -2.0, 2.0)", "action_bias": "(0.0, -2.0, 2.0)",
"source": "PPO_inference", "source": "PPO_inference",
"model_name": "d1a3o12_250525_imit_1L_1U", "model_name": "d1a3o14_250525_imit_1L_2U_600S",
"n_objects_env": 6, "n_objects_env": 6,
"obs_slice": "(0, 12)", "obs_slice": "(0, 12)",
"sensor_x": 30.0, "sensor_x": 30.0,
"pinball_front_x": 19.0, "pinball_front_x": 19.0,
"pinball_rear_x": 20.3, "pinball_rear_x": 20.3,
"target_type": "periodic", "target_type": "periodic",
"s_dim": 12, "s_dim": 14,
"u0": 0.01 "u0": 0.01
} }

26
src/SR_analysis/data/illusion/illusion_1L/norm.json
View File

@ -1,20 +1,20 @@
{ {
"force_norm_fact": 0.013487594202160835, "force_norm_fact": 0.01349810091778636,
"sens_deviation": [ "sens_deviation": [
0.9391862154006958, 0.9381087422370911,
-0.09573575109243393, -0.12299147248268127,
0.6525679230690002, 0.648796796798706,
0.03420928493142128, -0.022100985050201416,
0.949753999710083, 0.9596271514892578,
0.13210755586624146 0.11032649874687195
], ],
"sens_norm_fact": [ "sens_norm_fact": [
2.1654911041259766, 2.1753337383270264,
2.459106683731079, 2.5954699516296387,
0.7431825995445251, 0.707850456237793,
3.613541603088379, 3.554702043533325,
2.1102607250213623, 2.0688724517822266,
2.6434059143066406 2.5344114303588867
], ],
"action_bias": [ "action_bias": [
0.0, 0.0,

4
src/SR_analysis/data/illusion/illusion_1L/result.json
View File

@ -1,5 +1,5 @@
{ {
"scene": "illusion_1.0L", "scene": "illusion_1L",
"controlled": true, "controlled": true,
"similarity": 0.5543174409436081 "similarity": 0.9753732477509874
} }

842
src/SR_analysis/pysr.md
View File

@ -1,842 +0,0 @@
# Toy Examples with Code
## Preamble
```python
import numpy as np
from pysr import *
```
## 1. Simple search
Here's a simple example where we
find the expression `2 cos(x3) + x0^2 - 2`.
```python
X = 2 * np.random.randn(100, 5)
y = 2 * np.cos(X[:, 3]) + X[:, 0] ** 2 - 2
model = PySRRegressor(binary_operators=["+", "-", "*", "/"])
model.fit(X, y)
print(model)
```
## 2. Custom operator
Here, we define a custom operator and use it to find an expression:
```python
X = 2 * np.random.randn(100, 5)
y = 1 / X[:, 0]
model = PySRRegressor(
binary_operators=["+", "*"],
unary_operators=["inv(x) = 1/x"],
extra_sympy_mappings={"inv": lambda x: 1/x},
)
model.fit(X, y)
print(model)
```
## 3. Multiple outputs
Here, we do the same thing, but with multiple expressions at once,
each requiring a different feature.
```python
X = 2 * np.random.randn(100, 5)
y = 1 / X[:, [0, 1, 2]]
model = PySRRegressor(
binary_operators=["+", "*"],
unary_operators=["inv(x) = 1/x"],
extra_sympy_mappings={"inv": lambda x: 1/x},
)
model.fit(X, y)
```
## 4. Plotting an expression
For now, let's consider the expressions for output 0.
We can see the LaTeX version of this with:
```python
model.latex()[0]
```
or output 1 with `model.latex()[1]`.
Let's plot the prediction against the truth:
```python
from matplotlib import pyplot as plt
plt.scatter(y[:, 0], model.predict(X)[:, 0])
plt.xlabel('Truth')
plt.ylabel('Prediction')
plt.show()
```
Which gives us:
![Truth vs Prediction](/images/example_plot.png)
We may also plot the output of a particular expression
by passing the index of the expression to `predict` (or
`sympy` or `latex` as well)
## 5. Feature selection
PySR and evolution-based symbolic regression in general performs
very poorly when the number of features is large.
Even, say, 10 features might be too much for a typical equation search.
If you are dealing with high-dimensional data with a particular type of structure,
you might consider using deep learning to break the problem into
smaller "chunks" which can then be solved by PySR, as explained in the paper
[2006.11287](https://arxiv.org/abs/2006.11287).
For tabular datasets, this is a bit trickier. Luckily, PySR has a built-in feature
selection mechanism. Simply declare the parameter `select_k_features=5`, for selecting
the most important 5 features.
Here is an example. Let's say we have 30 input features and 300 data points, but only 2
of those features are actually used:
```python
X = np.random.randn(300, 30)
y = X[:, 3]**2 - X[:, 19]**2 + 1.5
```
Let's create a model with the feature selection argument set up:
```python
model = PySRRegressor(
binary_operators=["+", "-", "*", "/"],
unary_operators=["exp"],
select_k_features=5,
)
```
Now let's fit this:
```python
model.fit(X, y)
```
Before the Julia backend is launched, you can see the string:
```text
Using features ['x3', 'x5', 'x7', 'x19', 'x21']
```
which indicates that the feature selection (powered by a gradient-boosting tree)
has successfully selected the relevant two features.
This fit should find the solution quickly, whereas with the huge number of features,
it would have struggled.
This simple preprocessing step is enough to simplify our tabular dataset,
but again, for more structured datasets, you should try the deep learning
approach mentioned above.
## 6. Denoising
Many datasets, especially in the observational sciences,
contain intrinsic noise. PySR is noise robust itself, as it is simply optimizing a loss function,
but there are still some additional steps you can take to reduce the effect of noise.
One thing you could do, which we won't detail here, is to create a custom log-likelihood
given some assumed noise model. By passing weights to the fit function, and
defining a custom loss function such as `elementwise_loss="myloss(x, y, w) = w * (x - y)^2"`,
you can define any sort of log-likelihood you wish. (However, note that it must be bounded at zero)
However, the simplest thing to do is preprocessing, just like for feature selection. To do this,
set the parameter `denoise=True`. This will fit a Gaussian process (containing a white noise kernel)
to the input dataset, and predict new targets (which are assumed to be denoised) from that Gaussian process.
For example:
```python
X = np.random.randn(100, 5)
noise = np.random.randn(100) * 0.1
y = np.exp(X[:, 0]) + X[:, 1] + X[:, 2] + noise
```
Let's create and fit a model with the denoising argument set up:
```python
model = PySRRegressor(
binary_operators=["+", "-", "*", "/"],
unary_operators=["exp"],
denoise=True,
)
model.fit(X, y)
print(model)
```
If all goes well, you should find that it predicts the correct input equation, without the noise term!
## 7. Julia packages and types
PySR uses [SymbolicRegression.jl](https://github.com/MilesCranmer/SymbolicRegression.jl)
as its search backend. This is a pure Julia package, and so can interface easily with any other
Julia package.
For some tasks, it may be necessary to load such a package.
For example, let's say we wish to discovery the following relationship:
$$ y = p_{3x + 1} - 5, $$
where $p_i$ is the $i$th prime number, and $x$ is the input feature.
Let's see if we can discover this using
the [Primes.jl](https://github.com/JuliaMath/Primes.jl) package.
First, let's get the Julia backend:
```python
from pysr import jl
```
`jl` stores the Julia runtime.
Now, let's run some Julia code to add the Primes.jl
package to the PySR environment:
```python
jl.seval("""
import Pkg
Pkg.add("Primes")
""")
```
This imports the Julia package manager, and uses it to install
`Primes.jl`. Now let's import `Primes.jl`:
```python
jl.seval("import Primes")
```
Now, we define a custom operator:
```python
jl.seval("""
function p(i::T) where T
if (0.5 < i < 1000)
return T(Primes.prime(round(Int, i)))
else
return T(NaN)
end
end
""")
```
We have created a a function `p`, which takes an arbitrary number as input.
`p` first checks whether the input is between 0.5 and 1000.
If out-of-bounds, it returns `NaN`.
If in-bounds, it rounds it to the nearest integer, compures the corresponding prime number, and then
converts it to the same type as input.
Next, let's generate a list of primes for our test dataset.
Since we are using juliacall, we can just call `p` directly to do this:
```python
primes = {i: jl.p(i*1.0) for i in range(1, 999)}
```
Next, let's use this list of primes to create a dataset of $x, y$ pairs:
```python
import numpy as np
X = np.random.randint(0, 100, 100)[:, None]
y = [primes[3*X[i, 0] + 1] - 5 + np.random.randn()*0.001 for i in range(100)]
```
Note that we have also added a tiny bit of noise to the dataset.
Finally, let's create a PySR model, and pass the custom operator. We also need to define the sympy equivalent, which we can leave as a placeholder for now:
```python
from pysr import PySRRegressor
import sympy
class sympy_p(sympy.Function):
pass
model = PySRRegressor(
binary_operators=["+", "-", "*", "/"],
unary_operators=["p"],
niterations=100,
extra_sympy_mappings={"p": sympy_p}
)
```
We are all set to go! Let's see if we can find the true relation:
```python
model.fit(X, y)
```
if all works out, you should be able to see the true relation (note that the constant offset might not be exactly 1, since it is allowed to round to the nearest integer).
You can get the sympy version of the best equation with:
```python
model.sympy()
```
## 8. Complex numbers
PySR can also search for complex-valued expressions. Simply pass
data with a complex datatype (e.g., `np.complex128`),
and PySR will automatically search for complex-valued expressions:
```python
import numpy as np
X = np.random.randn(100, 1) + 1j * np.random.randn(100, 1)
y = (1 + 2j) * np.cos(X[:, 0] * (0.5 - 0.2j))
model = PySRRegressor(
binary_operators=["+", "-", "*"], unary_operators=["cos"], niterations=100,
)
model.fit(X, y)
```
You can see that all of the learned constants are now complex numbers.
We can get the sympy version of the best equation with:
```python
model.sympy()
```
We can also make predictions normally, by passing complex data:
```python
model.predict(X, -1)
```
to make predictions with the most accurate expression.
## 9. Custom objectives
You can also pass a custom objectives as a snippet of Julia code,
which might include symbolic manipulations or custom functional forms.
These do not even need to be differentiable! First, let's look at the
default objective used (a simplified version, without weights
and with mean square error), so that you can see how to write your own:
```julia
function default_objective(tree, dataset::Dataset{T,L}, options)::L where {T,L}
(prediction, completion) = eval_tree_array(tree, dataset.X, options)
if !completion
return L(Inf)
end
diffs = prediction .- dataset.y
return sum(diffs .^ 2) / length(diffs)
end
```
Here, the `where {T,L}` syntax defines the function for arbitrary types `T` and `L`.
If you have `precision=32` (default) and pass in regular floating point data,
then both `T` and `L` will be equal to `Float32`. If you pass in complex data,
then `T` will be `ComplexF32` and `L` will be `Float32` (since we need to return
a real number from the loss function). But, you don't need to worry about this, just
make sure to return a scalar number of type `L`.
The `tree` argument is the current expression being evaluated. You can read
about the `tree` fields [here](https://ai.damtp.cam.ac.uk/symbolicregression/stable/types/).
For example, let's fix a symbolic form of an expression,
as a rational function. i.e., $P(X)/Q(X)$ for polynomials $P$ and $Q$.
```python
objective = """
function my_custom_objective(tree, dataset::Dataset{T,L}, options) where {T,L}
# Require root node to be binary, so we can split it,
# otherwise return a large loss:
tree.degree != 2 && return L(Inf)
P = tree.l
Q = tree.r
# Evaluate numerator:
P_prediction, flag = eval_tree_array(P, dataset.X, options)
!flag && return L(Inf)
# Evaluate denominator:
Q_prediction, flag = eval_tree_array(Q, dataset.X, options)
!flag && return L(Inf)
# Impose functional form:
prediction = P_prediction ./ Q_prediction
diffs = prediction .- dataset.y
return sum(diffs .^ 2) / length(diffs)
end
"""
model = PySRRegressor(
niterations=100,
binary_operators=["*", "+", "-"],
loss_function=objective,
)
```
> **Warning**: When using a custom objective like this that performs symbolic
> manipulations, many functionalities of PySR will not work, such as `.sympy()`,
> `.predict()`, etc. This is because the SymPy parsing does not know about
> how you are manipulating the expression, so you will need to do this yourself.
Note how we did not pass `/` as a binary operator; it will just be implicit
in the functional form.
Let's generate an equation of the form $\frac{x_0^2 x_1 - 2}{x_2^2 + 1}$:
```python
X = np.random.randn(1000, 3)
y = (X[:, 0]**2 * X[:, 1] - 2) / (X[:, 2]**2 + 1)
```
Finally, let's fit:
```python
model.fit(X, y)
```
> Note that the printed equation is not the same as the evaluated equation,
> because the printing functionality does not know about the functional form.
We can get the string format with:
```python
model.get_best().equation
```
(or, you could use `model.equations_.iloc[-1].equation`)
For me, this equation was:
```text
(((2.3554819 + -0.3554746) - (x1 * (x0 * x0))) - (-1.0000019 - (x2 * x2)))
```
looking at the bracket structure of the equation, we can see that the outermost
bracket is split at the `-` operator (note that we ignore the root operator in
the evaluation, as we simply evaluated each argument and divided the result) into
`((2.3554819 + -0.3554746) - (x1 * (x0 * x0)))` and
`(-1.0000019 - (x2 * x2))`, meaning that our discovered equation is
equal to:
$\frac{x_0^2 x_1 - 2.0000073}{x_2^2 + 1.0000019}$, which
is nearly the same as the true equation!
## 10. Dimensional constraints
One other feature we can exploit is dimensional analysis.
Say that we know the physical units of each feature and output,
and we want to find an expression that is dimensionally consistent.
We can do this as follows, using `DynamicQuantities.jl` to assign units,
passing a string specifying the units for each variable.
First, let's make some data on Newton's law of gravitation, using
astropy for units:
```python
import numpy as np
from astropy import units as u, constants as const
M = (np.random.rand(100) + 0.1) * const.M_sun
m = 100 * (np.random.rand(100) + 0.1) * u.kg
r = (np.random.rand(100) + 0.1) * const.R_earth
G = const.G
F = G * M * m / r**2
```
We can see the units of `F` with `F.unit`.
Now, let's create our model.
Since this data has such a large dynamic range,
let's also create a custom loss function
that looks at the error in log-space:
```python
elementwise_loss = """function loss_fnc(prediction, target)
scatter_loss = abs(log((abs(prediction)+1e-20) / (abs(target)+1e-20)))
sign_loss = 10 * (sign(prediction) - sign(target))^2
return scatter_loss + sign_loss
end
"""
```
Now let's define our model:
```python
model = PySRRegressor(
binary_operators=["+", "-", "*", "/"],
unary_operators=["square"],
elementwise_loss=elementwise_loss,
complexity_of_constants=2,
maxsize=25,
niterations=100,
populations=50,
# Amount to penalize dimensional violations:
dimensional_constraint_penalty=10**5,
)
```
and fit it, passing the unit information.
To do this, we need to use the format of [DynamicQuantities.jl](https://symbolicml.org/DynamicQuantities.jl/dev/#Usage).
```python
# Get numerical arrays to fit:
X = pd.DataFrame(dict(
M=M.to("M_sun").value,
m=m.to("kg").value,
r=r.to("R_earth").value,
))
y = F.value
model.fit(
X,
y,
X_units=["Constants.M_sun", "kg", "Constants.R_earth"],
y_units="kg * m / s^2"
)
```
You can observe that all expressions with a loss under
our penalty are dimensionally consistent!
(The `"[⋅]"` indicates free units in a constant, which can cancel out other units in the expression.)
For example,
```julia
"y[m s⁻² kg] = (M[kg] * 2.6353e-22[⋅])"
```
would indicate that the expression is dimensionally consistent, with
a constant `"2.6353e-22[m s⁻²]"`.
Note that this expression has a large dynamic range so may be difficult to find. Consider searching with a larger `niterations` if needed.
Note that you can also search for exclusively dimensionless constants by settings
`dimensionless_constants_only` to `true`.
## 11. Expression Specifications
PySR 1.0 introduces powerful expression specifications that allow you to define structured equations. Here are two examples:
### Template Expressions
`TemplateExpressionSpec` allows you to define a specific structure for the equation.
For example, let's say we want to learn an equation of the form:
$$ y = \sin(f(x_1, x_2)) + g(x_3) $$
We can do this as follows:
```python
import numpy as np
from pysr import PySRRegressor, TemplateExpressionSpec
# Create data
X = np.random.randn(1000, 3)
y = np.sin(X[:, 0] + X[:, 1]) + X[:, 2]**2
# Define template: we want sin(f(x1, x2)) + g(x3)
template = TemplateExpressionSpec(
expressions=["f", "g"],
variable_names=["x1", "x2", "x3"],
combine="sin(f(x1, x2)) + g(x3)",
)
model = PySRRegressor(
expression_spec=template,
binary_operators=["+", "*", "-", "/"],
unary_operators=["sin"],
maxsize=10,
)
model.fit(X, y)
```
### Parametric Expressions
When your data has categories with shared equation structure but different parameters,
you can use the `parameters` argument of `TemplateExpressionSpec` to specify learned category-specific parameters.
For example, let's say we want to learn an equation of the form:
$$ y = \alpha \sin(x_1) + \beta $$
where $\alpha$ and $\beta$ are different for each category.
Further, let's say we have 3 categories,
with $\alpha \in \{0.1, 1.5, -0.5\}$ and $\beta \in \{1.0, 2.0, 0.5\}$.
```python
import numpy as np
from pysr import PySRRegressor, TemplateExpressionSpec
# Create data with 2 features and 3 categories
X = np.random.uniform(-3, 3, (1000, 2))
category = np.random.randint(0, 3, 1000)
# Parameters for each category
offsets = [0.1, 1.5, -0.5]
scales = [1.0, 2.0, 0.5]
# y = scale[category] * sin(x1) + offset[category]
y = np.array([
scales[c] * np.sin(x1) + offsets[c]
for x1, c in zip(X[:, 0], category)
])
```
Now, let's define our parametric expression:
```python
template = TemplateExpressionSpec(
expressions=["f"],
variable_names=["x1", "x2", "category"],
parameters={"p1": 3, "p2": 3}, # One parameter per category
combine="f(x1, x2, p1[category], p2[category])"
)
```
Next, we pass the category as a _column_ in `X`
corresponding to the index we defined in `variable_names`.
**Note that because Julia is 1-indexed, we need to add 1 to the category index.**
```python
category_p_one = category + 1
X_with_category = np.column_stack([X, category])
```
Now, we can fit our model:
```python
model = PySRRegressor(
expression_spec=template,
binary_operators=["+", "*", "-", "/"],
unary_operators=["sin"],
maxsize=10,
)
model.fit(X_with_category, y)
# Predicting on new data
# model.predict(X_test_with_category)
```
See [Expression Specifications](/api/#expression-specifications) for more details.
You can use this approach for more complex cases,
where you have multiple expressions in the template and parameters that vary by category.
## 12. Using TensorBoard for Logging
You can use TensorBoard to visualize the search progress, as well as
record hyperparameters and final metrics (like `min_loss` and `pareto_volume` - the latter of which
is a performance measure of the entire Pareto front).
```python
import numpy as np
from pysr import PySRRegressor, TensorBoardLoggerSpec
rstate = np.random.RandomState(42)
# Uniform dist between -3 and 3:
X = rstate.uniform(-3, 3, (1000, 2))
y = np.exp(X[:, 0]) + X[:, 1]
# Create a logger that writes to "logs/run*":
logger_spec = TensorBoardLoggerSpec(
log_dir="logs/run",
log_interval=10, # Log every 10 iterations
)
model = PySRRegressor(
binary_operators=["+", "*", "-", "/"],
logger_spec=logger_spec,
)
model.fit(X, y)
```
You can then view the logs with:
```bash
tensorboard --logdir logs/
```
## 13. Vector-valued expressions
You can use `TemplateExpressionSpec` to find expressions for vector-valued data,
where each component might share a common structure.
The trick is to put each vector element into your feature matrix `X`,
and then use a template expression to define the relationships.
For example, say we have 3-dimensional vectors where each component
follows a pattern with a shared term. Say the true model is:
$$\begin{align*}
y_1 &= \exp(x_1) + x_2^2 \\
y_2 &= \exp(x_1) + \sin(x_3) \\
y_3 &= \exp(x_1) + x_1 \cdot x_2
\end{align*}$$
Let's set this up:
```python
import numpy as np
from pysr import PySRRegressor, TemplateExpressionSpec
n = 200
rstate = np.random.RandomState(0)
x1 = rstate.uniform(-2, 2, n)
x2 = rstate.uniform(-2, 2, n)
x3 = rstate.uniform(-2, 2, n)
# True model with shared component exp(x1):
y1 = np.exp(x1) + x2**2
y2 = np.exp(x1) + np.sin(x3)
y3 = np.exp(x1) + x1 * x2
# Add some noise
y1 += 0.05 * rstate.randn(n)
y2 += 0.05 * rstate.randn(n)
y3 += 0.05 * rstate.randn(n)
```
Now, we put everything in `X`; BOTH features and targets:
```python
X = np.column_stack([x1, x2, x3, y1, y2, y3])
```
Now, we can define our template expression:
```python
spec = TemplateExpressionSpec(
expressions=["f1", "f2", "f3", "shared"],
variable_names=["x1", "x2", "x3", "y1", "y2", "y3"],
combine="""
v = shared(x1, x2, x3)
y1_predicted = v + f1(x1, x2, x3)
y2_predicted = v + f2(x1, x2, x3)
y3_predicted = v + f3(x1, x2, x3)
residuals = (
abs2(y1 - y1_predicted) +
abs2(y2 - y2_predicted) +
abs2(y3 - y3_predicted)
)
residuals
"""
)
```
Now, we can fit our model using this template. Since
we already computed the per-row squared error inside the template,
we can pass a dummy `y` to the `fit` method, and also define
an `elementwise_loss` that simply returns the residuals (which get
summed over the data):
```python
model = PySRRegressor(
expression_spec=spec,
binary_operators=["+", "-", "*", "/"],
unary_operators=["exp", "sin"],
maxsize=20,
niterations=50,
elementwise_loss="(pred, target) -> pred",
)
dummy_y = np.zeros(n)
model.fit(X, dummy_y)
```
After running, PySR should find both the shared component (`exp(x1)`) as well as individual components (`square(x2)`, `sin(x3)`, and `x1 * x2`).
You can access the individual expressions through the Julia objects:
```python
# Simply get the expression with the highest score:
idx = model.equations_.score.idxmax()
# Extract the Julia object:
julia_expr = model.equations_.loc[idx, 'julia_expression']
# Access individual subexpressions:
for name in ['f1', 'f2', 'f3', 'shared']:
tree = getattr(julia_expr.trees, name)
print(f"{name}: {tree}")
```
We can also evaluate individual expressions:
```python
from pysr import jl
from pysr.julia_helpers import jl_array
SR = jl.SymbolicRegression
# Get individual trees
f1_tree = julia_expr.trees.f1
shared_tree = julia_expr.trees.shared
# Evaluate at specific points (x1=1, x2=2, x3=3)
test_inputs = jl_array(np.array([[1.0], [2.0], [3.0]]))
f1_result, _ = SR.eval_tree_array(f1_tree, test_inputs, model.julia_options_)
shared_result, _ = SR.eval_tree_array(shared_tree, test_inputs, model.julia_options_)
print(f"f1 at (1,2,3): {f1_result[0]}") # Should be ~4.0 for x2^2
print(f"shared at (1,2,3): {shared_result[0]}") # Should be ~2.718 for exp(1)
```
## 14. Using differential operators
As part of the `TemplateExpressionSpec` described above,
you can also use differential operators within the template.
The operator for this is `D` which takes an expression as the first argument,
and the argument _index_ we are differentiating as the second argument.
This lets you compute integrals via evolution.
For example, let's say we wish to find the integral of $\frac{1}{x^2 \sqrt{x^2 - 1}}$
in the range $x > 1$.
We can compute the derivative of a function $f(x)$, and compare that
to numerical samples of $\frac{1}{x^2\sqrt{x^2-1}}$. Then, by extension,
$f(x)$ represents the indefinite integral of it with some constant offset!
```python
import numpy as np
from pysr import PySRRegressor, TemplateExpressionSpec
x = np.random.uniform(1, 10, (1000,)) # Integrand sampling points
y = 1 / (x**2 * np.sqrt(x**2 - 1)) # Evaluation of the integrand
expression_spec = TemplateExpressionSpec(
expressions=["f"],
variable_names=["x"],
combine="df = D(f, 1); df(x)",
)
model = PySRRegressor(
binary_operators=["+", "-", "*", "/"],
unary_operators=["sqrt"],
expression_spec=expression_spec,
maxsize=20,
)
model.fit(x[:, np.newaxis], y)
```
If everything works, you should find something that simplifies to $\frac{\sqrt{x^2 - 1}}{x}$.
Here, we write out a full function in Julia.
## 15. Additional features
For the many other features available in PySR, please
read the [Options section](options.md).

102
src/SR_analysis/sindy/illusion/pareto_illusion_1.5L.json
View File

@ -1,102 +0,0 @@
{
"scene": "illusion_1.5L",
"channels": [
{
"channel": "front",
"best_r2": 0.9594009004329207,
"best_nz": 21,
"pareto": [
{
"nz": 5,
"r2": 0.9393791282052957
},
{
"nz": 6,
"r2": 0.9467043546594699
},
{
"nz": 7,
"r2": 0.9468038158796819
},
{
"nz": 12,
"r2": 0.958013912668809
},
{
"nz": 17,
"r2": 0.9593925359990215
},
{
"nz": 18,
"r2": 0.9593997378572243
},
{
"nz": 21,
"r2": 0.9594009004329207
}
]
},
{
"channel": "top",
"best_r2": 0.9283646632651096,
"best_nz": 22,
"pareto": [
{
"nz": 2,
"r2": 0.8636623835462103
},
{
"nz": 5,
"r2": 0.9176885699701556
},
{
"nz": 6,
"r2": 0.9196961922610852
},
{
"nz": 11,
"r2": 0.9227131504032893
},
{
"nz": 13,
"r2": 0.926100716890473
},
{
"nz": 22,
"r2": 0.9283646632651096
}
]
},
{
"channel": "bottom",
"best_r2": 0.9318647363961834,
"best_nz": 21,
"pareto": [
{
"nz": 3,
"r2": 0.7872211556048209
},
{
"nz": 6,
"r2": 0.9223640246817683
},
{
"nz": 7,
"r2": 0.9258419638948643
},
{
"nz": 11,
"r2": 0.929107795801212
},
{
"nz": 16,
"r2": 0.9315566670173666
},
{
"nz": 21,
"r2": 0.9318647363961834
}
]
}
]
}

110
src/SR_analysis/sindy/illusion/pareto_illusion_1L.json
View File

@ -1,110 +0,0 @@
{
"scene": "illusion_1L",
"channels": [
{
"channel": "front",
"best_r2": 0.9793036424523165,
"best_nz": 21,
"pareto": [
{
"nz": 4,
"r2": 0.9718953011650306
},
{
"nz": 6,
"r2": 0.9752101462860064
},
{
"nz": 11,
"r2": 0.9785538576893853
},
{
"nz": 15,
"r2": 0.9791278336272012
},
{
"nz": 18,
"r2": 0.9792726941557117
},
{
"nz": 21,
"r2": 0.9793036424523165
}
]
},
{
"channel": "top",
"best_r2": 0.9838580289472151,
"best_nz": 22,
"pareto": [
{
"nz": 7,
"r2": 0.9786186299815743
},
{
"nz": 10,
"r2": 0.9828568398768021
},
{
"nz": 11,
"r2": 0.9833618417657272
},
{
"nz": 12,
"r2": 0.9835181072357578
},
{
"nz": 16,
"r2": 0.9837742843659423
},
{
"nz": 22,
"r2": 0.9838580289472151
}
]
},
{
"channel": "bottom",
"best_r2": 0.983658612471297,
"best_nz": 22,
"pareto": [
{
"nz": 4,
"r2": 0.9698703453821834
},
{
"nz": 6,
"r2": 0.9816905531339832
},
{
"nz": 7,
"r2": 0.9817463485828739
},
{
"nz": 8,
"r2": 0.9822685326747084
},
{
"nz": 11,
"r2": 0.9831202415012674
},
{
"nz": 12,
"r2": 0.983291741316277
},
{
"nz": 15,
"r2": 0.9836126332791877
},
{
"nz": 18,
"r2": 0.9836582680775273
},
{
"nz": 22,
"r2": 0.983658612471297
}
]
}
]
}

1486
src/SR_analysis/sindy/illusion/sindy_results.json
View File

File diff suppressed because it is too large Load Diff

118
src/SR_analysis/sindy/karman/pareto_karman_re100.json
View File

@ -1,118 +0,0 @@
{
"scene": "karman_re100",
"channels": [
{
"channel": "front",
"best_r2": 0.9950013415086292,
"best_nz": 21,
"pareto": [
{
"nz": 2,
"r2": 0.8681678005649112
},
{
"nz": 3,
"r2": 0.9692414821446799
},
{
"nz": 4,
"r2": 0.989378747581318
},
{
"nz": 6,
"r2": 0.9908017426802015
},
{
"nz": 12,
"r2": 0.9939890287801123
},
{
"nz": 13,
"r2": 0.9947660566975605
},
{
"nz": 18,
"r2": 0.9949963098276752
},
{
"nz": 21,
"r2": 0.9950013415086292
}
]
},
{
"channel": "top",
"best_r2": 0.9928439394510484,
"best_nz": 22,
"pareto": [
{
"nz": 1,
"r2": 0.44188145385324007
},
{
"nz": 2,
"r2": 0.9285296099294669
},
{
"nz": 6,
"r2": 0.9812946866555053
},
{
"nz": 12,
"r2": 0.9909377708950154
},
{
"nz": 17,
"r2": 0.9927843560231949
},
{
"nz": 20,
"r2": 0.992824536423302
},
{
"nz": 22,
"r2": 0.9928439394510484
}
]
},
{
"channel": "bottom",
"best_r2": 0.9965335484991993,
"best_nz": 22,
"pareto": [
{
"nz": 1,
"r2": 0.8456588970543057
},
{
"nz": 2,
"r2": 0.9030386794798415
},
{
"nz": 8,
"r2": 0.9947803148283133
},
{
"nz": 10,
"r2": 0.9962551509064745
},
{
"nz": 13,
"r2": 0.9963877299663628
},
{
"nz": 16,
"r2": 0.9964335809851655
},
{
"nz": 18,
"r2": 0.9965311727162228
},
{
"nz": 22,
"r2": 0.9965335484991993
}
]
}
]
}

2031
src/SR_analysis/sindy/karman/sindy_results.json
View File

File diff suppressed because it is too large Load Diff

766
src/SR_analysis/sindy/run_all_v2.py
View File

@ -1,766 +0,0 @@
#!/usr/bin/env python3
"""Unified SINDy fitting v2 for all scenes.
Uses V2 feature builder: no sin_ua/cos_ua, optional mu, weighted STLSQ.
Generates separate per-scene results and cross-scene comparisons.
Usage:
conda run -n pycuda_3_10 python sindy/run_all_v2.py \\
--scenes karman_re50,karman_re100,steady
# Karman cross-Re only
conda run -n pycuda_3_10 python sindy/run_all_v2.py --family karman
# Cloak family (Karman + steady + vortex)
conda run -n pycuda_3_10 python sindy/run_all_v2.py --family cloak
# All scenes
conda run -n pycuda_3_10 python sindy/run_all_v2.py --family all
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from typing import List, Optional
import numpy as np
_REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
if _REPO not in sys.path:
sys.path.insert(0, _REPO)
_SRC = os.path.join(_REPO, "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from SR_analysis.utils.sindy_fitter import (
fit_sindy_weighted, get_feature_matrix_v2, get_active_support,
get_feature_matrix_deriv, compute_action_deriv,
)
from SR_analysis.utils.feature_builder import CORE_FEAT_KEYS_V2, ALL_FEAT_KEYS_V2, PHYSICS_FEAT_KEYS
from SR_analysis.configs import get_scene, get_scene_list, SCENES
# Base output directory
SINDY_DIR = os.path.join(os.path.dirname(__file__))
# Threshold grid (more granular at low end for better Pareto selection)
THRESHOLDS = [0.0, 0.001, 0.002, 0.003, 0.005, 0.007, 0.01, 0.015, 0.02, 0.03, 0.05, 0.08, 0.1]
FAMILIES = {
"cloak": ["karman", "steady", "vortex"],
"karman": ["karman"],
"illusion": ["illusion"],
"vortex": ["vortex"],
"all": None, # all scenes
}
def load_data(scene_name: str) -> tuple:
"""Load sensors/forces/actions from scene's controlled.npz.
Returns actions in PHYSICAL omega (lattice units).
Also returns target_forces if available (for Illusion scenes).
"""
cfg = get_scene(scene_name)
data_dir = os.path.join(
os.path.dirname(__file__), "..", "data",
cfg["scene_id"], scene_name,
)
npz = np.load(os.path.join(data_dir, "controlled.npz"))
sensors = npz["sensors"].astype(np.float64)
forces = npz["forces"].astype(np.float64)
# actions in .npz are normalized [-1,1]; convert to physical omega
actions_norm = npz["actions"].astype(np.float64)
scale = cfg["action_scale"]
bias = np.array(cfg["action_bias"], dtype=np.float64)
u0 = cfg["u0"]
actions_phys = (actions_norm * scale + bias) * u0
# Load target_forces if available (Illusion scenes)
target_forces = None
if "target_forces" in npz:
target_forces = npz["target_forces"].astype(np.float64)
return sensors, forces, actions_phys, cfg, target_forces
def compute_scene_weight(scene_name: str, cfg: dict) -> float:
"""Compute scene quality weight from PPO similarity."""
result_path = os.path.join(
os.path.dirname(__file__), "..", "data",
cfg["scene_id"], scene_name, "result.json",
)
if os.path.isfile(result_path):
with open(result_path) as f:
r = json.load(f)
sim = r.get("similarity", r.get("avg_reward_last100", 0.5))
return float(sim) ** 2
return 1.0
def run_single_scene(
scene_name: str,
thresholds: Optional[List[float]] = None,
verbose: bool = True,
) -> dict:
"""Run SINDy v2 on a single scene. Returns result dict."""
if thresholds is None:
thresholds = THRESHOLDS
sensors, forces, actions_phys, cfg, target_forces = load_data(scene_name)
mu = cfg["mu"]
u0 = cfg["u0"]
scene_id = cfg["scene_id"]
T = sensors.shape[0]
if verbose:
print(f"\n{'='*60}")
print(f"Scene: {scene_name} ({scene_id}) T={T} mu={mu:.4f} u0={u0}")
print(f"{'='*60}")
# Determine if this is a single-scene (no mu) or cross-Re (with mu)
# For cross-Re, we'll handle separately; for single-scene, no mu
# But steady is special: it needs mu=constant, still no mu in features
# Single scene: no mu, no sin/cos
is_single_re = True # will be overridden for joint fitting
# Build feature matrix (V2: no sin/cos, no mu for single-scene)
# Use dt_c=sample_interval/T0 in T0 units
t0_steps = 2000 # T0 = D/U0 = 20/0.01 = 2000 LBM steps
dt_c = cfg.get("sample_interval", 800) / t0_steps
Theta_f, Theta_r, Y, fn_f, fn_r = get_feature_matrix_v2(
sensors, forces, actions_phys, mu,
u0=u0, include_mu=False, include_cos_sin=False,
use_time_norm=False, dt_c=dt_c, n_warmup=2,
target_forces=target_forces,
)
if verbose:
print(f" Features: front {Theta_f.shape[1]-1} + bias?0, rear {Theta_r.shape[1]-1} + bias?1")
print(f" dt_c={dt_c:.4f} (T0 units)")
# Scene quality weight for STLSQ
w_scene = compute_scene_weight(scene_name, cfg)
if verbose:
print(f" Scene quality weight: {w_scene:.4f}")
# ---------- Front channel (no bias) ----------
if verbose:
print(f"\n --- Front (no bias) ---")
front_results = fit_sindy_weighted(Theta_f, Y[:, 0], thresholds,
n_robust_passes=2)
# Find best by R2 (include th=0.0 for dense coefficients)
front_best = max(front_results, key=lambda r: r["r2"])
if verbose:
_print_active(fn_f, front_best["coef"])
# ---------- Top channel (rear shared-head, with bias) ----------
if verbose:
print(f"\n --- Top (rear shared-head, with bias) ---")
top_results = fit_sindy_weighted(Theta_r, Y[:, 2], thresholds,
n_robust_passes=2)
top_best = max(top_results, key=lambda r: r["r2"])
if verbose:
_print_active(fn_r, top_best["coef"])
# ---------- Bottom (independent, for comparison) ----------
if verbose:
print(f"\n --- Bottom (independent, with bias) ---")
bot_results = fit_sindy_weighted(Theta_r, Y[:, 1], thresholds,
n_robust_passes=2)
bot_best = max(bot_results, key=lambda r: r["r2"])
if verbose:
_print_active(fn_r, bot_best["coef"])
# Compute additional metrics for the best thresholds
n_warmup = 2
n_samples = Theta_f.shape[0]
result = {
"scene": scene_name,
"scene_id": scene_id,
"re_code": cfg.get("re_code", 100),
"mu": mu,
"u0": u0,
"dt_c": dt_c,
"n_samples": n_samples,
"thresholds": thresholds,
"feature_names_front": fn_f,
"feature_names_rear": fn_r,
"front": {
"results": _clean_results(front_results),
"best": _clean_single(front_best),
"best_coef": front_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in front_results],
},
"top": {
"results": _clean_results(top_results),
"best": _clean_single(top_best),
"best_coef": top_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in top_results],
},
"bottom": {
"results": _clean_results(bot_results),
"best": _clean_single(bot_best),
"best_coef": bot_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in bot_results],
},
}
if verbose:
print(f"\n Summary:")
print(f" Front: th={front_best['threshold']:.4f} nz={front_best['nz']:2d} R2={front_best['r2']:.4f}")
print(f" Top: th={top_best['threshold']:.4f} nz={top_best['nz']:2d} R2={top_best['r2']:.4f}")
print(f" Bottom: th={bot_best['threshold']:.4f} nz={bot_best['nz']:2d} R2={bot_best['r2']:.4f}")
return result
def _print_active(names: List[str], coef: list, rtol: float = 1e-4):
"""Print active features from coefficient vector."""
for i, c in enumerate(coef):
if abs(c) > rtol and i < len(names):
print(f" {names[i]:20s} = {c:+.6f}")
def _clean_results(results: List[dict]) -> List[dict]:
"""Keep all keys including coef."""
return [dict(r) for r in results]
def _clean_single(entry: dict) -> dict:
"""Remove coef from single entry."""
return {k: v for k, v in entry.items() if k != "coef"}
def run_joint_karman(
scene_names: List[str],
thresholds: Optional[List[float]] = None,
) -> dict:
"""Joint Karman cross-Re SINDy with mu terms and scene weighting."""
if thresholds is None:
thresholds = THRESHOLDS
print(f"\n{'='*60}")
print(f"Joint Karman cross-Re: {scene_names}")
print(f"{'='*60}")
# Load and concatenate all Karman scenes
all_sensors, all_forces, all_actions = [], [], []
all_weights = []
t0_steps = 2000
for sn in scene_names:
sensors, forces, actions, cfg, _ = load_data(sn)
mu = cfg["mu"]
u0 = cfg["u0"]
dt_c = cfg.get("sample_interval", 800) / t0_steps
# Build features WITH mu terms
Theta_f, Theta_r, Y, fn_f, fn_r = get_feature_matrix_v2(
sensors, forces, actions, mu,
u0=u0, include_mu=True, include_cos_sin=False,
use_time_norm=False, dt_c=dt_c, n_warmup=2,
)
w = compute_scene_weight(sn, cfg)
n = Theta_f.shape[0]
print(f" {sn}: {n} samples, weight={w:.4f}")
all_weights.append(np.full(n, w))
all_sensors.append(Theta_f)
all_forces.append(Theta_r) # reuse variable names for concat
all_actions.append(Y)
# Concatenate
Theta_f_j = np.vstack(all_sensors)
Theta_r_j = np.vstack(all_forces)
Y_j = np.vstack(all_actions)
W_j = np.concatenate(all_weights)
print(f" Joint: front {Theta_f_j.shape}, rear {Theta_r_j.shape}")
print(f" Weights: min={W_j.min():.4f} max={W_j.max():.4f}")
# ---------- Front (no bias, all features including mu) ----------
print(f"\n --- Joint Front (no bias, with mu) ---")
front_results = fit_sindy_weighted(Theta_f_j, Y_j[:, 0], thresholds,
sample_weights=W_j, n_robust_passes=2)
front_best = max(front_results, key=lambda r: r["r2"])
_print_active(fn_f, front_best["coef"])
# ---------- Top (with bias, all features including mu) ----------
print(f"\n --- Joint Top (with bias, with mu) ---")
top_results = fit_sindy_weighted(Theta_r_j, Y_j[:, 2], thresholds,
sample_weights=W_j, n_robust_passes=2)
top_best = max(top_results, key=lambda r: r["r2"])
_print_active(fn_r, top_best["coef"])
# ---------- Bottom (independent) ----------
print(f"\n --- Joint Bottom (independent) ---")
bot_results = fit_sindy_weighted(Theta_r_j, Y_j[:, 1], thresholds,
sample_weights=W_j, n_robust_passes=2)
bot_best = max(bot_results, key=lambda r: r["r2"])
_print_active(fn_r, bot_best["coef"])
return {
"joint_scenes": scene_names,
"thresholds": thresholds,
"n_samples_total": Theta_f_j.shape[0],
"feature_names_front": fn_f,
"feature_names_rear": fn_r,
"front": {
"results": _clean_results(front_results),
"best": _clean_single(front_best),
"best_coef": front_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in front_results],
},
"top": {
"results": _clean_results(top_results),
"best": _clean_single(top_best),
"best_coef": top_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in top_results],
},
"bottom": {
"results": _clean_results(bot_results),
"best": _clean_single(bot_best),
"best_coef": bot_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in bot_results],
},
}
def run_joint_illusion(
scene_names: List[str],
thresholds: Optional[List[float]] = None,
) -> dict:
"""Joint Illusion cross-diameter SINDy with target features and scene weighting."""
if thresholds is None:
thresholds = THRESHOLDS
print(f"\n{'='*60}")
print(f"Joint Illusion cross-diameter: {scene_names}")
print(f"{'='*60}")
all_sensors, all_forces, all_actions = [], [], []
all_weights = []
t0_steps = 2000
for sn in scene_names:
sensors, forces, actions, cfg, target_forces = load_data(sn)
mu = cfg["mu"]
u0 = cfg["u0"]
dt_c = cfg.get("sample_interval", 600) / t0_steps
# Build features WITH target features, no mu
Theta_f, Theta_r, Y, fn_f, fn_r = get_feature_matrix_v2(
sensors, forces, actions, mu,
u0=u0, include_mu=False, include_cos_sin=False,
use_time_norm=False, dt_c=dt_c, n_warmup=2,
target_forces=target_forces,
)
w = compute_scene_weight(sn, cfg)
n = Theta_f.shape[0]
print(f" {sn}: {n} samples, dt_c={dt_c:.4f}, weight={w:.4f}")
all_weights.append(np.full(n, w))
all_sensors.append(Theta_f)
all_forces.append(Theta_r)
all_actions.append(Y)
Theta_f_j = np.vstack(all_sensors)
Theta_r_j = np.vstack(all_forces)
Y_j = np.vstack(all_actions)
W_j = np.concatenate(all_weights)
print(f" Joint: front {Theta_f_j.shape}, rear {Theta_r_j.shape}")
print(f" Weights: min={W_j.min():.4f} max={W_j.max():.4f}")
# Front (no bias)
print(f"\n --- Joint Front (no bias) ---")
front_results = fit_sindy_weighted(Theta_f_j, Y_j[:, 0], thresholds,
sample_weights=W_j, n_robust_passes=2)
front_best = max(front_results, key=lambda r: r["r2"])
_print_active(fn_f, front_best["coef"])
# Top (with bias)
print(f"\n --- Joint Top (with bias) ---")
top_results = fit_sindy_weighted(Theta_r_j, Y_j[:, 2], thresholds,
sample_weights=W_j, n_robust_passes=2)
top_best = max(top_results, key=lambda r: r["r2"])
_print_active(fn_r, top_best["coef"])
# Bottom (independent)
print(f"\n --- Joint Bottom (independent) ---")
bot_results = fit_sindy_weighted(Theta_r_j, Y_j[:, 1], thresholds,
sample_weights=W_j, n_robust_passes=2)
bot_best = max(bot_results, key=lambda r: r["r2"])
_print_active(fn_r, bot_best["coef"])
return {
"joint_scenes": scene_names,
"thresholds": thresholds,
"n_samples_total": Theta_f_j.shape[0],
"feature_names_front": fn_f,
"feature_names_rear": fn_r,
"front": {
"results": _clean_results(front_results),
"best": _clean_single(front_best),
"best_coef": front_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in front_results],
},
"top": {
"results": _clean_results(top_results),
"best": _clean_single(top_best),
"best_coef": top_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in top_results],
},
"bottom": {
"results": _clean_results(bot_results),
"best": _clean_single(bot_best),
"best_coef": bot_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in bot_results],
},
}
def run_single_scene_deriv(
scene_name: str,
thresholds: Optional[List[float]] = None,
verbose: bool = True,
center_diff: bool = False,
augment_level: int = 0,
feat_keys: Optional[List[str]] = None,
output_mode: str = "deriv",
) -> dict:
"""Run SINDy in phase-state mode.
output_mode:
"deriv": Y = d(alpha)/dt (time-normalized). Closed-loop needs integration.
"absolute": Y = alpha (non-dimensional absolute action). No integration.
"""
if thresholds is None:
thresholds = THRESHOLDS
sensors, forces, actions_phys, cfg, target_forces = load_data(scene_name)
mu = cfg["mu"]
u0 = cfg["u0"]
scene_id = cfg["scene_id"]
T = sensors.shape[0]
t0_steps = 2000
dt_c = cfg.get("sample_interval", 800) / t0_steps
if verbose:
print(f"\n{'='*60}")
print(f"{'ABSOLUTE' if output_mode == 'absolute' else 'DERIV'} Scene: {scene_name} dt_c={dt_c:.4f} u0={u0}")
print(f"{'='*60}")
Theta_f, Theta_r, Y, fn_f, fn_r = get_feature_matrix_deriv(
sensors, forces, actions_phys, mu,
u0=u0, dt_c=dt_c,
target_forces=target_forces,
center_diff=center_diff,
augment_level=augment_level,
feat_keys=feat_keys,
output_mode=output_mode,
include_mu=feat_keys is not None and any("mu_" in k for k in feat_keys),
)
y_label = "alpha (abs)" if output_mode == "absolute" else "d(alpha)/dt"
if verbose:
print(f" Features: front {Theta_f.shape[1]} (no bias), rear {Theta_r.shape[1]} (with bias)")
print(f" Fitting {y_label}: {Y.shape}")
w_scene = compute_scene_weight(scene_name, cfg)
# Front (no bias)
if verbose:
print(f"\n --- Front (no bias, target = {y_label}_F) ---")
front_results = fit_sindy_weighted(Theta_f, Y[:, 0], thresholds, n_robust_passes=2)
front_best = max(front_results, key=lambda r: r["r2"])
if verbose:
_print_active(fn_f, front_best["coef"])
print(f" R2={front_best['r2']:.4f} nz={front_best['nz']}")
# Top (with bias, rear shared-head)
if verbose:
print(f"\n --- Top (with bias, target = {y_label}_T) ---")
top_results = fit_sindy_weighted(Theta_r, Y[:, 2], thresholds, n_robust_passes=2)
top_best = max(top_results, key=lambda r: r["r2"])
if verbose:
_print_active(fn_r, top_best["coef"])
print(f" R2={top_best['r2']:.4f} nz={top_best['nz']}")
# Bottom (independent)
if verbose:
print(f"\n --- Bottom (independent, target = {y_label}_B) ---")
bot_results = fit_sindy_weighted(Theta_r, Y[:, 1], thresholds, n_robust_passes=2)
bot_best = max(bot_results, key=lambda r: r["r2"])
if verbose:
_print_active(fn_r, bot_best["coef"])
print(f" R2={bot_best['r2']:.4f} nz={bot_best['nz']}")
result = {
"scene": scene_name,
"scene_id": scene_id,
"mode": output_mode,
"dt_c": dt_c,
"center_diff": center_diff,
"re_code": cfg.get("re_code", 100),
"mu": mu,
"u0": u0,
"n_samples": Theta_f.shape[0],
"thresholds": thresholds,
"feature_names_front": fn_f,
"feature_names_rear": fn_r,
"front": {
"results": _clean_results(front_results),
"best": _clean_single(front_best),
"best_coef": front_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in front_results],
},
"top": {
"results": _clean_results(top_results),
"best": _clean_single(top_best),
"best_coef": top_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in top_results],
},
"bottom": {
"results": _clean_results(bot_results),
"best": _clean_single(bot_best),
"best_coef": bot_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in bot_results],
},
}
return result
def run_joint_karman_deriv(
scene_names: List[str],
thresholds: Optional[List[float]] = None,
augment_level: int = 0,
) -> dict:
"""Joint Karman cross-Re derivative-mode SINDy."""
if thresholds is None:
thresholds = THRESHOLDS
print(f"\n{'='*60}")
print(f"DERIV Joint Karman: {scene_names}")
print(f"{'='*60}")
t0_steps = 2000
all_Theta_f, all_Theta_r, all_Y = [], [], []
all_weights = []
for sn in scene_names:
sensors, forces, actions, cfg, _ = load_data(sn)
mu = cfg["mu"]
u0 = cfg["u0"]
dt_c = cfg.get("sample_interval", 800) / t0_steps
Theta_f, Theta_r, Y, fn_f, fn_r = get_feature_matrix_deriv(
sensors, forces, actions, mu,
u0=u0, dt_c=dt_c, include_mu=True,
target_forces=None, augment_level=augment_level,
)
w = compute_scene_weight(sn, cfg)
n = Theta_f.shape[0]
print(f" {sn}: {n} samples, dt_c={dt_c:.4f}, weight={w:.4f}")
all_weights.append(np.full(n, w))
all_Theta_f.append(Theta_f)
all_Theta_r.append(Theta_r)
all_Y.append(Y)
Theta_f_j = np.vstack(all_Theta_f)
Theta_r_j = np.vstack(all_Theta_r)
Y_j = np.vstack(all_Y)
W_j = np.concatenate(all_weights)
print(f" Joint: front {Theta_f_j.shape}, rear {Theta_r_j.shape}")
# Front
print(f"\n --- Joint Front ---")
front_results = fit_sindy_weighted(Theta_f_j, Y_j[:, 0], thresholds, sample_weights=W_j, n_robust_passes=2)
front_best = max(front_results, key=lambda r: r["r2"])
_print_active(fn_f, front_best["coef"])
print(f" R2={front_best['r2']:.4f}")
# Top
print(f"\n --- Joint Top ---")
top_results = fit_sindy_weighted(Theta_r_j, Y_j[:, 2], thresholds, sample_weights=W_j, n_robust_passes=2)
top_best = max(top_results, key=lambda r: r["r2"])
_print_active(fn_r, top_best["coef"])
print(f" R2={top_best['r2']:.4f}")
# Bottom
print(f"\n --- Joint Bottom ---")
bot_results = fit_sindy_weighted(Theta_r_j, Y_j[:, 1], thresholds, sample_weights=W_j, n_robust_passes=2)
bot_best = max(bot_results, key=lambda r: r["r2"])
_print_active(fn_r, bot_best["coef"])
print(f" R2={bot_best['r2']:.4f}")
return {
"joint_scenes": scene_names,
"mode": "deriv",
"thresholds": thresholds,
"n_samples_total": Theta_f_j.shape[0],
"feature_names_front": fn_f,
"feature_names_rear": fn_r,
"front": {
"results": _clean_results(front_results),
"best": _clean_single(front_best),
"best_coef": front_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in front_results],
},
"top": {
"results": _clean_results(top_results),
"best": _clean_single(top_best),
"best_coef": top_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in top_results],
},
"bottom": {
"results": _clean_results(bot_results),
"best": _clean_single(bot_best),
"best_coef": bot_best["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in bot_results],
},
}
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--scenes", type=str, default=None,
help="Comma-separated scene names")
ap.add_argument("--family", type=str, default=None,
choices=list(FAMILIES.keys()) + [None],
help="Scene family")
ap.add_argument("--out", type=str, default=None,
help="Output directory (default: sindy/<scene_id>/")
ap.add_argument("--joint", action="store_true",
help="Run joint Karman cross-Re fitting")
ap.add_argument("--deriv", action="store_true",
help="Use derivative mode: fit d(alpha)/dt = g(physics_state)")
ap.add_argument("--center-diff", action="store_true",
help="Use centered difference for derivative (vs forward diff)")
ap.add_argument("--augment-level", type=int, default=0,
help="Observation augmentation: 0=static, 1=+lags, 2=+derivs, 3=both, 4=+action_lag")
ap.add_argument("--phase", action="store_true",
help="Use phase-state features: [u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear]")
ap.add_argument("--karman-expand", action="store_true",
help="Karman expanded: phase-state + u_m/u_c/v_a/Cl_diff (10 dim)")
ap.add_argument("--karman-mu", action="store_true",
help="Karman phase-state + mu modulation: 6-dim + mu*Cl_tot/mu*Cd_tot/mu*u_a (9 dim)")
ap.add_argument("--output-mode", type=str, default="deriv",
choices=["deriv", "absolute"],
help="'deriv': predict d(alpha)/dt; 'absolute': predict alpha directly")
args = ap.parse_args()
# Resolve scene list
if args.scenes:
scene_names = [s.strip() for s in args.scenes.split(",")]
elif args.family and args.family in FAMILIES:
family_ids = FAMILIES[args.family]
if family_ids is None:
scene_names = get_scene_list() # all
else:
scene_names = []
for fid in family_ids:
scene_names.extend(get_scene_list(fid))
else:
scene_names = get_scene_list("karman")
# Determine fitting function based on mode
if args.deriv:
om = args.output_mode
if args.phase:
from SR_analysis.utils.feature_builder import PHASE_STATE_KEYS, ILLUSION_PHASE_KEYS
def _phase_func(sn):
from SR_analysis.configs import get_scene
cfg = get_scene(sn)
feat_keys = ILLUSION_PHASE_KEYS if cfg["scene_id"] == "illusion" else PHASE_STATE_KEYS
return run_single_scene_deriv(sn, center_diff=args.center_diff,
augment_level=args.augment_level,
feat_keys=feat_keys, output_mode=om)
fit_func = _phase_func
elif args.karman_expand:
from SR_analysis.utils.feature_builder import KARMAN_EXPANDED_KEYS
fit_func = lambda sn: run_single_scene_deriv(sn, center_diff=args.center_diff,
augment_level=args.augment_level,
feat_keys=list(KARMAN_EXPANDED_KEYS),
output_mode=om)
elif args.karman_mu:
from SR_analysis.utils.feature_builder import PHASE_STATE_KEYS
# Phase-state + 3 selected mu modulation features
mu_feat = ["mu_Cl_tot", "mu_Cd_tot", "mu_u_a"]
feat_keys = list(PHASE_STATE_KEYS) + mu_feat
fit_func = lambda sn: run_single_scene_deriv(sn, center_diff=args.center_diff,
augment_level=args.augment_level,
feat_keys=feat_keys,
output_mode=om)
else:
fit_func = lambda sn: run_single_scene_deriv(sn, center_diff=args.center_diff,
augment_level=args.augment_level,
output_mode=om)
joint_func_karman = lambda names: run_joint_karman_deriv(names, augment_level=args.augment_level)
else:
fit_func = run_single_scene
joint_func_karman = run_joint_karman
print(f"Running SINDy {'deriv' if args.deriv else 'v2'} for scenes: {scene_names}")
# Run per-scene
for sn in scene_names:
result = fit_func(sn)
# Save per-scene to its scene_id directory
scene_id = result["scene_id"]
out_dir = os.path.join(SINDY_DIR, scene_id)
os.makedirs(out_dir, exist_ok=True)
suffix = "_deriv" if args.deriv else "_v2"
out_path = os.path.join(out_dir, f"sindy_results{suffix}.json")
# Load existing if any, update per_scene
if os.path.isfile(out_path):
with open(out_path) as f:
existing = json.load(f)
else:
existing = {
"thresholds": THRESHOLDS,
"all_feature_names_front": result["feature_names_front"],
"all_feature_names_rear": result["feature_names_rear"],
"per_scene": {},
}
existing["per_scene"][sn] = result
with open(out_path, "w") as f:
json.dump(existing, f, indent=2)
print(f" Saved: {out_path}")
# Joint Karman cross-Re (training Re only: 50, 100, 200, 400)
if args.joint:
karman_train = ["karman_re50", "karman_re100", "karman_re200", "karman_re400"]
joint_result = joint_func_karman(karman_train)
joint_suffix = "_joint_deriv" if args.deriv else "_joint_v2"
out_path = os.path.join(SINDY_DIR, "karman", f"sindy{joint_suffix}.json")
with open(out_path, "w") as f:
json.dump(joint_result, f, indent=2)
print(f"\nSaved joint: {out_path}")
# Joint Illusion (only in non-deriv mode for now)
if args.joint and not args.deriv:
illusion_names = get_scene_list("illusion")
joint_illusion = run_joint_illusion(illusion_names)
out_path = os.path.join(SINDY_DIR, "illusion", "sindy_joint_v2.json")
os.makedirs(os.path.dirname(out_path), exist_ok=True)
with open(out_path, "w") as f:
json.dump(joint_illusion, f, indent=2)
print(f"\nSaved joint illusion: {out_path}")
if __name__ == "__main__":
main()

133
src/SR_analysis/sindy/run_illusion.py
View File

@ -1,133 +0,0 @@
"""SINDy fitting for Illusion scenes.
Usage:
conda run -n pycuda_3_10 python sindy/run_illusion.py
conda run -n pycuda_3_10 python sindy/run_illusion.py --diameters 0.75,1.0
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from typing import List, Optional
import numpy as np
_REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
if _REPO not in sys.path:
sys.path.insert(0, _REPO)
_SRC = os.path.join(_REPO, "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from SR_analysis.utils.sindy_fitter import fit_sindy, get_feature_matrix_from_data
from SR_analysis.configs import get_scene, get_scene_list
SINDY_DIR = os.path.join(os.path.dirname(__file__), "..", "sindy", "illusion")
THRESHOLDS = [0.0, 0.001, 0.002, 0.005, 0.01, 0.015, 0.02, 0.03, 0.05, 0.1]
def load_data(scene_name: str) -> tuple:
data_dir = os.path.join(os.path.dirname(__file__), "..", "data", "illusion", scene_name)
npz = np.load(os.path.join(data_dir, "controlled.npz"))
sensors = npz["sensors"].astype(np.float64)
forces = npz["forces"].astype(np.float64)
actions = npz["actions"].astype(np.float64)
return sensors, forces, actions
def run(scene_names: Optional[List[str]] = None):
if scene_names is None:
scene_names = get_scene_list("illusion")
per_scene = {}
for sn in scene_names:
print(f"\n{'='*60}")
print(f"Scene: {sn}")
print(f"{'='*60}")
cfg = get_scene(sn)
sensors, forces, actions_phys = load_data(sn)
mu = cfg["mu"]
print(f" T={sensors.shape[0]}, mu={mu:.6f}")
Theta_f, Theta_r, Y, fn_f, fn_r = get_feature_matrix_from_data(
sensors, forces, actions_phys, mu, u0=cfg["u0"],
alpha_mode=False, include_mu=True, n_warmup=2,
)
print(f" Front: {Theta_f.shape}, Rear: {Theta_r.shape}")
# Front channel
print(f"\n --- Front (no bias) ---")
front_results = fit_sindy(Theta_f, Y[:, 0], THRESHOLDS)
best_f = max(front_results, key=lambda r: r["r2"])
print(f" Best: th={best_f['threshold']:.4f} nz={best_f['nz']:2d} R2={best_f['r2']:.6f}")
# Top channel (rear shared-head)
print(f"\n --- Top (rear shared-head) ---")
top_results = fit_sindy(Theta_r, Y[:, 2], THRESHOLDS)
best_t = max(top_results, key=lambda r: r["r2"])
print(f" Best: th={best_t['threshold']:.4f} nz={best_t['nz']:2d} R2={best_t['r2']:.6f}")
# Bottom (independent)
print(f"\n --- Bottom (independent) ---")
bot_results = fit_sindy(Theta_r, Y[:, 1], THRESHOLDS)
best_b = max(bot_results, key=lambda r: r["r2"])
print(f" Best: th={best_b['threshold']:.4f} nz={best_b['nz']:2d} R2={best_b['r2']:.6f}")
per_scene[sn] = {
"scene": sn,
"re_code": cfg["re_code"],
"mu": mu,
"n_samples": Theta_f.shape[0],
"feature_names_front": fn_f,
"feature_names_rear": fn_r,
"front": {
"results": [{k: v for k, v in r.items() if k != "coef"} for r in front_results],
"best": {k: v for k, v in best_f.items() if k != "coef"},
"best_coef": best_f["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in front_results],
},
"top": {
"results": [{k: v for k, v in r.items() if k != "coef"} for r in top_results],
"best": {k: v for k, v in best_t.items() if k != "coef"},
"best_coef": best_t["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in top_results],
},
"bottom": {
"results": [{k: v for k, v in r.items() if k != "coef"} for r in bot_results],
"best": {k: v for k, v in best_b.items() if k != "coef"},
"best_coef": best_b["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in bot_results],
},
}
os.makedirs(SINDY_DIR, exist_ok=True)
out_path = os.path.join(SINDY_DIR, "sindy_results.json")
result = {"thresholds": THRESHOLDS, "per_scene": per_scene}
with open(out_path, "w") as f:
json.dump(result, f, indent=2)
print(f"\nSaved: {out_path}")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--diameters", type=str, default=None,
help="Comma-separated diameters (e.g. 0.75,1.0,1.5)")
ap.add_argument("--scene-names", type=str, default=None)
args = ap.parse_args()
if args.scene_names:
names = [s.strip() for s in args.scene_names.split(",")]
elif args.diameters:
names = [f"illusion_{d.strip()}L" for d in args.diameters.split(",")]
else:
names = None
run(names)
if __name__ == "__main__":
main()

159
src/SR_analysis/sindy/run_karman.py
View File

@ -1,159 +0,0 @@
"""SINDy fitting for Karman cloak scenes.
Runs STLSQ threshold grid for Karman scenes (training Re or all Re).
Usage:
conda run -n pycuda_3_10 python sindy/run_karman.py --re-codes 50,100,200
conda run -n pycuda_3_10 python sindy/run_karman.py
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from typing import List, Optional
import numpy as np
_REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
if _REPO not in sys.path:
sys.path.insert(0, _REPO)
_SRC = os.path.join(_REPO, "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from SR_analysis.utils.sindy_fitter import fit_sindy, get_feature_matrix_from_data
from SR_analysis.utils.feature_builder import ALL_FEAT_KEYS, U0
from SR_analysis.configs import get_scene, get_scene_list, SCENES
SINDY_DIR = os.path.join(os.path.dirname(__file__), "..", "sindy", "karman")
THRESHOLDS = [0.0, 0.001, 0.002, 0.005, 0.01, 0.015, 0.02, 0.03, 0.05, 0.1]
def load_data(scene_name: str, scene_subdir: str = "karman") -> tuple:
"""Load sensors/forces/actions from a scene's controlled.npz."""
data_dir = os.path.join(os.path.dirname(__file__), "..", "data", scene_subdir, scene_name)
npz = np.load(os.path.join(data_dir, "controlled.npz"))
sensors = npz["sensors"].astype(np.float64)
forces = npz["forces"].astype(np.float64)
actions = npz["actions"].astype(np.float64)
return sensors, forces, actions
def run(scene_names: Optional[List[str]] = None):
"""Run SINDy fitting for given scene names."""
if scene_names is None:
scene_names = get_scene_list("karman")
per_scene = {}
for sn in scene_names:
print(f"\n{'='*60}")
print(f"Scene: {sn}")
print(f"{'='*60}")
cfg = get_scene(sn)
sensors, forces, actions_phys = load_data(sn, cfg["scene_id"])
T = sensors.shape[0]
mu = cfg["mu"]
print(f" T={T}, mu={mu:.6f}")
# Build feature matrices
Theta_f, Theta_r, Y, fn_f, fn_r = get_feature_matrix_from_data(
sensors, forces, actions_phys, mu, u0=cfg["u0"],
alpha_mode=False, include_mu=True, n_warmup=2,
)
print(f" Front features: {Theta_f.shape}")
print(f" Rear features: {Theta_r.shape}")
print(f" Y: {Y.shape}")
# Front channel (ci=0, no bias)
print(f"\n --- Front (no bias) ---")
front_results = fit_sindy(Theta_f, Y[:, 0], THRESHOLDS)
best_f = max(front_results, key=lambda r: r["r2"])
print(f" Best: th={best_f['threshold']:.4f} nz={best_f['nz']:2d} R2={best_f['r2']:.6f}")
# Top channel (ci=2, rear shared-head)
print(f"\n --- Top (rear shared-head) ---")
top_results = fit_sindy(Theta_r, Y[:, 2], THRESHOLDS)
best_t = max(top_results, key=lambda r: r["r2"])
print(f" Best: th={best_t['threshold']:.4f} nz={best_t['nz']:2d} R2={best_t['r2']:.6f}")
# Bottom (independent, for comparison)
print(f"\n --- Bottom (independent) ---")
bot_results = fit_sindy(Theta_r, Y[:, 1], THRESHOLDS)
best_b = max(bot_results, key=lambda r: r["r2"])
print(f" Best: th={best_b['threshold']:.4f} nz={best_b['nz']:2d} R2={best_b['r2']:.6f}")
per_scene[sn] = {
"scene": sn,
"re_code": cfg["re_code"],
"mu": mu,
"n_samples": Theta_f.shape[0],
"n_features_front": Theta_f.shape[1],
"n_features_rear": Theta_r.shape[1],
"feature_names_front": fn_f,
"feature_names_rear": fn_r,
"front": {
"results": [{k: v for k, v in r.items() if k != "coef"}
for r in front_results],
"best": {k: v for k, v in best_f.items() if k != "coef"},
"best_coef": best_f["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"])
for r in front_results],
},
"top": {
"results": [{k: v for k, v in r.items() if k != "coef"}
for r in top_results],
"best": {k: v for k, v in best_t.items() if k != "coef"},
"best_coef": best_t["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"])
for r in top_results],
},
"bottom": {
"results": [{k: v for k, v in r.items() if k != "coef"}
for r in bot_results],
"best": {k: v for k, v in best_b.items() if k != "coef"},
"best_coef": best_b["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"])
for r in bot_results],
},
}
# Save
os.makedirs(SINDY_DIR, exist_ok=True)
out_path = os.path.join(SINDY_DIR, "sindy_results.json")
result = {
"thresholds": THRESHOLDS,
"all_feature_names_front": fn_f,
"all_feature_names_rear": fn_r,
"per_scene": per_scene,
}
with open(out_path, "w") as f:
json.dump(result, f, indent=2)
print(f"\nSaved: {out_path}")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--re-codes", type=str, default=None,
help="Comma-separated Re codes (default: all karman)")
ap.add_argument("--scene-names", type=str, default=None,
help="Comma-separated scene names (overrides --re-codes)")
args = ap.parse_args()
if args.scene_names:
names = [s.strip() for s in args.scene_names.split(",")]
elif args.re_codes:
codes = [int(r) for r in args.re_codes.split(",")]
names = [f"karman_re{rc}" for rc in codes]
else:
names = None # all karman
run(names)
if __name__ == "__main__":
main()

148
src/SR_analysis/sindy/run_pareto.py
View File

@ -1,148 +0,0 @@
"""Pareto analysis of SINDy threshold grid for any scene.
Loads sindy_results.json and prints Pareto-optimal tradeoffs.
Usage:
python sindy/run_pareto.py --scene karman_re100
python sindy/run_pareto.py --scene illusion_1L
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from typing import List, Tuple
import numpy as np
_REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
if _REPO not in sys.path:
sys.path.insert(0, _REPO)
_SRC = os.path.join(_REPO, "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
def load_sindy(scene_name: str, sindy_dir: str) -> dict:
"""Load sindy_results.json for a scene."""
path = os.path.join(sindy_dir, scene_name, "sindy_results.json")
if os.path.isfile(path):
return json.load(path)
raise FileNotFoundError(f"Missing {path}")
def pareto(points: List[Tuple[float, float]]) -> List[Tuple[float, float]]:
"""Compute Pareto frontier: lower nz and lower error is better."""
sp = sorted(points, key=lambda x: (x[0], x[1]))
front, best = [], float("inf")
for c, e in sp:
if e < best:
front.append((c, e))
best = e
return front
def fmt(fn: List[str], coef: List[float], threshold: float) -> str:
"""Format a control law string, showing terms above relative threshold."""
ca = np.array(coef, dtype=np.float64)
sc = np.max(np.abs(ca)) if np.max(np.abs(ca)) > 0 else 1.0
mask = np.abs(ca) / sc >= threshold
terms = [f"{ca[i]:+.4f}*{fn[i]}" for i in range(len(fn)) if mask[i]]
return " ".join(terms) if terms else "0"
def analyze(name: str, feat_names: List[str], channel_data: dict) -> dict:
"""Print and return Pareto analysis for one channel."""
grid = channel_data["results"]
pts = [(g["nz"], 1.0 - g["r2"]) for g in grid]
front = pareto(pts)
best = channel_data["best"]
coef = channel_data["best_coef"]
print(f"\n {name}:")
for nz, err in front:
r2 = 1.0 - err
for g in grid:
if g["nz"] == nz and abs(1.0 - g["r2"] - err) < 1e-10:
th = g["threshold"]
print(f" nz={nz:2d} R2={r2:.6f} th={th:.4f}")
if nz <= 8:
s = fmt(feat_names, coef, th)
print(f" {s[:120]}")
print(f" Best: R2={best['r2']:.6f}")
return {
"channel": name,
"best_r2": best["r2"],
"best_nz": sum(1 for c in coef if abs(float(c)) > 1e-8),
"pareto": [{"nz": nz, "r2": 1.0 - e} for nz, e in front],
}
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--scene", type=str, required=True,
help="Scene name (e.g. karman_re100, illusion_1L)")
ap.add_argument("--sindy-dir", type=str, default=None,
help="SINDy results directory")
ap.add_argument("--out", type=str, default=None,
help="Output JSON path")
args = ap.parse_args()
if args.sindy_dir is None:
args.sindy_dir = os.path.join(os.path.dirname(__file__), "..", "sindy")
# Map scene name to subdirectory
# Extract series prefix: karman_* -> karman, illusion_* -> illusion, etc.
first_part = args.scene.split("_")[0]
known_series = {"karman": "karman", "illusion": "illusion", "vortex": "vortex", "steady": "steady"}
series_dir = known_series.get(first_part, first_part)
# Try sindy/{series}/sindy_results.json
json_path = os.path.join(args.sindy_dir, series_dir, "sindy_results.json")
if not os.path.isfile(json_path):
# Fallback: try flat file
json_path = os.path.join(args.sindy_dir, "sindy_results.json")
if not os.path.isfile(json_path):
print(f"ERROR: No sindy results found for {args.scene}")
print(f" Tried: {os.path.join(args.sindy_dir, series_dir, 'sindy_results.json')}")
print(f" Tried: {json_path}")
return 1
with open(json_path) as f:
data = json.load(f)
# Look up the scene in per_scene (multi-scene format)
per = data.get("per_scene", {}).get(args.scene)
if per is not None:
fn_f = per["feature_names_front"]
fn_r = per["feature_names_rear"]
chs = [("front", fn_f, per["front"]),
("top", fn_r, per["top"]),
("bottom", fn_r, per["bottom"])]
else:
# Single-scene format
fn_f = data.get("feature_names_front")
fn_r = data.get("feature_names_rear")
if fn_f is None:
print(f"ERROR: No scene data found for {args.scene} in {json_path}")
return 1
chs = [("front", fn_f, data["front"]),
("top", fn_r, data["top"]),
("bottom", fn_r, data["bottom"])]
print(f"Pareto SR: {args.scene}")
results = {"scene": args.scene,
"channels": [analyze(*c) for c in chs]}
if args.out:
os.makedirs(os.path.dirname(args.out), exist_ok=True)
with open(args.out, "w") as f:
json.dump(results, f, indent=2)
print(f"Saved: {args.out}")
if __name__ == "__main__":
main()

160
src/SR_analysis/sindy/run_pysr.py
View File

@ -1,15 +1,11 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Restricted PySR on SINDy whitelist features. """Restricted PySR on phase-state features.
Uses the frozen whitelists (whitelist_karman.json, whitelist_illusion.json)
and the controlled CFD data to search for compact closed-form control laws.
Environment: env_sr (conda).
Usage: Usage:
conda run -n env_sr python src/SR_analysis/sindy/run_pysr.py --scene karman_re100 conda run -n sr_env python src/SR_analysis/sindy/run_pysr.py --scene illusion_1L --phase
conda run -n env_sr python src/SR_analysis/sindy/run_pysr.py --scene illusion_1L conda run -n sr_env python src/SR_analysis/sindy/run_pysr.py --scene karman_re100 --phase
conda run -n env_sr python src/SR_analysis/sindy/run_pysr.py --scene all conda run -n sr_env python src/SR_analysis/sindy/run_pysr.py --scene karman_re100 --phase --include-mu
conda run -n sr_env python src/SR_analysis/sindy/run_pysr.py --scene all --phase
""" """
from __future__ import annotations from __future__ import annotations
@ -17,6 +13,7 @@ import argparse
import json import json
import os import os
import sys import sys
import traceback
from typing import Dict, List, Optional from typing import Dict, List, Optional
import numpy as np import numpy as np
@ -31,15 +28,19 @@ sys.path.insert(0, _SRC)
from SR_analysis.utils.feature_builder import ( from SR_analysis.utils.feature_builder import (
compute_dimensionless, compute_features, build_feature_matrix, compute_dimensionless, compute_features, build_feature_matrix,
CORE_FEAT_KEYS_V2, PHASE_STATE_KEYS, ILLUSION_PHASE_KEYS, MU_FEAT_KEYS,
) )
from SR_analysis.configs import get_scene from SR_analysis.configs import get_scene
SINDY_DIR = os.path.join(os.path.dirname(__file__)) SINDY_DIR = os.path.join(os.path.dirname(__file__))
# All mu keys produced by compute_features(include_mu=True)
# (MU_FEAT_KEYS has 5, but compute_features generates an additional mu_Cl_tot)
ALL_MU_KEYS = list(MU_FEAT_KEYS) + ["mu_Cl_tot"]
def load_controlled_data(scene_name: str): def load_controlled_data(scene_name: str):
"""Load controlled data for a scene, returning sensors, forces, actions_phys, target_forces.""" """Load controlled data for a scene."""
cfg = get_scene(scene_name) cfg = get_scene(scene_name)
scene_id = cfg["scene_id"] scene_id = cfg["scene_id"]
data_dir = os.path.join(SINDY_DIR, "..", "data", scene_id, scene_name) data_dir = os.path.join(SINDY_DIR, "..", "data", scene_id, scene_name)
@ -52,34 +53,47 @@ def load_controlled_data(scene_name: str):
return sensors, forces, actions_phys, target_forces, cfg return sensors, forces, actions_phys, target_forces, cfg
def run_pysr_scene(scene_name: str, out_dir: str): def get_feature_keys(scene_id: str, use_phase: bool, include_mu: bool) -> List[str]:
"""Determine feature keys for a scene."""
if not use_phase:
# Fallback: use CORE_FEAT_KEYS_V2 style (old approach)
from SR_analysis.utils.feature_builder import CORE_FEAT_KEYS_V2
keys = list(CORE_FEAT_KEYS_V2)
elif scene_id == "illusion":
keys = list(ILLUSION_PHASE_KEYS)
else:
# karman, steady, vortex
keys = list(PHASE_STATE_KEYS)
if include_mu:
keys += ALL_MU_KEYS
return keys
def run_pysr_scene(scene_name: str, out_dir: str, use_phase: bool, include_mu: bool):
"""Run PySR on a single scene.""" """Run PySR on a single scene."""
# Select whitelist based on scene family
cfg = get_scene(scene_name) cfg = get_scene(scene_name)
scene_id = cfg["scene_id"] scene_id = cfg["scene_id"]
if scene_id == "karman":
whitelist_path = os.path.join(SINDY_DIR, "whitelist_karman.json")
elif scene_id == "illusion":
whitelist_path = os.path.join(SINDY_DIR, "whitelist_illusion.json")
else:
print(f" SKIP: no whitelist for scene_id={scene_id}")
return
with open(whitelist_path) as f:
wl = json.load(f)
sensors, forces, actions_phys, target_forces, cfg = load_controlled_data(scene_name)
mu = cfg["mu"]
u0 = cfg["u0"] u0 = cfg["u0"]
sample_interval = cfg.get("sample_interval", 800) sample_interval = cfg.get("sample_interval", 800)
t0_steps = 2000 # T0 = D/U0 = 20/0.01 = 2000 LBM steps t0_steps = 2000 # T0 = D/U0 = 20/0.01 = 2000 LBM steps
dt_c = sample_interval / t0_steps dt_c = sample_interval / t0_steps
front_keys = wl["front_active"] # Determine feature keys
rear_keys = [k for k in wl["rear_active"] if k != "bias"] front_keys = get_feature_keys(scene_id, use_phase, include_mu)
rear_keys = list(front_keys) # rear shares same physics features; bias added by build_feature_matrix
# Build features with PROPER lags (matching sindy_fitter.py) print(f"\n{'='*60}")
print(f"PySR: {scene_name} (scene_id={scene_id}, phase={use_phase}, mu={include_mu})")
print(f" Front keys ({len(front_keys)}): {front_keys}")
print(f" Rear keys ({len(rear_keys)}): {rear_keys}")
print(f"{'='*60}")
# Load data
sensors, forces, actions_phys, target_forces, cfg = load_controlled_data(scene_name)
mu_val = cfg["mu"]
# Build features with PROPER lags and sensors_raw/forces_raw for derivatives
T = sensors.shape[0] T = sensors.shape[0]
a_prev = np.zeros((T, 3), dtype=np.float64) a_prev = np.zeros((T, 3), dtype=np.float64)
a_prev2 = np.zeros((T, 3), dtype=np.float64) a_prev2 = np.zeros((T, 3), dtype=np.float64)
@ -87,29 +101,41 @@ def run_pysr_scene(scene_name: str, out_dir: str):
a_prev2[2:] = actions_phys[:-2] a_prev2[2:] = actions_phys[:-2]
dim = compute_dimensionless(sensors, forces, u0=u0, d=20.0) dim = compute_dimensionless(sensors, forces, u0=u0, d=20.0)
has_mu = any("mu" in k for k in front_keys)
sym = compute_features( sym = compute_features(
dim, a_prev, a_prev2, mu, dim, a_prev, a_prev2, mu_val,
alpha_mode=False, include_mu=has_mu, alpha_mode=False, include_mu=include_mu,
include_cos_sin=False, u0=u0, include_cos_sin=False, u0=u0,
target_forces=target_forces, target_forces=target_forces,
dt_c=dt_c, dt_c=dt_c,
sensors_raw=sensors, # CRITICAL: enables du_a_dt, dCl_tot_dt, etc.
forces_raw=forces, # CRITICAL: enables derivative features
) )
n_warmup = 2 n_warmup = 2
X_front = build_feature_matrix(sym, front_keys, add_bias=False)[n_warmup:] X_front = build_feature_matrix(sym, front_keys, add_bias=False)[n_warmup:]
X_rear = build_feature_matrix(sym, rear_keys, add_bias=True)[n_warmup:] X_rear = build_feature_matrix(sym, rear_keys, add_bias=True)[n_warmup:]
Y = actions_phys[n_warmup:] # target = omega(t) for t >= n_warmup Y = actions_phys[n_warmup:] # physical omega (target)
# Also compute non-dimensional alpha for reference
Y_alpha = Y / u0
print(f"\n{'='*60}")
print(f"PySR: {scene_name}")
print(f" Front features ({len(front_keys)}): {front_keys}")
print(f" Rear features ({len(rear_keys)}): {rear_keys}")
print(f" Samples: {X_front.shape[0]}") print(f" Samples: {X_front.shape[0]}")
print(f"{'='*60}") print(f" Y (omega) range: [{Y.min():.6f}, {Y.max():.6f}]")
print(f" alpha range: [{Y_alpha.min():.6f}, {Y_alpha.max():.6f}]")
# --- Front channel --- # Quick sanity check on derivative features before running PySR
print(f"\n=== PySR: {scene_name} Front ===") for key in front_keys:
val = sym.get(key, None)
if val is not None:
print(f" {key}: mean={np.mean(val):.6f}, std={np.std(val):.6f}")
else:
print(f" {key}: MISSING from sym dict!")
# Suffix for output filename
suffix = "_mu" if include_mu else ""
# --- Front channel (no bias, predict alpha_F) ---
print(f"\n=== PySR: {scene_name} Front (predict alpha_F) ===")
model_f = PySRRegressor( model_f = PySRRegressor(
binary_operators=["+", "-", "*", "/"], binary_operators=["+", "-", "*", "/"],
unary_operators=["square"], unary_operators=["square"],
@ -121,24 +147,26 @@ def run_pysr_scene(scene_name: str, out_dir: str):
extra_sympy_mappings={}, extra_sympy_mappings={},
batching=False, batching=False,
) )
model_f.fit(X_front, Y[:, 0], variable_names=front_keys) model_f.fit(X_front, Y_alpha[:, 0], variable_names=front_keys)
results = { results = {
"scene": scene_name, "scene": scene_name,
"scene_id": scene_id,
"channel": "front", "channel": "front",
"output": "alpha",
"feature_names": front_keys, "feature_names": front_keys,
"equations": model_f.equations_.to_dict(orient="records"), "equations": model_f.equations_.to_dict(orient="records"),
"best_sympy": str(model_f.sympy()), "best_sympy": str(model_f.sympy()),
"best_score": float(model_f.score(X_front, Y[:, 0])), "best_score": float(model_f.score(X_front, Y_alpha[:, 0])),
} }
out_path = os.path.join(out_dir, f"pysr_{scene_name}_front.json") out_path = os.path.join(out_dir, f"pysr_{scene_name}_front{suffix}.json")
with open(out_path, "w") as f: with open(out_path, "w") as f:
json.dump(results, f, indent=2, default=str) json.dump(results, f, indent=2, default=str)
print(f" Best: {model_f.sympy()}") print(f" Best sympy: {model_f.sympy()}")
print(f" Score: {results['best_score']:.4f}") print(f" Score (R2): {results['best_score']:.4f}")
print(f" Saved: {out_path}") print(f" Saved: {out_path}")
# --- Rear/Top channel --- # --- Rear/Top channel (with bias, predict alpha_T) ---
print(f"\n=== PySR: {scene_name} Top ===") print(f"\n=== PySR: {scene_name} Top (predict alpha_T, v23 shared-head) ===")
model_t = PySRRegressor( model_t = PySRRegressor(
binary_operators=["+", "-", "*", "/"], binary_operators=["+", "-", "*", "/"],
unary_operators=["square"], unary_operators=["square"],
@ -150,45 +178,61 @@ def run_pysr_scene(scene_name: str, out_dir: str):
extra_sympy_mappings={}, extra_sympy_mappings={},
batching=False, batching=False,
) )
model_t.fit(X_rear, Y[:, 2], variable_names=["bias"] + rear_keys) model_t.fit(X_rear, Y_alpha[:, 2], variable_names=["bias"] + rear_keys)
results_t = { results_t = {
"scene": scene_name, "scene": scene_name,
"scene_id": scene_id,
"channel": "top", "channel": "top",
"output": "alpha",
"feature_names": ["bias"] + rear_keys, "feature_names": ["bias"] + rear_keys,
"equations": model_t.equations_.to_dict(orient="records"), "equations": model_t.equations_.to_dict(orient="records"),
"best_sympy": str(model_t.sympy()), "best_sympy": str(model_t.sympy()),
"best_score": float(model_t.score(X_rear, Y[:, 2])), "best_score": float(model_t.score(X_rear, Y_alpha[:, 2])),
} }
out_path = os.path.join(out_dir, f"pysr_{scene_name}_top.json") out_path = os.path.join(out_dir, f"pysr_{scene_name}_top{suffix}.json")
with open(out_path, "w") as f: with open(out_path, "w") as f:
json.dump(results_t, f, indent=2, default=str) json.dump(results_t, f, indent=2, default=str)
print(f" Best: {model_t.sympy()}") print(f" Best sympy: {model_t.sympy()}")
print(f" Score: {results_t['best_score']:.4f}") print(f" Score (R2): {results_t['best_score']:.4f}")
print(f" Saved: {out_path}") print(f" Saved: {out_path}")
# Verify front formula is no-bias (action ≈ 0 when features ≈ 0)
zero_vec = np.zeros((1, len(front_keys)), dtype=np.float64)
pred_zero = float(model_f.predict(zero_vec)[0])
print(f"\n Sanity check: front prediction at zero-input = {pred_zero:.6f} (should be ~0)")
def main(): def main():
ap = argparse.ArgumentParser(description="Restricted PySR on SINDy whitelists") ap = argparse.ArgumentParser(
description="PySR on phase-state features (uses phase-state by default)"
)
ap.add_argument("--scene", type=str, default="all", ap.add_argument("--scene", type=str, default="all",
help='Scene name like "karman_re100" or "illusion_1L", or "all"') help='Scene name like "karman_re100" or "illusion_1L", or "all"')
ap.add_argument("--out", type=str, default=None) ap.add_argument("--out", type=str, default=None)
ap.add_argument("--phase", action="store_true", default=True,
help="Use phase-state features (default: True)")
ap.add_argument("--no-phase", action="store_false", dest="phase",
help="Use CORE_FEAT_KEYS_V2 instead of phase-state")
ap.add_argument("--include-mu", action="store_true", default=False,
help="Include mu (1/Re) modulation features (Karman cross-Re)")
args = ap.parse_args() args = ap.parse_args()
if args.out is None: if args.out is None:
args.out = os.path.join(SINDY_DIR, "..", "validate", "results") args.out = os.path.join(SINDY_DIR, "..", "validate", "results")
os.makedirs(args.out, exist_ok=True) os.makedirs(args.out, exist_ok=True)
# Default scenes (skip 1.5L — bang-bang regime)
if args.scene == "all": if args.scene == "all":
scenes = ["karman_re100", "illusion_0.75L", "illusion_1L", "illusion_1.5L"] scenes = ["illusion_0.75L", "illusion_1L", "karman_re100"]
else: else:
scenes = [args.scene] scenes = [args.scene]
for sn in scenes: for sn in scenes:
try: try:
run_pysr_scene(sn, args.out) run_pysr_scene(sn, args.out, args.phase, args.include_mu)
except Exception as e: except Exception as e:
print(f" ERROR on {sn}: {e}") print(f"\n ERROR on {sn}: {e}")
import traceback; traceback.print_exc() traceback.print_exc()
print("\nDone.") print("\nDone.")

133
src/SR_analysis/sindy/run_vortex.py
View File

@ -1,133 +0,0 @@
"""SINDy fitting for Vortex scenes.
Usage:
conda run -n pycuda_3_10 python sindy/run_vortex.py
conda run -n pycuda_3_10 python sindy/run_vortex.py --vortex-types lamb,taylor
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from typing import List, Optional
import numpy as np
_REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
if _REPO not in sys.path:
sys.path.insert(0, _REPO)
_SRC = os.path.join(_REPO, "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from SR_analysis.utils.sindy_fitter import fit_sindy, get_feature_matrix_from_data
from SR_analysis.configs import get_scene, get_scene_list
SINDY_DIR = os.path.join(os.path.dirname(__file__), "..", "sindy", "vortex")
THRESHOLDS = [0.0, 0.001, 0.002, 0.005, 0.01, 0.015, 0.02, 0.03, 0.05, 0.1]
def load_data(scene_name: str) -> tuple:
data_dir = os.path.join(os.path.dirname(__file__), "..", "data", "vortex", scene_name)
npz = np.load(os.path.join(data_dir, "controlled.npz"))
sensors = npz["sensors"].astype(np.float64)
forces = npz["forces"].astype(np.float64)
actions = npz["actions"].astype(np.float64)
return sensors, forces, actions
def run(scene_names: Optional[List[str]] = None):
if scene_names is None:
scene_names = get_scene_list("vortex")
per_scene = {}
for sn in scene_names:
print(f"\n{'='*60}")
print(f"Scene: {sn}")
print(f"{'='*60}")
cfg = get_scene(sn)
sensors, forces, actions_phys = load_data(sn)
mu = cfg["mu"]
print(f" T={sensors.shape[0]}, mu={mu:.6f}")
Theta_f, Theta_r, Y, fn_f, fn_r = get_feature_matrix_from_data(
sensors, forces, actions_phys, mu, u0=cfg["u0"],
alpha_mode=False, include_mu=True, n_warmup=2,
)
print(f" Front: {Theta_f.shape}, Rear: {Theta_r.shape}")
# Front channel
print(f"\n --- Front (no bias) ---")
front_results = fit_sindy(Theta_f, Y[:, 0], THRESHOLDS)
best_f = max(front_results, key=lambda r: r["r2"])
print(f" Best: th={best_f['threshold']:.4f} nz={best_f['nz']:2d} R2={best_f['r2']:.6f}")
# Top channel
print(f"\n --- Top (rear shared-head) ---")
top_results = fit_sindy(Theta_r, Y[:, 2], THRESHOLDS)
best_t = max(top_results, key=lambda r: r["r2"])
print(f" Best: th={best_t['threshold']:.4f} nz={best_t['nz']:2d} R2={best_t['r2']:.6f}")
# Bottom
print(f"\n --- Bottom (independent) ---")
bot_results = fit_sindy(Theta_r, Y[:, 1], THRESHOLDS)
best_b = max(bot_results, key=lambda r: r["r2"])
print(f" Best: th={best_b['threshold']:.4f} nz={best_b['nz']:2d} R2={best_b['r2']:.6f}")
per_scene[sn] = {
"scene": sn,
"re_code": cfg["re_code"],
"mu": mu,
"n_samples": Theta_f.shape[0],
"feature_names_front": fn_f,
"feature_names_rear": fn_r,
"front": {
"results": [{k: v for k, v in r.items() if k != "coef"} for r in front_results],
"best": {k: v for k, v in best_f.items() if k != "coef"},
"best_coef": best_f["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in front_results],
},
"top": {
"results": [{k: v for k, v in r.items() if k != "coef"} for r in top_results],
"best": {k: v for k, v in best_t.items() if k != "coef"},
"best_coef": best_t["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in top_results],
},
"bottom": {
"results": [{k: v for k, v in r.items() if k != "coef"} for r in bot_results],
"best": {k: v for k, v in best_b.items() if k != "coef"},
"best_coef": best_b["coef"],
"sparsity_curve": [(r["threshold"], r["nz"], r["r2"]) for r in bot_results],
},
}
os.makedirs(SINDY_DIR, exist_ok=True)
out_path = os.path.join(SINDY_DIR, "sindy_results.json")
result = {"thresholds": THRESHOLDS, "per_scene": per_scene}
with open(out_path, "w") as f:
json.dump(result, f, indent=2)
print(f"\nSaved: {out_path}")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--vortex-types", type=str, default=None,
help="Comma-separated vortex types (e.g. lamb,taylor)")
ap.add_argument("--scene-names", type=str, default=None)
args = ap.parse_args()
if args.scene_names:
names = [s.strip() for s in args.scene_names.split(",")]
elif args.vortex_types:
names = [f"vortex_{v.strip()}" for v in args.vortex_types.split(",")]
else:
names = None
run(names)
if __name__ == "__main__":
main()

110
src/SR_analysis/sindy/vortex/pareto_vortex_lamb.json
View File

@ -1,110 +0,0 @@
{
"scene": "vortex_lamb",
"channels": [
{
"channel": "front",
"best_r2": 0.9035051679446613,
"best_nz": 21,
"pareto": [
{
"nz": 3,
"r2": 0.8536388609680176
},
{
"nz": 8,
"r2": 0.8957625139671737
},
{
"nz": 9,
"r2": 0.8985847902462778
},
{
"nz": 16,
"r2": 0.9017891237504362
},
{
"nz": 20,
"r2": 0.9035028044287374
},
{
"nz": 21,
"r2": 0.9035051679446613
}
]
},
{
"channel": "top",
"best_r2": 0.979810012198325,
"best_nz": 22,
"pareto": [
{
"nz": 0,
"r2": -0.2926478447353347
},
{
"nz": 1,
"r2": 0.9262664550660489
},
{
"nz": 2,
"r2": 0.9602153300731789
},
{
"nz": 5,
"r2": 0.9685659511773673
},
{
"nz": 6,
"r2": 0.9752589669369754
},
{
"nz": 19,
"r2": 0.9796029724451923
},
{
"nz": 20,
"r2": 0.9797408009698567
},
{
"nz": 22,
"r2": 0.979810012198325
}
]
},
{
"channel": "bottom",
"best_r2": 0.9334422885170565,
"best_nz": 22,
"pareto": [
{
"nz": 0,
"r2": -5.4349952892154265
},
{
"nz": 1,
"r2": 0.6935730348615337
},
{
"nz": 5,
"r2": 0.8721441125489685
},
{
"nz": 7,
"r2": 0.8963211646824344
},
{
"nz": 11,
"r2": 0.9294742132351927
},
{
"nz": 15,
"r2": 0.9318911728011019
},
{
"nz": 22,
"r2": 0.9334422885170565
}
]
}
]
}

58
src/SR_analysis/sindy/vortex/pareto_vortex_taylor.json
View File

@ -1,58 +0,0 @@
{
"scene": "vortex_taylor",
"channels": [
{
"channel": "front",
"best_r2": 0.9603622630700551,
"best_nz": 21,
"pareto": [
{
"nz": 0,
"r2": -1762.7026716770156
},
{
"nz": 21,
"r2": 0.9603622630700551
}
]
},
{
"channel": "top",
"best_r2": 0.809824114603052,
"best_nz": 22,
"pareto": [
{
"nz": 0,
"r2": -3813.3981416722418
},
{
"nz": 1,
"r2": 4.909409545561516e-09
},
{
"nz": 22,
"r2": 0.809824114603052
}
]
},
{
"channel": "bottom",
"best_r2": 0.6431303693566448,
"best_nz": 22,
"pareto": [
{
"nz": 0,
"r2": -11389.175969107222
},
{
"nz": 1,
"r2": 2.5346330034814457e-08
},
{
"nz": 22,
"r2": 0.6431303693566448
}
]
}
]
}

996
src/SR_analysis/sindy/vortex/sindy_results.json
View File

@ -1,996 +0,0 @@
{
"thresholds": [
0.0,
0.001,
0.002,
0.005,
0.01,
0.015,
0.02,
0.03,
0.05,
0.1
],
"per_scene": {
"vortex_lamb": {
"scene": "vortex_lamb",
"re_code": 100,
"mu": 0.02,
"n_samples": 148,
"feature_names_front": [
"u_m",
"u_a",
"u_c",
"v_a",
"Cd_tot",
"Cd_rear",
"Cl_tot",
"Cl_diff",
"sin_ua",
"cos_ua",
"aF_lag1",
"aB_lag1",
"aT_lag1",
"daF",
"daB",
"daT",
"mu",
"mu_u_a",
"mu_v_a",
"mu_Cd_tot",
"mu_Cl_diff"
],
"feature_names_rear": [
"bias",
"u_m",
"u_a",
"u_c",
"v_a",
"Cd_tot",
"Cd_rear",
"Cl_tot",
"Cl_diff",
"sin_ua",
"cos_ua",
"aF_lag1",
"aB_lag1",
"aT_lag1",
"daF",
"daB",
"daT",
"mu",
"mu_u_a",
"mu_v_a",
"mu_Cd_tot",
"mu_Cl_diff"
],
"front": {
"results": [
{
"threshold": 0.0,
"nz": 21,
"r2": 0.9035051679446613,
"mae": 0.05502827225008196
},
{
"threshold": 0.001,
"nz": 20,
"r2": 0.9035028044287374,
"mae": 0.054966606007031876
},
{
"threshold": 0.002,
"nz": 20,
"r2": 0.9035028044287374,
"mae": 0.054966606007031876
},
{
"threshold": 0.005,
"nz": 16,
"r2": 0.9017891237504362,
"mae": 0.053710513734883655
},
{
"threshold": 0.01,
"nz": 9,
"r2": 0.8985847902462778,
"mae": 0.054491226319598914
},
{
"threshold": 0.015,
"nz": 8,
"r2": 0.8957625139671737,
"mae": 0.05306021520916354
},
{
"threshold": 0.02,
"nz": 8,
"r2": 0.8957625139671737,
"mae": 0.05306021520916354
},
{
"threshold": 0.03,
"nz": 8,
"r2": 0.8957625139671737,
"mae": 0.05306021520916354
},
{
"threshold": 0.05,
"nz": 3,
"r2": 0.8536388609680176,
"mae": 0.048637995761707624
},
{
"threshold": 0.1,
"nz": 3,
"r2": 0.8536388609680176,
"mae": 0.048637995761707624
}
],
"best": {
"threshold": 0.0,
"nz": 21,
"r2": 0.9035051679446613,
"mae": 0.05502827225008196
},
"best_coef": [
0.0072499249196507284,
-0.013406647625693383,
-0.00038101504054493135,
-0.01577043426582744,
-0.021357793291808133,
0.09538149016633536,
-0.17597430070013861,
0.01105460767691196,
0.013467531810937875,
-0.008447877114831191,
-0.009842767133851753,
0.019209968175028087,
-0.036801729600496,
0.0036358387699529284,
0.006079481319894673,
0.003342294121453103,
-0.2971343137386559,
-0.6703323813926846,
-0.7885217073102587,
-1.0678896759007668,
0.5527301613931733
],
"sparsity_curve": [
[
0.0,
21,
0.9035051679446613
],
[
0.001,
20,
0.9035028044287374
],
[
0.002,
20,
0.9035028044287374
],
[
0.005,
16,
0.9017891237504362
],
[
0.01,
9,
0.8985847902462778
],
[
0.015,
8,
0.8957625139671737
],
[
0.02,
8,
0.8957625139671737
],
[
0.03,
8,
0.8957625139671737
],
[
0.05,
3,
0.8536388609680176
],
[
0.1,
3,
0.8536388609680176
]
]
},
"top": {
"results": [
{
"threshold": 0.0,
"nz": 22,
"r2": 0.979810012198325,
"mae": 0.009405479490894075
},
{
"threshold": 0.001,
"nz": 20,
"r2": 0.9797408009698567,
"mae": 0.009464924709826631
},
{
"threshold": 0.002,
"nz": 19,
"r2": 0.9796029724451923,
"mae": 0.009503048001896178
},
{
"threshold": 0.005,
"nz": 6,
"r2": 0.9752589669369754,
"mae": 0.009083428201966606
},
{
"threshold": 0.01,
"nz": 5,
"r2": 0.9685659511773673,
"mae": 0.01027038394304665
},
{
"threshold": 0.015,
"nz": 2,
"r2": 0.9602153300731789,
"mae": 0.012152564325363832
},
{
"threshold": 0.02,
"nz": 2,
"r2": 0.9602153300731789,
"mae": 0.012152564325363832
},
{
"threshold": 0.03,
"nz": 2,
"r2": 0.9602153300731789,
"mae": 0.012152564325363832
},
{
"threshold": 0.05,
"nz": 1,
"r2": 0.9262664550660489,
"mae": 0.013270022046144844
},
{
"threshold": 0.1,
"nz": 0,
"r2": -0.2926478447353347,
"mae": 0.10377883511859723
}
],
"best": {
"threshold": 0.0,
"nz": 22,
"r2": 0.979810012198325,
"mae": 0.009405479490894075
},
"best_coef": [
1.7326565320927485,
-0.021510750075876994,
0.0007162636287450304,
0.003690011764588065,
-0.001255780396435901,
0.008069876064011276,
-0.027165643005235728,
0.019732103606457083,
-0.007023240411992188,
-0.00413143280625655,
0.0016423995129504244,
0.0008079534031552331,
-0.005050487340309005,
0.011582526033145867,
0.0010455718346845003,
0.0012590379370475096,
0.0019097817325384912,
0.0346531306245988,
0.03581318435812541,
-0.06278853147020641,
0.40349380438892807,
-0.3511614171629877
],
"sparsity_curve": [
[
0.0,
22,
0.979810012198325
],
[
0.001,
20,
0.9797408009698567
],
[
0.002,
19,
0.9796029724451923
],
[
0.005,
6,
0.9752589669369754
],
[
0.01,
5,
0.9685659511773673
],
[
0.015,
2,
0.9602153300731789
],
[
0.02,
2,
0.9602153300731789
],
[
0.03,
2,
0.9602153300731789
],
[
0.05,
1,
0.9262664550660489
],
[
0.1,
0,
-0.2926478447353347
]
]
},
"bottom": {
"results": [
{
"threshold": 0.0,
"nz": 22,
"r2": 0.9334422885170565,
"mae": 0.0062053698726041735
},
{
"threshold": 0.001,
"nz": 15,
"r2": 0.9318911728011019,
"mae": 0.00612106433470266
},
{
"threshold": 0.002,
"nz": 11,
"r2": 0.9294742132351927,
"mae": 0.006263040564511156
},
{
"threshold": 0.005,
"nz": 7,
"r2": 0.8963211646824344,
"mae": 0.006779203944819282
},
{
"threshold": 0.01,
"nz": 5,
"r2": 0.8721441125489685,
"mae": 0.007291293800356046
},
{
"threshold": 0.015,
"nz": 1,
"r2": 0.6935730348615337,
"mae": 0.009771975563999018
},
{
"threshold": 0.02,
"nz": 1,
"r2": 0.6935730348615337,
"mae": 0.009771975563999018
},
{
"threshold": 0.03,
"nz": 1,
"r2": 3.300804074513053e-12,
"mae": 0.029445380091027484
},
{
"threshold": 0.05,
"nz": 1,
"r2": 3.300804074513053e-12,
"mae": 0.029445380091027484
},
{
"threshold": 0.1,
"nz": 0,
"r2": -5.4349952892154265,
"mae": 0.0796928089615461
}
],
"best": {
"threshold": 0.0,
"nz": 22,
"r2": 0.9334422885170565,
"mae": 0.0062053698726041735
},
"best_coef": [
0.6365299396095444,
-0.00680014903595817,
-0.0014763845224131282,
0.0008331648138976364,
-0.0013963333722837737,
0.0026865900945637236,
-0.014425589645304323,
-0.008472370593784707,
-6.004332296000475e-05,
-0.0014926889561804521,
-0.0004258139447154254,
-0.0018644416734531306,
0.006364757842372448,
-0.003199874412701356,
0.001370932025212518,
0.002115447057480394,
0.0019992859663057654,
0.012730598783987558,
-0.07381922521108229,
-0.069816526655473,
0.13432950141437106,
-0.0030019765287261236
],
"sparsity_curve": [
[
0.0,
22,
0.9334422885170565
],
[
0.001,
15,
0.9318911728011019
],
[
0.002,
11,
0.9294742132351927
],
[
0.005,
7,
0.8963211646824344
],
[
0.01,
5,
0.8721441125489685
],
[
0.015,
1,
0.6935730348615337
],
[
0.02,
1,
0.6935730348615337
],
[
0.03,
1,
3.300804074513053e-12
],
[
0.05,
1,
3.300804074513053e-12
],
[
0.1,
0,
-5.4349952892154265
]
]
}
},
"vortex_taylor": {
"scene": "vortex_taylor",
"re_code": 100,
"mu": 0.02,
"n_samples": 148,
"feature_names_front": [
"u_m",
"u_a",
"u_c",
"v_a",
"Cd_tot",
"Cd_rear",
"Cl_tot",
"Cl_diff",
"sin_ua",
"cos_ua",
"aF_lag1",
"aB_lag1",
"aT_lag1",
"daF",
"daB",
"daT",
"mu",
"mu_u_a",
"mu_v_a",
"mu_Cd_tot",
"mu_Cl_diff"
],
"feature_names_rear": [
"bias",
"u_m",
"u_a",
"u_c",
"v_a",
"Cd_tot",
"Cd_rear",
"Cl_tot",
"Cl_diff",
"sin_ua",
"cos_ua",
"aF_lag1",
"aB_lag1",
"aT_lag1",
"daF",
"daB",
"daT",
"mu",
"mu_u_a",
"mu_v_a",
"mu_Cd_tot",
"mu_Cl_diff"
],
"front": {
"results": [
{
"threshold": 0.0,
"nz": 21,
"r2": 0.9603622630700551,
"mae": 5.5722956333111334e-05
},
{
"threshold": 0.001,
"nz": 0,
"r2": -1762.7026716770156,
"mae": 0.019129430850011273
},
{
"threshold": 0.002,
"nz": 0,
"r2": -1762.7026716770156,
"mae": 0.019129430850011273
},
{
"threshold": 0.005,
"nz": 0,
"r2": -1762.7026716770156,
"mae": 0.019129430850011273
},
{
"threshold": 0.01,
"nz": 0,
"r2": -1762.7026716770156,
"mae": 0.019129430850011273
},
{
"threshold": 0.015,
"nz": 0,
"r2": -1762.7026716770156,
"mae": 0.019129430850011273
},
{
"threshold": 0.02,
"nz": 0,
"r2": -1762.7026716770156,
"mae": 0.019129430850011273
},
{
"threshold": 0.03,
"nz": 0,
"r2": -1762.7026716770156,
"mae": 0.019129430850011273
},
{
"threshold": 0.05,
"nz": 0,
"r2": -1762.7026716770156,
"mae": 0.019129430850011273
},
{
"threshold": 0.1,
"nz": 0,
"r2": -1762.7026716770156,
"mae": 0.019129430850011273
}
],
"best": {
"threshold": 0.0,
"nz": 21,
"r2": 0.9603622630700551,
"mae": 5.5722956333111334e-05
},
"best_coef": [
0.00013185241334928363,
0.0013869486093058166,
-2.0689454061431898e-05,
0.0003650495654675631,
-7.646260510377518e-05,
0.0016834710794748663,
0.002138870811614627,
0.0009672545588015969,
-0.0011958663539923654,
-0.00015017143903531285,
0.007403929744140764,
-0.0015484862269750206,
-0.0003240821985992223,
-0.0010978795962499714,
-0.0002774795287867582,
-0.00041159658606479107,
0.00011190538695352297,
0.06934742736821846,
0.018252511507361284,
-0.0038231311262574906,
0.04836265570630274
],
"sparsity_curve": [
[
0.0,
21,
0.9603622630700551
],
[
0.001,
0,
-1762.7026716770156
],
[
0.002,
0,
-1762.7026716770156
],
[
0.005,
0,
-1762.7026716770156
],
[
0.01,
0,
-1762.7026716770156
],
[
0.015,
0,
-1762.7026716770156
],
[
0.02,
0,
-1762.7026716770156
],
[
0.03,
0,
-1762.7026716770156
],
[
0.05,
0,
-1762.7026716770156
],
[
0.1,
0,
-1762.7026716770156
]
]
},
"top": {
"results": [
{
"threshold": 0.0,
"nz": 22,
"r2": 0.809824114603052,
"mae": 0.00019280734797529785
},
{
"threshold": 0.001,
"nz": 1,
"r2": 4.909409545561516e-09,
"mae": 0.0004627442799109909
},
{
"threshold": 0.002,
"nz": 1,
"r2": 4.909409545561516e-09,
"mae": 0.0004627442799109909
},
{
"threshold": 0.005,
"nz": 0,
"r2": -3813.3981416722418,
"mae": 0.06224470555379584
},
{
"threshold": 0.01,
"nz": 0,
"r2": -3813.3981416722418,
"mae": 0.06224470555379584
},
{
"threshold": 0.015,
"nz": 0,
"r2": -3813.3981416722418,
"mae": 0.06224470555379584
},
{
"threshold": 0.02,
"nz": 0,
"r2": -3813.3981416722418,
"mae": 0.06224470555379584
},
{
"threshold": 0.03,
"nz": 0,
"r2": -3813.3981416722418,
"mae": 0.06224470555379584
},
{
"threshold": 0.05,
"nz": 0,
"r2": -3813.3981416722418,
"mae": 0.06224470555379584
},
{
"threshold": 0.1,
"nz": 0,
"r2": -3813.3981416722418,
"mae": 0.06224470555379584
}
],
"best": {
"threshold": 0.0,
"nz": 22,
"r2": 0.809824114603052,
"mae": 0.00019280734797529785
},
"best_coef": [
-0.003855648809544881,
-0.00016402917649747364,
-0.00930469925984968,
0.0003725286141146794,
-0.0018143741612594137,
-0.0006228672933056743,
-0.004123657830294623,
0.009617379353839065,
-0.0020693580870050163,
0.0064472830467429834,
-0.0009363904689749538,
0.002848747579355727,
0.008718629599353982,
0.006611618826853361,
0.0026442625498493562,
-0.000822158875902874,
-0.00019526846815097482,
-7.71129751901901e-05,
-0.4652349584088788,
-0.09071890000133181,
-0.031143453311218216,
-0.10346632929440941
],
"sparsity_curve": [
[
0.0,
22,
0.809824114603052
],
[
0.001,
1,
4.909409545561516e-09
],
[
0.002,
1,
4.909409545561516e-09
],
[
0.005,
0,
-3813.3981416722418
],
[
0.01,
0,
-3813.3981416722418
],
[
0.015,
0,
-3813.3981416722418
],
[
0.02,
0,
-3813.3981416722418
],
[
0.03,
0,
-3813.3981416722418
],
[
0.05,
0,
-3813.3981416722418
],
[
0.1,
0,
-3813.3981416722418
]
]
},
"bottom": {
"results": [
{
"threshold": 0.0,
"nz": 22,
"r2": 0.6431303693566448,
"mae": 0.00012115305583366485
},
{
"threshold": 0.001,
"nz": 1,
"r2": 2.5346330034814457e-08,
"mae": 0.00029202565622359403
},
{
"threshold": 0.002,
"nz": 1,
"r2": 2.5346330034814457e-08,
"mae": 0.00029202565622359403
},
{
"threshold": 0.005,
"nz": 1,
"r2": 2.5346330034814457e-08,
"mae": 0.00029202565622359403
},
{
"threshold": 0.01,
"nz": 1,
"r2": 2.5346330034814457e-08,
"mae": 0.00029202565622359403
},
{
"threshold": 0.015,
"nz": 1,
"r2": 2.5346330034814457e-08,
"mae": 0.00029202565622359403
},
{
"threshold": 0.02,
"nz": 1,
"r2": 2.5346330034814457e-08,
"mae": 0.00029202565622359403
},
{
"threshold": 0.03,
"nz": 0,
"r2": -11389.175969107222,
"mae": 0.05019249195686063
},
{
"threshold": 0.05,
"nz": 0,
"r2": -11389.175969107222,
"mae": 0.05019249195686063
},
{
"threshold": 0.1,
"nz": 0,
"r2": -11389.175969107222,
"mae": 0.05019249195686063
}
],
"best": {
"threshold": 0.0,
"nz": 22,
"r2": 0.6431303693566448,
"mae": 0.00012115305583366485
},
"best_coef": [
0.024327554221764875,
-0.0005249278989068122,
-0.0012993577286068104,
2.2680966036791235e-07,
-0.00010725143675978186,
-6.83864821520398e-05,
0.0010096871646746155,
0.010716542986185212,
0.0009908500661735154,
0.0011374615793514392,
0.0001758268322788275,
-0.0008997304730805852,
-0.0002053415257431248,
0.0003868365622255378,
0.005272105474899691,
0.0033010584355117824,
0.0027208004849191268,
0.0004865510823514911,
-0.06496786884052097,
-0.005364174292120229,
-0.003419346453559459,
0.04954205911186436
],
"sparsity_curve": [
[
0.0,
22,
0.6431303693566448
],
[
0.001,
1,
2.5346330034814457e-08
],
[
0.002,
1,
2.5346330034814457e-08
],
[
0.005,
1,
2.5346330034814457e-08
],
[
0.01,
1,
2.5346330034814457e-08
],
[
0.015,
1,
2.5346330034814457e-08
],
[
0.02,
1,
2.5346330034814457e-08
],
[
0.03,
0,
-11389.175969107222
],
[
0.05,
0,
-11389.175969107222
],
[
0.1,
0,
-11389.175969107222
]
]
}
}
}
}

30
src/SR_analysis/sindy/whitelist_illusion.json
View File

@ -1,30 +0,0 @@
{
"scene": "illusion_joint",
"threshold": 0.001,
"selection": "Cleaned features: removed aF_lag1/daF etc, replaced with daF_dt (time-normalized). Features from union across 0.75L/1L/1.5L dense fits with target_Cd/target_Cl.",
"front_active": [
"Cd_tot",
"Cd_rear",
"Cl_tot",
"Cl_diff",
"daF_dt",
"daB_dt",
"daT_dt",
"target_Cd",
"target_Cl"
],
"rear_active": [
"bias",
"Cd_tot",
"Cd_rear",
"Cl_tot",
"Cl_diff",
"daF_dt",
"daB_dt",
"daT_dt",
"target_Cd",
"target_Cl"
],
"feature_count": 16,
"note": "Cleaned Illusion whitelist: no discrete lags, only time-derivatives + physical features. Separate SINDy closed-loop sims: 0.75L=0.908, 1L=0.962, 1.5L=0.926."
}

18
src/SR_analysis/sindy/whitelist_karman.json
View File

@ -1,18 +0,0 @@
{
"scene": "karman_re100",
"threshold": 0.003,
"front_active": [
"daF_dt"
],
"rear_active": [
"bias",
"u_a",
"Cd_rear",
"Cl_diff",
"daF_dt",
"daB_dt",
"daT_dt"
],
"feature_count": 14,
"note": "Cleaned Karman whitelist: removed aF_lag1/daF, replaced with daF_dt (time-normalized). Based on th=0.003 support from joint fit. Separate SINDy closed-loop sim=0.901 (94.4% of PPO)."
}

66
src/SR_analysis/sindy/wrap_joint.py
View File

@ -1,66 +0,0 @@
#!/usr/bin/env python3
"""Wrap joint model into per_scene format.
Usage: python3 src/SR_analysis/sindy/wrap_joint.py [--scene illusion]
"""
import json, sys, os
sys.path.insert(0, os.path.abspath("."))
sys.path.insert(0, os.path.abspath("src"))
import argparse
from SR_analysis.configs import get_scene_list
def wrap_joint(scene_id):
joint_path = f"src/SR_analysis/sindy/{scene_id}/sindy_joint_v2.json"
out_path = f"src/SR_analysis/sindy/{scene_id}/sindy_joint_wrapped.json"
with open(joint_path) as f:
joint = json.load(f)
fn_f = joint["feature_names_front"]
fn_r = joint["feature_names_rear"]
wrapped = {
"thresholds": joint["thresholds"],
"feature_names_front": fn_f,
"feature_names_rear": fn_r,
"per_scene": {},
}
# Include all scenes in the family (training + generalization)
all_scene_names = get_scene_list(scene_id)
for scene_name in all_scene_names:
wrapped["per_scene"][scene_name] = {
"scene": scene_name, "re_code": "0", "mu": 0.0,
"feature_names_front": fn_f, "feature_names_rear": fn_r,
"front": {
"results": joint["front"]["results"],
"best": joint["front"]["best"],
"best_coef": joint["front"]["best_coef"],
"sparsity_curve": joint["front"]["sparsity_curve"],
},
"top": {
"results": joint["top"]["results"],
"best": joint["top"]["best"],
"best_coef": joint["top"]["best_coef"],
"sparsity_curve": joint["top"]["sparsity_curve"],
},
"bottom": {
"results": joint["bottom"]["results"],
"best": joint["bottom"]["best"],
"best_coef": joint["bottom"]["best_coef"],
"sparsity_curve": joint["bottom"]["sparsity_curve"],
},
}
with open(out_path, "w") as f:
json.dump(wrapped, f, indent=2)
print(f"Saved {out_path}")
print(f" Scenes ({len(all_scene_names)}): {all_scene_names}")
if __name__ == "__main__":
ap = argparse.ArgumentParser()
ap.add_argument("--scene", type=str, default="karman",
choices=["karman", "illusion"])
args = ap.parse_args()
wrap_joint(args.scene)

339
src/SR_analysis/sindy_sr_knoeledge.md
View File

@ -1,339 +0,0 @@
# SINDy 与 SR 背景知识
## 文档作用
这份文档只负责一件事：**给正在工作的 coder 提供背景知识、已经确认的经验、已踩过的坑和当前结论强度。**
它不是任务清单，不直接安排“下一步做什么”。凡是执行顺序、阶段划分、最小交付物，统一写在 `sindy_sr_notes`。这份 knowledge 只保留：
- 已确认的技术事实
- 历史错误与纠正
- 结果该如何理解
- 哪些话可以说，哪些话现在还不能说
- 代码和实验上最容易踩的坑
---
## 一、这条线在项目里的位置
SINDy 与 SR 不是独立课题，而是 pinball 后处理主线中的一段工具链。项目真正要解释的是：
\[
\text{obs} \rightarrow \text{act} \rightarrow \text{flow structure} \rightarrow \text{signature}
\]
SINDy 与 SR 当前只直接处理其中的 `obs -> act` 白箱化，但它们的价值在于：
- 检验控制是否真的依赖少数物理量
- 识别不同 cloak 场景中是否复用了同一类反馈结构
- 为后续把控制律与 force、mean wake、observable-related structure 接起来提供接口
因此，任何 SINDy/SR 结果都不应脱离项目总体物理主线单独解读。
---
## 二、当前已经确认的技术事实
### 1. Kármán cloak 的跨 \(Re_D\) 统一骨架存在
这是目前最硬的一批证据之一。跨 Re 的 leave-one-Re-out，尤其 holdout_200，已经显示：
- 用 Re50 + Re100 拟合，可高精度预测 Re200
- 这说明统一骨架不是偶然的特征工程产物，而是真实存在于 PPO 策略中的共享结构
### 2. 对称性问题已经纠偏
最重要的 bug 是镜像变换 \(G\) 对动作的写法错误。
**错误版本**：
\[
[a_F,a_T,a_B] \mapsto [-a_F,a_B,a_T]
\]
**正确版本**：
\[
[a_F,a_T,a_B] \mapsto [-a_F,-a_B,-a_T]
\]
也就是说：
- top / bottom 不仅交换
- 三个动作都要变号
修正后，rear equivariance 误差从约 100% 降到约 10%，原来“PPO 不尊重交换对称性”的结论应正式撤回。
当前应保留的结构关系是：
\[
\alpha_F(Gx) \approx -\alpha_F(x)
\]
\[
\alpha_B(x) \approx -\alpha_T(Gx)
\]
### 3. front no-bias 被数据支持
front 通道不需要常偏置。去掉 bias 后：
- one-step 基本不变
- 关键闭环也基本不变
因此，front odd structure 现在可以作为默认先验，而不是可选修饰。
### 4. rear shared-head 不是纯粹美学约束，而是有效结构
`bottom(x) = -top(Gx)` 的结构不是只让模型更优雅，它在闭环里确实提供了稳健性。v23 的结果说明：
- 结构约束有助于防止 rear 两通道在闭环中各走各路
- 它比无结构的独立 rear 拟合更适合作为解释模型
### 5. 无量纲化不是问题根源
这件事已经确认，不应再反复争论。
- \(u \to u/U_0\)
- \(F \to C_D, C_L\)
- \(\Omega \to \alpha\)
这些都是可逆缩放，不会丢失信息。早期 v3 崩坏来自：
- 错误 \(G\)
- 多项改动一次性叠加
而不是无量纲化本身。
### 6. one-step 与闭环是两回事
这是这条线最重要的工作方法教训之一：
\[
\text{one-step R² 高} \not\Rightarrow \text{闭环好}
\]
早期 v3(old) 就是明确反例。因此：
- one-step 只能说明局部拟合能力
- 闭环验证是核心证据，不是附加项
### 7. PySR 现在可用
之前关于“PySR 不可用”的说法应删除。当前已知：
- `sr_env` 下 PySR 可用
- 后续 SR 主工具应优先考虑 PySR
- threshold Pareto 扫描仍有价值，但不能再混称为完整 SR
---
## 三、哪些结论现在还不能说得太满
### 1. “所有 cloak 已经共享同一骨架”
还不能这么说。当前最强证据只够支持：
- Kármán cloak 跨 \(Re_D\) 统一骨架成立
- steady 初步显示出明显简化版结构
但 all-cloak 统一骨架仍是当前主问题，不是已证结论。
### 2. “steady cloak 已经严格证明是 Kármán 的子模型”
这个说法也太满。更稳的表述是：
- 在当前 steady 数据定义下，steady 的 support 呈现出 Kármán support 的明显简化版
- 这支持 `shared backbone + scene-specific activation` 方向
但 steady 当前的证据强度仍弱于 Kármán，因为 steady 不是同类 DRL 闭环策略数据。
### 3. “高 Re 退化已经证明是采样率问题”
现在还不能这么写。更稳的说法是：
- 这是一个强工作假设
- 需要在时间尺度显式化后重新检验
### 4. “SR 已经做完一轮”
如果实际做的只是 threshold 网格 + Pareto 分析，就不能写成“完整 SR 已完成”。
要区分：
- `threshold Pareto scan`
- `true constrained SR`
---
## 四、统一变量与约束的背景知识
### 1. primitive variables 应统一
后续所有 cloak 场景都应基于同一批 primitive variables：
- \(\hat u, \hat v\)
- \(C_D, C_L\)
- \(\alpha\)
- lagged \(\alpha\)
- \(\Delta\alpha\)
- \(\mu = 1/Re_D\)
- scene metadata 与 \(\Delta t_c\)
### 2. \(G\) 算子必须在 primitive level 定义
不要再直接对压缩特征猜符号。统一规则为：
| 量 | 变换 |
|---|---|
| \((u_B,u_C,u_T)\) | \((u_T,u_C,u_B)\) |
| \((v_B,v_C,v_T)\) | \((-v_T,-v_C,-v_B)\) |
| \((C_{D,F},C_{D,T},C_{D,B})\) | \((C_{D,F},C_{D,B},C_{D,T})\) |
| \((C_{L,F},C_{L,T},C_{L,B})\) | \((-C_{L,F},-C_{L,B},-C_{L,T})\) |
| \((\alpha_F,\alpha_T,\alpha_B)\) | \((-\alpha_F,-\alpha_B,-\alpha_T)\) |
| lag / increment | 同动作规则 |
| \(\mu\) | 不变 |
并且：
\[
G(G(x)) = x
\]
必须作为基本测试。
### 3. 默认结构约束
当前最稳的默认结构是：
- front no-bias
- front odd structure
- rear shared-head
即：
\[
\alpha_T(x)=g_R(x),
\qquad
\alpha_B(x)=-g_R(Gx)
\]
不再把三通道完全独立当默认。
---
## 五、SINDy 与 SR 的正确分工
### SINDy 负责什么
SINDy 的主要价值是：
- 在受限物理库上识别主项
- 给出 support 证据
- 支持跨场景比较
- 给 SR 提供 whitelist
SINDy 是骨架识别器，不是最终公式生成器。
### SR 负责什么
SR 的价值不止是压短公式。它还可以：
- 吸收若干看似分散的 SINDy 项
- 暴露不同场景是否存在同形公式
- 给出比 threshold scan 更强的闭式线索
因此 SR 应该在受限物理库上做，而不是在 raw feature 上自由乱搜。
---
## 六、时间尺度问题的背景知识
### 1. 当前公式混有采样周期信息
lagged action 和 \(\Delta a\) 都隐式绑定了 control interval。也就是说，当前公式里混着：
- 物理骨架
- 离散实现方式
- 采样周期
因此时间尺度问题是结构问题，不是附带工程问题。
### 2. 控制频率测试的严格原则
不能把在 800-step cadence 下拟合出的系数，直接拿到 400-step 或 200-step cadence 下执行，并把结果当正式证据。因为此时：
- 输入分布变了
- memory 项的物理意义变了
- \(\Delta a\) 的尺度也变了
因此，如果要比较不同 control interval，必须：
1. 在目标 \(\Delta t_c\) 下重新采集特征
2. 重新拟合模型
3. 再比较 support、系数结构与闭环
之前的频率扫描结果最多只能当线索，不能当结论。
---
## 七、steady 结果应该怎样理解
当前 steady 的结果有启发性，但需要克制解释。
可以说：
- steady front 全零很合理
- steady rear 比 Kármán 更简单
- steady 当前 support 呈现出 Kármán 的明显简化版
不宜说：
- steady 已经严格证明是 Kármán 的子模型
- steady 与 Kármán 现在证据强度相同
因为 steady 当前的数据来源与 Kármán 不完全对等。
---
## 八、代码与工程层面的已知经验
### 1. 环境分工
- `pycuda_3_10`：CFD、DRL 模型加载、数据采集、SINDy
- `sr_env`：PySR 与 SR 相关工作
### 2. 常见坑
- feature names 与矩阵列顺序不一致
- JSON 保存时未统一处理 numpy 类型
- 不同脚本用不同 channel 命名规则
- 没有统一 validator，导致 \(G\) 与闭环输入错位难以及早发现
### 3. 推荐工程习惯
- 任何场景都先过 validator，再进拟合
- separate fit 的结果按场景 × 方法存储
- support、公式、闭环三类结果必须一起保存
---
## 九、当前最值得牢记的判断
这条线现在最稳的总结是：
\[
\boxed{
\text{Kármán cloak 的跨 }Re_D\text{ 统一骨架已确认，且满足明确的镜像等变结构；v23 是当前最可信的解释模型。}
}
\]
同时必须保留另一句：
\[
\boxed{
\text{all-cloak 的最终 shared backbone 还没有定论；steady、单涡、时间尺度显式化与真正的受限 SR 仍在探索中。}
}
\]
这两句话一起保留，能避免后续工作再次滑向“把局部结果写成全局结论”。

143
src/SR_analysis/sindy_sr_knowledge.md
View File

@ -32,15 +32,29 @@ SINDy 与 SR 当前只直接处理其中的 `obs -> act` 白箱化，但它们
--- ---
## 二、当前已经确认的技术事实 ## 二、当前已经确认的技术事实（2026-06-23 更新）
### 1. Kármán cloak 的跨 Re_D 统一骨架存在 ### 1. Karman 跨 Re 联合公式成立（2026-06-23 新增）
这是目前最硬的一批证据之一。跨 Re 的 leave-one-Re-out，尤其 holdout_200，已经显示： \[
- 用 Re50 + Re100 拟合，可高精度预测 Re200 \alpha_F = \Delta a_F / \Delta t - 14.952 \cdot \mu C_{l,\text{tot}}
- 这说明统一骨架不是偶然的特征工程产物，而是真实存在于 PPO 策略中的共享结构 \]
### 2. 对称性问题已经纠偏 四个雷诺数（50/100/200/400）CFD 闭环验证全部通过，平均 0.847，远超旧基线 0.735。**这是目前最硬的统一公式证据。** 关键设计：`daF_dt` 用 `dt_c` 归一化消除采样率依赖，`mu * Cl_tot` 项自适应粘度变化。
### 2. 独立 Re PySR 公式形态各异但都成功（2026-06-23 新增）
四个雷诺数的独立公式闭环均 >0.89，但公式形态完全不同：从 Re=50 的阻力阻尼主导，逐步过渡到 Re=400 的升力主导。这反映 pinball 流态在 Re=50~400 区间经历了分岔，PPO 学会了针对性策略。
### 3. 训练分布偏移的假象项（2026-06-23 新增）
PySR 联合搜索初期返回的公式包含 `daB_dt` 项（`alpha_F = daF_dt*0.84 + ...`），但这是 PPO 轨迹中前后端动作共线导致的**虚假相关**。部署时后端恒速旋转，`daB_dt = 0`，该项无贡献。**最终公式必须手动审查，去除此类假象项。**
### 4. Kármán cloak 的跨 Re_D 统一骨架存在
这是旧的事实，与上述联合公式结论一致。
### 5. 对称性问题已经纠偏
最重要的 bug 是镜像变换 G 对动作的写法错误。 最重要的 bug 是镜像变换 G 对动作的写法错误。
@ -56,9 +70,11 @@ SINDy 与 SR 当前只直接处理其中的 `obs -> act` 白箱化，但它们
修正后，rear equivariance 误差从约 100% 降到约 10%，原来"PPO 不尊重交换对称性"的结论应正式撤回。 修正后，rear equivariance 误差从约 100% 降到约 10%，原来"PPO 不尊重交换对称性"的结论应正式撤回。
### 3. front no-bias 被数据支持，rear shared-head 是有效结构 ### 6. front no-bias 被数据支持，rear shared-head 是有效结构
### 4. one-step R² 与闭环是两回事 所有独立和联合公式均验证：前端公式无偏置项（零输入时输出接近零）。
### 7. one-step R² 与闭环是两回事
\[ \[
\text{one-step R}^2 \gg 0.95 \not\Rightarrow \text{闭环好} \text{one-step R}^2 \gg 0.95 \not\Rightarrow \text{闭环好}
@ -66,81 +82,126 @@ SINDy 与 SR 当前只直接处理其中的 `obs -> act` 白箱化，但它们
**关键证据**：full-lag 16-dim 模型 R²=0.939 但闭环仅 0.619；static 8-dim 模型 R²=0.321 但闭环 0.745。 **关键证据**：full-lag 16-dim 模型 R²=0.939 但闭环仅 0.619；static 8-dim 模型 R²=0.321 但闭环 0.745。
### 5. 时间特征加剧 rollout mismatch（2026-06-15 确认） ### 8. Illusion 0.75L 和 1L 的 PySR 公式已跑通（2026-06-23 新增）
训练时使用 `x(t-1)` 或 `dx/dt` 特征让 one-step R² 跃升，但闭环验证下降。原因是**分布偏移**：训练时特征来自 PPO 真实轨迹，部署时来自 SINDy 控制轨迹。带时间记忆的模型更容易在自由滚动时暴露分布偏移。 | 场景 | Front 公式 | 闭环 | % of PPO |
|------|-----------|:----:|:--------:|
| 0.75L | `-0.169*(Cl_tot + dCl_tot_dt) - 1.240` | **0.979** | 100.7% |
| 1L | `(du_a_dt + u_a + 26.5)*0.0123` | **0.957** | 98.4% |
### 6. phase-state + absolute action 路线已被验证（2026-06-15 核心结论） 两公式结构不同（0.75L 用压升力、1L 用相位），说明**目标尺寸差异导致控制机制不同**。
Illusion 0.75L 和 1L 已证明**不需要动作历史特征**也能达到 PPO 的 96%+ 闭环性能： ### 9. Illusion 1.5L 是独立机制（2026-06-25 修正）
- 输入：相位状态 `u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear` + error-state
- 输出：直接预测 absolute alpha，不做导数积分
- 结构：v23（front no-bias, rear shared-head）
### 7. Illusion 1.5L 是独立机制（2026-06-15 确认） 1.5L 之前被描述为"bang-bang（动作饱和）"，但实际分析后发现并非如此：
- 动作使用完整 [-1, 1] 范围但仅在极端值处饱和约 1% 的时间
- 控制是**高频率、高振幅的周期调制**，频率 f=0.24（控制空间），是目标涡脱频率的 5.6 倍
- 所有 10 维 ILLUSION_PHASE 特征与动作之间的线性相关性均 < 0.33
- 动作自相关 lag-2 ≈ -0.9，说明每两步切换一次方向
- 标准 PySR/SINDy 均不适用（已确认 PySR 给出 `aF_lag1 * 0.01` 的平凡解）
1.5L 动作饱和到 [-8,8] 范围，自相关 r=0.07，线性 SINDy 不适用。 ### 10. Illusion 联合公式已验证（2026-06-25 新增）
联合公式（0.75L + 1L 数据拼接，无 target_diameter 标记）：
- Front: `Cd_tot - (Cd_err + 5.428) - 0.00978*(du_a_dt + u_a)`，R2=0.907
- Top: `(Cd_err - (Cd_rear - Cl_err))*0.535 + 2.782`，R2=0.828
- CFD 闭环：**0.75L=0.978**，**1L=0.970**
- 联合公式能在两场景上都取得接近最优的效果，说明控制器通用性强
方法 A（target_diameter 标记法）确认：target_diameter **被最优公式使用**，说明两场景公式结构确实不同。
### 11. Karman 联合公式泛化到 Vortex 场景（2026-06-25 新增）
Karman 跨 Re 联合 PySR 公式直接应用于 vortex cloak 场景（未重新训练）：
- vortex_lamb（Lamb 偶极子）：CFD 闭环 **0.949**（超越 PPO 基线 0.942）
- vortex_taylor（Taylor 单极子）：CFD 闭环 **0.905**（接近 PPO 基线 0.916）
- 说明联合公式提取的物理反馈结构具有很强的通用性
### 12. Karman re400 最优控制间隔（2026-06-25 新增）
re400 的 PPO 效果较差（相似度 0.795），怀疑是采样间隔 SI=800 过长。
测试不同 SI 下联合公式的 CFD 闭环表现：
- SI=800: 0.806（基线）
- **SI=400: 0.819（最优）**
- SI=200: 0.794
结论：re400 最优控制间隔为 SI=400，但仍受限于 PPO 本身在高 Re 下的控制质量。
--- ---
## 三、哪些结论现在还不能说得太满 ## 三、哪些结论现在还不能说得太满
### 1. "所有 cloak 已经共享同一骨架" ### 1. "所有 cloak 已经共享同一骨架"
还不能这么说。all-cloak 统一骨架仍是当前主问题。 不能。Karman 和 Illusion 的公式结构不同，不能统一。
### 2. "Karman 新路线已经成功" ### 2. "Karman 跨 Re 联合公式已完全解决"
不能。Karman 无动作历史最佳仅 0.699（旧 v23 为 0.901），说明状态还不充分。 不能。后端为常数（`alpha_T = 3.414`），说明后端控制信息未被充分利用。且 re400 独立公式未做闭环验证。
### 3. "高 Re 退化已经证明是采样率问题" ### 3. "高 Re 退化已经证明是采样率问题"
这是一个强工作假设，需要在时间尺度显式化后重新检验。 这是一个强工作假设，需要在时间尺度显式化后重新检验。
### 4. "SR 已经做完一轮" ### 4. "SR 已经做完一轮"
如果实际做的只是 threshold 网格 + Pareto 分析，不能写成"完整 SR 已完成"。 Karman 完整，Illusion 联合公式已验证 + 结构比较已完成，1.5L 已分析为不可 SR 的独立机制。
--- ---
## 四、代码与工程层面的已知经验 ## 四、代码与工程层面的已知经验
### 1. 环境分工 ### 1. 环境分工
- `pycuda_3_10`：CFD、DRL 模型加载、数据采集、SINDy（pysindy） - `pycuda_3_10`：CFD、DRL 模型加载、数据采集、SINDy（pysindy）、CFD 闭环验证
- `sr_env`：PySR 与 SR 相关工作 - `sr_env`：PySR 符号回归
### 2. 常见坑 ### 2. 常见坑
- **actions.npz 是归一化动作 [-1,1]，不是物理 omega**。所有 SINDy 拟合代码需通过 `(norm * scale + bias) * u0` 转换。 - **actions.npz 是归一化动作 [-1,1]，不是物理 omega**。所有拟合代码需通过 `(norm * scale + bias) * u0` 转换。
- **Illusion "2U" 误解**："2U" 在模型名中表示 S_DIM=14（多了2维目标力观测），不是两倍来流速度。u0 始终为 0.01。 - **Illusion "2U" 误解**："2U" 在模型名中表示 S_DIM=14（多了2维目标力观测），不是两倍来流速度。u0 始终为 0.01。
- **FIFO bias ≠ DRL action bias**：FIFO bias 用 1U `[0, -U0, U0]` 填充历史；DRL action decoder 用 2U `action*8+[0,-2,2]`。 - **FIFO bias ≠ DRL action bias**：FIFO bias 用 1U `[0, -U0, U0]` 填充历史；DRL action decoder 用 2U `action*8+[0,-2,2]`。
- **SAMPLE_INTERVAL 因场景而异**：0.75L=400, 1L=600, 1.5L=800, Karman=800。 - **SAMPLE_INTERVAL 因场景而异**：0.75L=400, 1L=600, 1.5L=800, Karman=800。
- **验证器 `n_steps` 不能过短**：需保证控制传播到传感器，S=400→320步, S=600→214步, S=800→160步。 - **验证器 `n_steps` 不能过短**：需保证控制传播到传感器，S=400→320步, S=600→214步, S=800→160步。
- **保存 `controlled.npz` 时必须包含 `target_forces` 字段**（Illusion 场景的 s_dim=14 推理数据）。 - **保存 `controlled.npz` 时必须包含 `target_forces` 字段**（Illusion 场景的 s_dim=14 推理数据）。
- **单步 validator（predict_v23_deriv）中需要传入 `sensors_raw`/`forces_raw`** 以计算相位特征中的导数项。 - **单步 validator 中需要传入 `sensors_raw`/`forces_raw`** 以计算相位特征中的导数项。
- **PySR 搜索时必须传 `sensors_raw`/`forces_raw`** 给 `compute_features()`，否则所有 `du_a_dt`, `dCl_tot_dt` 等导数特征均为零。这是 run_pysr.py 的 CRITICAL bug（已修复）。
- **输出目标必须是 alpha 而非 omega**：`Y = actions_phys / u0`，否则 coefficients 受 u0 缩放。
- **联合公式中的 `daB_dt` 是训练分布假象**：PPO 轨迹中前后端动作共线导致虚假相关，部署时后端恒速该项为零，应手动移除。
### 3. 当前可信结果（2026-06-15） ### 3. 当前可信结果（2026-06-25 更新）
| 场景 | 路线 | 特征 | 输出 | 是否有动作历史 | 闭环 sim | % of PPO | | 场景 | 路线 | 特征 | 输出 | 动作历史? | 闭环 sim | % of PPO |
|------|------|------|:----:|:-------------:|:--------:|:--------:| |------|------|------|:----:|:---------:|:--------:|:--------:|
| illusion_0.75L | phase+error | ILLUSION_PHASE (10dim) | alpha | **否** | **0.974** | 100.2% | | illusion_0.75L | PySR | ILLUSION_PHASE (10dim) | alpha | 否 | **0.979** | **100.7%** |
| illusion_1L | phase+error | ILLUSION_PHASE (10dim) | alpha | **否** | **0.958** | 98.5% | | illusion_1L | PySR | ILLUSION_PHASE (10dim) | alpha | 否 | **0.957** | 98.4% |
| illusion_1.5L | phase+error | ILLUSION_PHASE (10dim) | alpha | **否** | **N/A** | bang-bang | | **illusion joint** | **PySR** | **ILLUSION_PHASE (10dim)** | **alpha** | **否** | **0.978/0.970** | — |
| karman_re100 | phase-state | PHASE_STATE (6dim) | alpha | **否** | **0.699** | 73.3% | | illusion_1.5L | — | — | — | — | N/A | 高频周期调制 |
| karman_re100 | old v23 | CORE_FEAT_KEYS_V2 + a_lag | alpha | **是** | **0.901** | 94.4% | | **karman 联合** | **PySR** | **PHYS_DADT+mu (17dim)** | **alpha** | **daF_dt** | **re50=0.847, re100=0.888, re200=0.845, re400=0.806** | **统一公式** |
| karman_re50 | PySR | PHYS_DADT+mu | alpha | daF_dt | **0.895** | **153.8%** |
| karman_re100 | PySR | PHYS_DADT+mu | alpha | daF_dt | **0.888** | 98.6% |
| karman_re200 | PySR | PHYS_DADT+mu | alpha | daF_dt | **0.916** | **115.5%** |
| karman_re400 (SI=400) | PySR | PHYS_DADT+mu | alpha | daF_dt | **0.819** | 最优SI |
| **vortex_lamb** | **Karman joint** | — | alpha | daF_dt | **0.949** | **超越PPO(0.942)** |
| **vortex_taylor** | **Karman joint** | — | alpha | daF_dt | **0.905** | 接近PPO(0.916) |
--- ---
## 五、Bug Audit 补充（2026-06-14~15） ## 五、Bug Audit 全部记录
### 新发现的 Bug ### 2026-06-14~15 发现的 Bug
| # | Bug | 文件 | 严重度 | 修复 | | # | Bug | 文件 | 严重度 | 修复 |
|---|-----|------|--------|------| |---|-----|------|--------|------|
| 13 | `run_pysr.py` 滞后构造错误：`actions_phys` 直接当 `actions_prev` 传入 `compute_features`，导致 `aF_lag1 = alpha(t)` 而非 `alpha(t-1)` — 拟合恒等式 | sindy/run_pysr.py | CRITICAL | 改为正确滞后：`a_prev[1:]=actions_phys[:-1]` | | 13 | `run_pysr.py` 滞后构造错误：`actions_phys` 直接当 `actions_prev` 传入 `compute_features`，导致 `aF_lag1 = alpha(t)` 而非 `alpha(t-1)` — 拟合恒等式 | sindy/run_pysr.py | CRITICAL | `a_prev[1:]=actions_phys[:-1]` |
| 14 | `predict_v23_deriv` 中 `needs_aug` 只检测 `"lag1"` 关键字，忽略 `"_dt"` 结尾的导数特征 → 闭环中 phase-state 的 `du_a_dt` 等始终为零 | validate/run_closed_loop.py | CRITICAL | `needs_aug = any(k.endswith("_dt") or k.endswith("_lag1") for k in ...)` | | 14 | `predict_v23_deriv` 中 `needs_aug` 只检测 `"lag1"` 关键字，忽略 `"_dt"` 结尾的导数特征 → 闭环中 phase-state 的 `du_a_dt` 等始终为零 | validate/run_closed_loop.py | CRITICAL | `needs_aug = any(k.endswith("_dt") or k.endswith("_lag1") ...)` |
| 15 | `compute_features` 中 `target_forces` 未处理 1D 输入（单步验证时 shape 为 (2,) 而非 (1,2)） | utils/feature_builder.py | 中 | `if tf.ndim == 1: tf = tf.reshape(1, -1)` | | 15 | `compute_features` 中 `target_forces` 未处理 1D 输入 | utils/feature_builder.py | 中 | `if tf.ndim == 1: tf = tf.reshape(1, -1)` |
### 关键方法教训（2026-06-15 更新） ### 2026-06-23 修复的关键 Bug
| # | Bug | 文件 | 严重度 | 修复 |
|---|-----|------|--------|------|
| **16** | **`run_pysr.py` 调用 `compute_features()` 时未传 `sensors_raw`/`forces_raw`**，导致所有 `du_a_dt`, `dCl_tot_dt`, `dCd_err_dt`, `dCl_err_dt` 等相位导数特征在 `sym` 字典中缺失 → `build_feature_matrix()` 用全零填充 → PySR 用零向量拟合真实动作 → 公式必然古怪 | **sindy/run_pysr.py** | **CRITICAL** | 在 `compute_features()` 调用中增加 `sensors_raw=sensors, forces_raw=forces` |
### 关键方法教训
1. **PPO 验证优先于 SINDy**。在确认 PPO 推理复现正确之前，SINDy 结果无意义。 1. **PPO 验证优先于 SINDy**。在确认 PPO 推理复现正确之前，SINDy 结果无意义。
2. **控制时长必须 >= NX/U0**。50 步验证结果不可靠。 2. **控制时长必须 >= NX/U0**。50 步验证结果不可靠。
3. **DDF 保存位置决定验证器起始状态**。正确模式：`stabilize → save_ddf(1) → norm → apply_ddf → bias FIFO → save_ddf(2) → apply_ddf`。 3. **DDF 保存位置决定验证器起始状态**。正确模式：`stabilize → save_ddf(1) → norm → apply_ddf → bias FIFO → save_ddf(2) → apply_ddf`。
4. **状态消融比输出消融更重要**：`x_n` 不够 → 加 `x_{n-1}` 大幅提升 → 加 `a_{n-1}` 边际仅 3% → 说明缺的是相位状态，不是动作历史。 4. **状态消融比输出消融更重要**：`x_n` 不够 → 加 `x_{n-1}` 大幅提升 → 加 `a_{n-1}` 边际仅 3% → 说明缺的是相位状态，不是动作历史。
5. **离线 rollout 好 ≠ CFD 闭环好**。phase-state 模型 offline 50 步漂移仅 11%，但 CFD 闭环 -7% vs 静态。最终判据是 CFD 短闭环。 5. **离线 rollout 好 ≠ CFD 闭环好**。最终判据是 CFD 短闭环。
6. **如遇数值错误，先检查 obs 切片索引、添加顺序、target_forces 维度**。 6. **联合公式最终必须人工审查**：移除训练分布中的假象项（如 `daB_dt` 在后端恒速时恒为零）。
7. **PySR 输出的最佳公式不一定合适**：因为 SR 只优化 loss，不考虑部署分布偏移。检查 Pareto 前沿，理解每个项在闭环中是否真正有贡献。

264
src/SR_analysis/sindy_sr_notes.md
View File

@ -1,188 +1,138 @@
# SINDy 与 SR 执行计划（2026-06-15 修订版 v4） # SINDy-SR Task List & Execution Notes
## 文档作用 > This file is the **active task list** for SINDy / PySR symbolic regression work.
> It is referenced by `sindy_sr_knowledge.md` as the source of truth for execution
这份文档只回答一件事：**接下来要做什么。** > order, phase boundaries, and current task status.
>
- 只写执行路线、阶段目标、优先级、输出要求 > Background knowledge, confirmed facts, bug history, and result interpretation
- 不长篇复述历史争论 > belong in `sindy_sr_knowledge.md`. This file only tracks **what to do next**.
- 凡是背景知识、已知结论、踩坑记录，统一放到 `sindy_sr_knowledge.md`
--- ---
## 当前总目标 ## 1. Current Task Status (2026-06-28)
用 SINDy + SR 在所有 cloak / illusion 场景上产出有物理结论的控制律公式，并验证其跨场景泛化能力。 ### Completed
回答三个问题： - [x] Karman cross-Re joint formula: `α_F = daF_dt − 14.952·μ·Cl_tot`, `α_T = 3.414` (constant)
1. **哪些物理量是控制的核心变量？**（力、速度、相位、误差之间的取舍） - [x] Karman deep individual per-Re formulas (niter=120)
2. **不同场景是否共享同一公式骨架？**（family 内 -> family 间） - [x] Karman re400 short-SI test (SI=400 optimal at 0.819)
3. **白箱控制律能否在未训练工况工作？**（泛化性 + 失效边界） - [x] Illusion 0.75L individual PySR: `α_F = −0.169·(Cl_tot + dCl_tot/dt) − 1.240`
- [x] Illusion 1L individual PySR: `α_F = (du_a/dt + u_a + 26.5)·0.0123`
- [x] Illusion joint formula (Method B): `α_F = Cd_tot − (Cd_err + 5.428) − 0.00978·(du_a/dt + u_a)`, CFD 0.75L=0.978, 1L=0.970
- [x] Illusion generalization CFD (0.5L−2.0L) via joint formula
- [x] Vortex generalization (Karman joint formula): lamb=0.949, taylor=0.905
- [x] Illusion 1.5L characterized as high-frequency periodic modulation (not SR-amenable)
- [x] G-equivariance bug fixed
- [x] Bug #13/#14/#15/#16 all fixed
- [x] `docs/SR_analysis_report.md` written (465 lines)
- [x] `docs/illusion_joint_formula_analysis.md` written
### In Progress (Phase P0 — 2026-06-28)
- [ ] P0.1: Triage 136 JSON result files → classify, archive intermediates
- [ ] P0.2: Create `validate/results/README.md` (reference table for canonical files)
- [ ] P0.3: Update `README.md` directory tree and Key Documentation table
- [ ] P0.4: Update `SR_analysis_report.md` §8 with generalization rows
- [ ] P0.5: Fix doc error in `illusion_joint_formula_analysis.md` line 46 (1.5L value)
### Pending (Phase P2)
- [ ] P2.1: Regenerate PPO rollouts for 0.75L, 1L, 1.5L
- [ ] P2.2: Run SR closed-loop CFD for 1.2L and 2L with full telemetry
- [ ] P2.3: Create `scripts/analyze_illusion_degradation.py`
- [ ] P2.4: Document degradation analysis in `illusion_joint_formula_analysis.md` §2.3
### Pending (Phase P3)
- [ ] P3.1: Run 1L with-target CFD validation
- [ ] P3.2: Document results
--- ---
## 已完成的工作（2026-06-14~15） ## 2. Known Open Items (Cannot Claim Closure Yet)
### 核心成果 From `sindy_sr_knowledge.md` §三:
1. **Phase-state 特征体系**：`PHASE_STATE_KEYS` (6维) + `ILLUSION_PHASE_KEYS` (10维) — 无需动作历史 1. **"All cloak share same skeleton"** — Cannot claim. Karman and Illusion formulas are structurally different.
2. **绝对动作输出模式**：`get_feature_matrix_deriv(output_mode="absolute")` — 无积分累积 2. **"Karman cross-Re joint formula fully solved"** — Cannot claim. Rear is constant (α_T = 3.414), rear control information not fully utilized.
3. **闭合验证器支持**：`predict_v23_deriv(output_mode="absolute")` + 闭环 `mode="abs"` 3. **"High-Re degradation proven to be sampling rate issue"** — Strong hypothesis, not proven.
4. **离线 rollout 评估**：`validate/eval_rollout.py` — 多步滚动误差分析 4. **"SR is completely done"** — Karman is complete. Illusion has gaps: 1.5L mechanism unexplained, 1L with-target untested, joint + with-target not run.
5. **消融实验**：5 种输入配置 × 2 种输出模式 × CFD 闭环验证
### Illusion 新路线成功
| 场景 | 闭环 sim | 动作历史 | 结论 |
|------|:--------:|:--------:|------|
| 0.75L | **0.974** | 无 | 新路线成立 |
| 1L | **0.958** | 无 | 新路线成立 |
| 1.5L | N/A | 无 | bang-bang/不同机制 |
### Karman 需要补状态
| 配置 | 闭环 sim | 对比旧 v23 |
|------|:--------:|:----------:|
| 旧 v23 (a_lag) | 0.901 | 基线 |
| phase-state → abs (最佳新) | **0.699** | -22% |
| phase-state → deriv | 0.656 | -27% |
--- ---
## 当前最值得优先做的实验 ## 3. Unfinished Work from Handover (2026-06-25)
### 第一优先级：Illusion 0.75L / 1L → separate PySR → 公式比较 | # | Item | Priority |
|---|------|----------|
输入特征（10 维）： | 1 | Rebuild `sindy_sr_notes.md` | P0 |
```python | 2 | Write generalization results table into `SR_analysis_report.md` §8 | P0 |
ILLUSION_PHASE_KEYS = [ | 3 | Fix `illusion_joint_formula_analysis.md` 1.5L line 46 value | P0 |
"u_a", "du_a_dt", # 振荡相位 | 4 | README directory tree: add `gen_illusion_target.py`, `batch_illusion_generalization.sh` | P0 |
"Cl_tot", "dCl_tot_dt", # 升力动力学 | 5 | README Key Documentation table: add `illusion_joint_formula_analysis.md` | P0 |
"Cd_tot", "Cd_rear", # 阻力反馈 | 6 | Create `validate/results/README.md` | P0 |
"Cd_err", "Cl_err", # 力误差 | 7 | Illusion large-diameter deep analysis (1.5L/2.0L) | P2 |
"dCd_err_dt", "dCl_err_dt", # 误差动力学 | 8 | 1L with-target CFD validation | P3 |
] | 9 | Illusion joint + with-target search | Future |
``` | 10 | re400 independent PySR formula CFD | Future |
| 11 | Erase scene (no PPO data yet) | Blocked |
输出：absolute alpha（非维动作，不积分）
环境：`conda run -n sr_env`
代码：`sindy/run_pysr.py`
三个产出要求：
- 最短可接受公式
- 闭环可运行公式
- 0.75L 与 1L 的公式形态比较 / 同形骨架判断
### 第二优先级：Karman 状态补强（CCD/OID 进场）
当前 Karman 新路线最佳 0.699，需要在 phase-state 基础上补充缺失状态量：
- 候选：回流区长度、尾迹中心线偏移、POD 模态系数
- 方法：CCD（Correlation-based Decomposition）或 OID（Observer-based Identification）
- 目标：在无动作历史前提下将 Karman 闭环提升到 0.85+
### 第三优先级：Illusion 跨直径联合
条件：0.75L 和 1L 的 PySR 公式确认同形骨架后
方法：error-state 特征 + 目标 signature 描述符（target St、amplitude）
--- ---
## 当前暂不建议做的事 ## 4. Recommended Formula Files
- 不回退到动作历史主导（Illusion 已经证明不需要） ### Canonical (use these for all papers/reports)
- 不先做跨 Re Karman 联合拟合（单 Re 还没站稳）
- 不先做 vortex 扩展（不是当前瓶颈） | Scene | Front JSON | Top JSON | CFD Similarity |
- 不先上 PySR 到 Karman（状态还有问题，公式不会更物理） |-------|-----------|----------|:---:|
| Karman cross-Re (joint) | `validate/results/karman_joint_deep_front.json` | `validate/results/karman_joint_deep_top.json` | 0.847 avg |
| Illusion joint (0.75L+1L) | `validate/results/pysr_illusion_joint_front.json` | `validate/results/pysr_illusion_joint_top.json` | 0.978/0.970 |
| Karman re50 (individual) | `validate/results/pysr_karman_re50_front_deep.json` | `validate/results/pysr_karman_re50_top_deep.json` | 0.895 |
| Karman re100 (individual) | `validate/results/pysr_karman_re100_front_deep.json` | `validate/results/pysr_karman_re100_top_deep.json` | 0.888 |
| Karman re200 (individual) | `validate/results/pysr_karman_re200_front_deep.json` | `validate/results/pysr_karman_re200_top_deep.json` | 0.916 |
| Illusion 0.75L (individual) | `validate/results/pysr_illusion_0.75L_front.json` | `validate/results/pysr_illusion_0.75L_top.json` | 0.979 |
| Illusion 1L (individual) | `validate/results/pysr_illusion_1L_front.json` | `validate/results/pysr_illusion_1L_top.json` | 0.957 |
--- ---
## 常用命令 ## 5. Priority Markers
| Marker | Meaning |
|--------|---------|
| **P0** | Must complete for paper-ready state |
| **P1** | Analysis deepening (defer until P0 done) |
| **P2** | Research extension (1.5L/2.0L mechanism) |
| **P3** | Optional micro-experiments |
| **Future** | Nice-to-have, not urgent |
| **Blocked** | Prerequisites not met |
---
## 6. Quick Commands Reference
### SINDy 拟合
```bash ```bash
# Illusion phase-state + absolute action # PPO inference (pycuda_3_10, GPU 2)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \ conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_karman.py --re 100 --device 2
--scenes illusion_0.75L,illusion_1L --deriv --phase --output-mode absolute conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_illusion.py --diameter 1.0 --device 2
conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_vortex.py --type lamb --device 2
# Karman phase-state + absolute action # Target generation for generalization scenes
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \ conda run -n pycuda_3_10 python src/SR_analysis/scripts/gen_illusion_target.py --scene all --device 2
--scenes karman_re100 --deriv --phase --output-mode absolute
# Karman expanded features # PySR search (sr_env)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \ conda run -n sr_env python src/SR_analysis/sindy/run_pysr_deep.py --both
--scenes karman_re100 --deriv --karman-expand --output-mode absolute conda run -n sr_env python src/SR_analysis/sindy/run_pysr_deep_illusion.py --both
# Karman phase-state + mu modulation # CFD closed-loop validation (pycuda_3_10, GPU 2)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \ conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scenes karman_re100 --deriv --karman-mu --output-mode absolute --scene karman_re100 --device 2 --steps 200 --mode pysr \
--pysr-front validate/results/karman_joint_deep_front.json \
--pysr-top validate/results/karman_joint_deep_top.json
# Illusion separate with error-state (old style for comparison)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes illusion_0.75L,illusion_1L,illusion_1.5L
# Karman cross-Re joint
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes "karman_re50,karman_re100,karman_re200,karman_re400" --joint
```
### CFD 闭环验证
```bash
# Illusion phase-state + absolute action
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop_illusion.py \ conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop_illusion.py \
--scene illusion_0.75L --device 0 --steps 320 \ --scene illusion_1L --device 2 --steps 214 --mode pysr \
--sindy-results src/SR_analysis/sindy/illusion/sindy_results_deriv.json --pysr-front validate/results/pysr_illusion_joint_front.json \
--pysr-top validate/results/pysr_illusion_joint_top.json
# Karman phase-state + absolute action # Batch generalization validation
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \ bash src/SR_analysis/validate/batch_illusion_generalization.sh
--scene karman_re100 --device 1 --steps 200 --mode abs \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json
# Karman old v23 for comparison
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scene karman_re100 --device 1 --steps 200 --mode v23 \
--sindy-results src/SR_analysis/sindy/karman/sindy_joint_wrapped.json
``` ```
### 离线 rollout 评估
```bash
python3 src/SR_analysis/validate/eval_rollout.py \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json \
--scene karman_re100
```
### PySR 符号回归
```bash
conda run -n sr_env python src/SR_analysis/sindy/run_pysr.py \
--scene illusion_1L
```
---
## 关键文件索引
| 文件 | 用途 |
|------|------|
| `configs.py` | 统一场景元数据（10+场景） |
| `utils/feature_builder.py` | 特征工程：PHYSICS_FEAT_KEYS, PHASE_STATE_KEYS, ILLUSION_PHASE_KEYS, 导数/滞后特征 |
| `utils/sindy_fitter.py` | STLSQ拟合：get_feature_matrix_v2, get_feature_matrix_deriv, compute_action_deriv |
| `utils/cfd_interface.py` | LegacyCelerisLab封装（GPU, 推理, norm） |
| `utils/g_operator.py` | 等变性诊断 |
| `scripts/infer_*.py` | Karman/Illusion/Vortex 推理管线 |
| `sindy/run_all_v2.py` | 统一 SINDy 拟合入口（支持 --deriv, --phase, --output-mode, --karman-expand 等） |
| `sindy/run_pysr.py` | 受限 PySR 符号回归 |
| `sindy/wrap_joint.py` | 联合模型 → wrapped 格式 |
| `validate/run_closed_loop.py` | Karman 闭环验证器（v23, deriv, abs 模式） |
| `validate/run_closed_loop_illusion.py` | Illusion 闭环验证器 |
| `validate/eval_rollout.py` | **新**：离线多步 rollout 评估 |
| `compare/support_overlap.py` | 跨场景 support 比较 |
| `compare/shared_core.py` | 多场景 shared core 检测 |
| `docs/SR_analysis_results.md` | **新**：完整分析报告 |
| `docs/figures/SR_analysis/fig*.png` | **新**：结果图表（6张） |
## 环境
| 环境 | conda 名 | 用途 |
|------|---------|------|
| CFD + DRL + SINDy | `pycuda_3_10` | infer, run_all_v2, closed-loop |
| PySR 符号回归 | `sr_env` | run_pysr.py |
| GPU | device 0, device 1 | |

43
src/SR_analysis/validate/run_closed_loop.py
View File

@ -436,6 +436,13 @@ def run_validation(
feat_keys_front=coefs["feat_keys_front"], feat_keys_front=coefs["feat_keys_front"],
feat_keys_rear=coefs["feat_keys_rear"], feat_keys_rear=coefs["feat_keys_rear"],
output_mode="absolute") output_mode="absolute")
elif mode == "pysr":
from SR_analysis.validate.predict_pysr import pysr_predict_v23
omega_phys = pysr_predict_v23(
obs, a_prev_phys, a_prev2_phys, mu, u0, dt_c,
coefs["front_func"], coefs["top_func"],
feat_keys_front=coefs["feat_keys_front"],
feat_keys_rear=coefs["feat_keys_rear"])
elif mode == "unstructured": elif mode == "unstructured":
omega_phys = predict_unstructured( omega_phys = predict_unstructured(
obs, a_prev_phys, a_prev2_phys, mu, u0, obs, a_prev_phys, a_prev2_phys, mu, u0,
@ -487,21 +494,49 @@ def main():
ap.add_argument("--device", type=int, default=2, help="GPU device") ap.add_argument("--device", type=int, default=2, help="GPU device")
ap.add_argument("--steps", type=int, default=100) ap.add_argument("--steps", type=int, default=100)
ap.add_argument("--mode", type=str, default="v23", ap.add_argument("--mode", type=str, default="v23",
choices=["v23", "unstructured", "deriv", "abs"], choices=["v23", "unstructured", "deriv", "abs", "pysr"],
help="Control law mode: v23 (default), unstructured, deriv, or abs") help="Control law mode: v23 (default), unstructured, deriv, abs, or pysr")
ap.add_argument("--sindy-results", type=str, default=None, ap.add_argument("--sindy-results", type=str, default=None,
help="Path to sindy_results.json") help="Path to sindy_results.json (not used in pysr mode)")
ap.add_argument("--threshold", type=float, default=None, ap.add_argument("--threshold", type=float, default=None,
help="SINDy threshold for sparsity (default: best_R2)") help="SINDy threshold for sparsity (default: best_R2)")
ap.add_argument("--pysr-front", type=str, default=None,
help="PySR front JSON (required for --mode pysr)")
ap.add_argument("--pysr-top", type=str, default=None,
help="PySR top JSON (required for --mode pysr)")
ap.add_argument("--out", type=str, default=None, ap.add_argument("--out", type=str, default=None,
help="Output directory for result JSON") help="Output directory for result JSON")
args = ap.parse_args() args = ap.parse_args()
if args.mode == "pysr":
# Load PySR formulas directly
if args.pysr_front is None or args.pysr_top is None:
raise ValueError("--pysr-front and --pysr-top required for --mode pysr")
from SR_analysis.validate.predict_pysr import (
load_pysr_formulas, build_pysr_functions,
)
front_expr, top_expr, fn_f, fn_r = load_pysr_formulas(
args.pysr_front, args.pysr_top)
front_func, top_func = build_pysr_functions(
front_expr, top_expr, fn_f, fn_r)
# Package as pseudo-coefs dict for run_validation
feat_keys_front = [k for k in fn_f if k != "bias"]
feat_keys_rear = [k for k in fn_r if k != "bias"]
coefs = {
"mode": "pysr",
"front_func": front_func,
"top_func": top_func,
"feat_keys_front": feat_keys_front,
"feat_keys_rear": feat_keys_rear,
"feat_names_front": fn_f,
"feat_names_rear": fn_r,
}
else:
if args.sindy_results is None: if args.sindy_results is None:
args.sindy_results = os.path.join( args.sindy_results = os.path.join(
os.path.dirname(__file__), "..", "sindy", "karman", "sindy_results_v2.json") os.path.dirname(__file__), "..", "sindy", "karman", "sindy_results_v2.json")
coefs = load_sindy_coefs(args.sindy_results, args.scene, threshold=args.threshold) coefs = load_sindy_coefs(args.sindy_results, args.scene, threshold=args.threshold)
result = run_validation(args.scene, coefs, args.device, result = run_validation(args.scene, coefs, args.device,
n_steps=args.steps, mode=args.mode) n_steps=args.steps, mode=args.mode)

199
src/SR_analysis/validate/run_closed_loop_illusion.py
View File

@ -54,12 +54,15 @@ def run_validation_illusion(
n_steps: int = 0, n_steps: int = 0,
threshold: Optional[float] = None, threshold: Optional[float] = None,
out_dir: Optional[str] = None, out_dir: Optional[str] = None,
mode_override: Optional[str] = None,
) -> dict: ) -> dict:
"""Run closed-loop validation for an Illusion scene. """Run closed-loop validation for an Illusion scene.
Parameters Parameters
---------- ----------
n_steps : int, default=0 (auto: >= 2*NX/U0 / sample_interval) n_steps : int, default=0 (auto: >= 2*NX/U0 / sample_interval)
mode_override : str, optional
Override auto-detected mode: "v23", "absolute", or None for auto-detect.
""" """
cfg = get_scene(scene_name) cfg = get_scene(scene_name)
u0 = cfg["u0"] u0 = cfg["u0"]
@ -175,8 +178,10 @@ def run_validation_illusion(
a_prev = bias_norm.copy() a_prev = bias_norm.copy()
a_prev2 = a_prev.copy() a_prev2 = a_prev.copy()
# Determine mode from SINDy results metadata (default: v23) # Determine mode from SINDy results metadata (default: v23), unless overridden
sindy_mode = coefs.get("mode", "v23") sindy_mode = mode_override if mode_override else coefs.get("mode", "v23")
if mode_override:
print(f" mode overridden: {mode_override}")
for step in range(n_steps): for step in range(n_steps):
obs = fifo[-1] if fifo else np.zeros(12, dtype=np.float32) obs = fifo[-1] if fifo else np.zeros(12, dtype=np.float32)
@ -190,7 +195,10 @@ def run_validation_illusion(
if target_harmonics is not None: if target_harmonics is not None:
target_forces_step = gen_target_states_at(step, target_harmonics) target_forces_step = gen_target_states_at(step, target_harmonics)
if sindy_mode == "absolute": if sindy_mode == "pysr":
# PySR mode -- should not reach here (uses run_validation_pysr_illusion instead)
raise ValueError("PySR mode should use run_validation_pysr_illusion()")
elif sindy_mode == "absolute":
omega_phys = predict_v23_deriv( omega_phys = predict_v23_deriv(
obs, obs_prev, a_prev_phys, a_prev2_phys, mu, u0, sample_interval/2000.0, obs, obs_prev, a_prev_phys, a_prev2_phys, mu, u0, sample_interval/2000.0,
coefs["front_coef"], coefs["top_coef"], coefs["front_coef"], coefs["top_coef"],
@ -296,6 +304,162 @@ def run_validation_illusion(
} }
def run_validation_pysr_illusion(
scene_name: str,
device_id: int,
front_func,
top_func,
feat_names_front: list,
feat_names_rear: list,
n_steps: int = 0,
out_dir: Optional[str] = None,
) -> dict:
"""Run closed-loop validation using PySR formulas for an Illusion scene."""
from SR_analysis.validate.predict_pysr import pysr_predict_v23
cfg = get_scene(scene_name)
u0 = cfg["u0"]
mu = cfg["mu"]
l0 = 20.0
sample_interval = cfg["sample_interval"]
conv_len = cfg.get("conv_len", 36)
action_scale = cfg["action_scale"]
action_bias = cfg["action_bias"]
n_obj_total = cfg["n_objects_env"]
sensor_x = cfg["sensor_x"]
front_x = cfg["pinball_front_x"]
rear_x = cfg["pinball_rear_x"]
min_steps = int(1 * NX / u0 / sample_interval)
if n_steps == 0 or n_steps < min_steps:
n_steps = max(min_steps, 200)
print(f" auto-set steps={n_steps} (min_steps={min_steps})")
print(f"\n=== Validating {scene_name} (mode=pysr, device={device_id}, steps={n_steps}) ===")
if out_dir is None:
out_dir = os.path.join(os.path.dirname(__file__), "results")
os.makedirs(out_dir, exist_ok=True)
# Load target data
data_dir = os.path.join(
os.path.dirname(__file__), "..", "data", "illusion", scene_name,
)
target_npz = np.load(os.path.join(data_dir, "target.npz"))
target_states = target_npz.get("target_sensors", target_npz["target_states"][:, 2:8])
print(f" target loaded: {target_states.shape}")
target_harmonics = None
harm_path = os.path.join(data_dir, "target_harmonics.json")
if os.path.isfile(harm_path):
with open(harm_path) as f:
target_harmonics = json.load(f)
print(f" target harmonics loaded: {len(target_harmonics)} channels")
# Build environment
cuda_cfg, field_cfg = load_legacy_configs(LEGACY_CFG_DIR)
field_cfg = field_cfg._replace(viscosity=float(cfg["nu"]))
ff = FlowField(field_cfg, cuda_cfg, device_id=device_id)
ny = ff.FIELD_SHAPE[1]
for y_off in [2.0, 0.0, -2.0]:
sc = (sensor_x * l0, (ny - 1) / 2 + y_off * l0, 0.0)
ff.add_sensor(sc, l0 / 4.0)
ff.add_cylinder((front_x * l0, (ny - 1) / 2, 0.0), l0 / 2.0)
ff.add_cylinder((rear_x * l0, (ny - 1) / 2 + 0.75 * l0, 0.0), l0 / 2.0)
ff.add_cylinder((rear_x * l0, (ny - 1) / 2 - 0.75 * l0, 0.0), l0 / 2.0)
n_obj = ff.obs.size // 2
assert n_obj == 6, f"Expected 6 objects, got {n_obj}"
stabilize_steps = int(4 * NX / u0)
print(f" stabilising pinball ({stabilize_steps} steps)...")
ff.run(stabilize_steps, np.zeros(n_obj, dtype=DATA_TYPE))
ff.get_ddf()
ff.save_ddf()
# Norm + bias FIFO (same as run_validation_illusion)
print(f" norm collection ({FIFO_LEN} steps)...")
fifo = deque(maxlen=FIFO_LEN)
for _ in range(FIFO_LEN):
ff.run(sample_interval, np.zeros(n_obj, dtype=DATA_TYPE))
fifo.append(ff.obs.copy()[0:12])
ff.apply_ddf()
bias_arr = np.zeros(n_obj, dtype=DATA_TYPE)
bias_arr[3] = 0.0
bias_arr[4] = -1.0 * u0
bias_arr[5] = 1.0 * u0
fifo.clear()
for _ in range(FIFO_LEN):
ff.run(sample_interval, bias_arr)
fifo.append(ff.obs.copy()[0:12])
ff.get_ddf()
ff.save_ddf()
ff.apply_ddf()
# Closed-loop with PySR
feat_keys_front = [k for k in feat_names_front if k != "bias"]
feat_keys_rear = [k for k in feat_names_rear if k != "bias"]
sens_list, actions_list = [], []
bias_norm = np.array([0.0, 0.125, -0.125], dtype=np.float64)
a_prev = bias_norm.copy()
a_prev2 = a_prev.copy()
for step in range(n_steps):
obs = fifo[-1] if fifo else np.zeros(12, dtype=np.float32)
a_prev_phys = (a_prev * action_scale + np.array(action_bias, dtype=np.float64)) * u0
a_prev2_phys = (a_prev2 * action_scale + np.array(action_bias, dtype=np.float64)) * u0
target_forces_step = None
if target_harmonics is not None:
target_forces_step = gen_target_states_at(step, target_harmonics)
omega_phys = pysr_predict_v23(
obs, a_prev_phys, a_prev2_phys, mu, u0, sample_interval / 2000.0,
front_func, top_func,
feat_keys_front, feat_keys_rear,
target_forces=target_forces_step,
)
norm_a = (omega_phys / u0 - np.array(action_bias, dtype=np.float64)) / action_scale
norm_a = np.clip(norm_a, -1.0, 1.0).astype(np.float32)
action_arr = scale_action(norm_a, scale=action_scale, bias=action_bias,
u0=u0, n_total_bodies=n_obj_total)
ff.context.push()
ff.run(sample_interval, action_arr)
ff.context.pop()
obs_new = ff.obs.copy()[0:12]
fifo.append(obs_new)
sens_list.append(obs_new[0:6])
actions_list.append(omega_phys.copy())
a_prev2 = a_prev.copy()
a_prev = norm_a.copy().astype(np.float64)
sens_arr = np.array(sens_list, dtype=np.float32)
from SR_analysis.utils.cfd_interface import compute_similarity
sim_full = compute_similarity(target_states, sens_arr, conv_len)
sim_tail = compute_similarity(target_states[-conv_len:], sens_arr[-conv_len:], conv_len // 2)
action_range = float(np.max(np.abs(actions_list)))
print(f" similarity (full)={sim_full:.4f} (tail)={sim_tail:.4f} action_range={action_range:.4f}")
del ff
return {
"scene": scene_name,
"mode": "pysr",
"similarity_full": sim_full,
"similarity_tail": sim_tail,
"action_range": action_range,
"n_steps": n_steps,
}
def main(): def main():
ap = argparse.ArgumentParser(description="Illusion closed-loop SINDy validation") ap = argparse.ArgumentParser(description="Illusion closed-loop SINDy validation")
ap.add_argument("--scene", type=str, required=True) ap.add_argument("--scene", type=str, required=True)
@ -304,22 +468,47 @@ def main():
help="Steps (default: auto-set to cover 1*NX/U0)") help="Steps (default: auto-set to cover 1*NX/U0)")
ap.add_argument("--threshold", type=float, default=None) ap.add_argument("--threshold", type=float, default=None)
ap.add_argument("--sindy-results", type=str, default=None) ap.add_argument("--sindy-results", type=str, default=None)
ap.add_argument("--mode", type=str, default=None,
help='Override mode: "v23", "absolute", "pysr" (default: auto-detect)')
ap.add_argument("--pysr-front", type=str, default=None,
help="PySR front JSON (required for --mode pysr)")
ap.add_argument("--pysr-top", type=str, default=None,
help="PySR top JSON (required for --mode pysr)")
ap.add_argument("--out", type=str, default=None) ap.add_argument("--out", type=str, default=None)
args = ap.parse_args() args = ap.parse_args()
if args.mode == "pysr":
if args.pysr_front is None or args.pysr_top is None:
raise ValueError("--pysr-front and --pysr-top required for --mode pysr")
from SR_analysis.validate.predict_pysr import (
load_pysr_formulas, build_pysr_functions, pysr_predict_v23,
)
front_expr, top_expr, fn_f, fn_r = load_pysr_formulas(
args.pysr_front, args.pysr_top)
front_func, top_func = build_pysr_functions(
front_expr, top_expr, fn_f, fn_r)
result = run_validation_pysr_illusion(
args.scene, args.device, front_func, top_func, fn_f, fn_r,
n_steps=args.steps, out_dir=args.out,
)
else:
if args.sindy_results is None: if args.sindy_results is None:
args.sindy_results = os.path.join( args.sindy_results = os.path.join(
os.path.dirname(__file__), "..", "sindy", "illusion", "sindy_results_v2.json") os.path.dirname(__file__), "..", "sindy", "illusion", "sindy_results_v2.json")
result = run_validation_illusion( result = run_validation_illusion(
args.scene, args.sindy_results, args.device, args.scene, args.sindy_results, args.device,
n_steps=args.steps, threshold=args.threshold, out_dir=args.out, n_steps=args.steps, threshold=args.threshold, out_dir=args.out,
mode_override=args.mode,
) )
if args.mode == "pysr":
out_name = f"{args.scene}_pysr"
else:
th_str = f"_th{args.threshold}" if args.threshold is not None else "" th_str = f"_th{args.threshold}" if args.threshold is not None else ""
out_name = f"{args.scene}_v23{th_str}"
out_dir = args.out or os.path.join(os.path.dirname(__file__), "results") out_dir = args.out or os.path.join(os.path.dirname(__file__), "results")
os.makedirs(out_dir, exist_ok=True) os.makedirs(out_dir, exist_ok=True)
out_path = os.path.join(out_dir, f"{args.scene}_v23{th_str}.json") out_path = os.path.join(out_dir, f"{out_name}.json")
with open(out_path, "w") as f: with open(out_path, "w") as f:
json.dump(result, f, indent=2) json.dump(result, f, indent=2)
print(f"Saved: {out_path}") print(f"Saved: {out_path}")

15
src/drl_pinball/knowledge.md
View File

@ -578,24 +578,25 @@ All trained from scratch (no transfer). Obs reduction experiments for experiment
| `d1a3o12_250525_imit_075L_1U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 0.75L cylinder, U0=0.01 | 600 | | `d1a3o12_250525_imit_075L_1U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 0.75L cylinder, U0=0.01 | 600 |
| `d1a3o12_250525_imit_1L_1U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder, U0=0.01 | 600 | | `d1a3o12_250525_imit_1L_1U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder, U0=0.01 | 600 |
| `d1a3o12_250525_imit_1L_1U_trans.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder, U0=0.01 | 600 (transfer) | | `d1a3o12_250525_imit_1L_1U_trans.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder, U0=0.01 | 600 (transfer) |
| `d1a3o14_250525_imit_075L_2U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 0.75L cylinder, 2×U0 | 600 | | `d1a3o14_250525_imit_075L_2U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 0.75L cylinder | 600 |
| `d1a3o14_250525_imit_075L_2U_1.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 0.75L cylinder | ~600 | | `d1a3o14_250525_imit_075L_2U_1.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 0.75L cylinder | ~600 |
| `d1a3o14_250525_imit_075L_2U_400S.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 0.75L cylinder | 400 | | `d1a3o14_250525_imit_075L_2U_400S.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 0.75L cylinder | 400 |
| `d1a3o14_250525_imit_15L_2U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.5L cylinder, 2×U0 | 600 | | `d1a3o14_250525_imit_15L_2U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.5L cylinder | 800 |
| `d1a3o14_250525_imit_1L_2U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder, 2×U0 | 600 | | `d1a3o14_250525_imit_1L_2U.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | 600 |
| `d1a3o14_250525_imit_1L_2U_1.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | ~600 | | `d1a3o14_250525_imit_1L_2U_1.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | ~600 |
| `d1a3o14_250525_imit_1L_2U_400S_02Vis.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | 400 | | `d1a3o14_250525_imit_1L_2U_400S_02Vis.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | 400 |
| `d1a3o14_250525_imit_1L_2U_600S.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | 600 | | `d1a3o14_250525_imit_1L_2U_600S.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | 600 |
| `d1a3o14_250525_imit_1L_2U_800S_08Vis.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | 800 | | `d1a3o14_250525_imit_1L_2U_800S_08Vis.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | 800 |
| `d1a3o14_250525_imit_1L_2U_1000S_08Vis.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | 1000 | | `d1a3o14_250525_imit_1L_2U_1000S_08Vis.zip` | Illusion | 14 | 3 | 8 / [0,-2,2] | 1.0L cylinder | 1000 |
**Naming conventions**: **Naming conventions (CORRECTED 2026-06-12)**:
- `075L` = target cylinder diameter = 0.75×L0 - `075L` = target cylinder diameter = 0.75×L0
- `1L` = target cylinder diameter = 1.0×L0 - `1L` = target cylinder diameter = 1.0×L0
- `15L` = target cylinder diameter = 1.5×L0 - `15L` = target cylinder diameter = 1.5×L0
- `1U` = U0=0.01 (standard), `2U` = 2×U0=0.02 - `1U` = S_DIM=12 (6 sensor + 6 force). `2U` = S_DIM=14 (6 sensor + 6 force + 2 target).
- `400S` etc. = SAMPLE_INTERVAL **"U" does NOT refer to velocity u0.** u0 is always 0.01.
- `02Vis` etc. = ν (viscosity) multiplier (e.g. 0.08×ν) - `400S` etc. = SAMPLE_INTERVAL. No S suffix = 800 (default).
- `02Vis` etc. = ν multiplier applied to default nu=0.004 (e.g. `02Vis` = 0.004×0.02 = 0.00008)
- `trans` = transfer learning model - `trans` = transfer learning model
- `_1` suffix = variant - `_1` suffix = variant