Frank14f/DynamisLab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: project-wide cleanup — consolidate docs, remove obsolete code, update .gitignore

Browse Source 
- Remove obsolete docs (OID_handover, SR_analysis_results, ccd_* handover)
- Remove CCD legacy output_redux and old scripts
- Remove SR old sindy scripts and compare modules
- Update .gitignore to cover all analysis-generated outputs
- Retain all active code in OID/SR/CCD analysis directories

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Frank14f
2026-06-28 16:52:48 +08:00
co-authored by Cursor
parent c918ac0de4
commit 56e3c78a83
73 changed files with 1365 additions and 11072 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/SR_analysis/README.md
+145 -146
View File
@@ -1,20 +1,78 @@
# SR_analysis: Unified SINDy-SR Analysis Pipeline
# SR_analysis: Symbolic Regression Analysis Pipeline
## Overview
Extracts interpretable control laws (`obs -> act`) from DRL-trained policies for the
fluidic pinball. Uses **PySR symbolic regression** on dimensionless physical features with
G-equivariant structural constraints (v23: front no-bias, rear shared-head).
This directory consolidates the SINDy-and-symbolic-regression analysis pipeline
for the DynamisLab fluidic pinball project. It replaces the old
`src/analysis_crossre/` and `src/analysis_cloak/` directories with a unified
structure.
## Current Results (2026-06-25)
The pipeline fits **sparse interpretable control laws** (`obs -> act`) for all
cloak and illusion scenes, using dimensionless physical features,
G-equivariant structural constraints, and STLSQ threshold grids.
### Karman Cloak — Cross-Re Unified Formula
For background, see:
- `sindy_sr_notes.md` -- execution plan and task tracking
- `sindy_sr_knowledge.md` -- confirmed facts and known pitfalls
- `../../docs/SR_analysis_results.md` -- comprehensive results report
| Scene | Front Formula | Top Formula | CFD Closed-Loop |
|-------|--------------|-------------|:---------------:|
| Joint (Re50-400) | `daF_dt - 14.952*mu*Cl_tot` | `alpha_T = 3.414` (const) | **0.847 avg** |
| Re50 independent | PySR per-Re best | — | **0.895** |
| Re100 independent | PySR per-Re best | — | **0.888** |
| Re200 independent | PySR per-Re best | — | **0.916** |
| Re400 (SI=400 opt) | Joint formula | Joint formula | **0.819** |
### Illusion
| Scene | Front Formula | CFD Closed-Loop | % of PPO |
|-------|--------------|:---------------:|:--------:|
| 0.75L | `-0.169*(Cl_tot + dCl_tot_dt) - 1.240` | **0.979** | 100.7% |
| 1L | `(du_a_dt + u_a + 26.5)*0.0123` | **0.957** | 98.4% |
| **Joint (0.75L+1L)** | `target_Cd - 5.428 + 0.0098*(du_a_dt + u_a)` | **0.978 / 0.970** | — |
| 1.5L | High-freq periodic modulation (not SR-amenable) | — | — |
**Key finding**: 0.75L and 1L formulas have fundamentally different skeletons (Cl_tot vs u_a
dominant). Joint formula still achieves excellent CFD results on both although the underlying
mechanisms differ.
### Illusion Generalization (Joint Formula, No PPO)
| Diameter | Similarity | Notes |
|:--------:|:----------:|-------|
| 0.5L | 0.854 | Signal weak, noise-dominated |
| 0.6L | **0.939** | Generalizes well |
| 0.8L | **0.908** | Generalizes well |
| 1.2L | 0.849 | Begins to degrade |
| 1.5L | N/A | High-frequency regime, different mechanism |
| 2.0L | 0.676 | Degraded, near 1.5L regime |
Valid range: 0.6L-1.0L (similarity > 0.90).
### Vortex Cloak (Generalization)
Karman joint formula tested on vortex scenes (no retraining):
| Scene | Karman Joint Formula | PPO Baseline |
|-------|:-------------------:|:------------:|
| vortex_lamb | **0.949** | 0.942 |
| vortex_taylor | **0.905** | 0.916 |
---
## Pipeline Overview
```
controlled.npz (PPO rollout)
|
v
compute_features() --> dimensionless physics features (ILLUSION_PHASE_KEYS, etc.)
|
v
PySR symbolic regression --> sparse interpretable formulas
|
v
CFD closed-loop validation --> final similarity score
```
### Key Design Decisions
1. **Feature levels**: Static (8-dim) -> Phase-state (6-dim) -> Illusion-phase (10-dim)
2. **Output target**: Non-dimensional alpha, not physical omega
3. **v23 structure**: Front no-bias, rear shared-head (Bottom = -Top(Gx))
4. **Final judge**: CFD closed-loop similarity, not one-step R2
---
@@ -22,164 +80,105 @@ For background, see:
```
SR_analysis/
configs.py # Unified scene metadata (all 10+ scenes)
configs/
legacy/ # Legacy CFD configs
configs.py # Scene metadata (Karman, Illusion, Vortex)
configs/legacy/ # Legacy CFD configs (config_cuda.json, config_flowfield.json)
utils/
__init__.py # Selective exports (no pycuda dependency)
feature_builder.py # Dimensionless features + G-operator + phase-state features
sindy_fitter.py # STLSQ + feature matrices + derivative/absolute modes
__init__.py # Exports (no pycuda dependency)
feature_builder.py # Dimensionless features, G-operator, phase-state features
sindy_fitter.py # STLSQ fitting + feature matrices
cfd_interface.py # LegacyCelerisLab wrapper (requires pycuda_3_10)
g_operator.py # Equivariance diagnostics
data/
karman/ # Karman cloak: karman_re50/100/200/400
steady/ # Steady cloak
illusion/ # Illusion: illusion_0.75L/1L/1.5L
vortex/ # Vortex cloak
data/ # Inference output data (controlled.npz, target.npz)
karman/ karman_re50..400/
illusion/ illusion_0.75L,1L,1.5L/
vortex/ vortex_lamb,taylor/
scripts/
infer_karman.py # Inference: LegacyCFD + PPO -> controlled.npz
infer_illusion.py # Inference for illusion scenes
infer_vortex.py # Inference for vortex scenes
infer_karman.py # PPO inference -> controlled.npz
infer_illusion.py # PPO inference -> controlled.npz
infer_vortex.py # PPO inference -> controlled.npz
gen_illusion_target.py # Target data generation for generalization scenes
visualize_ppo_illusion.py# PPO visualization with vorticity
sindy/
run_all_v2.py # Unified SINDy fitting (supports --deriv, --phase, --output-mode etc.)
run_pysr.py # Restricted PySR symbolic regression
wrap_joint.py # Joint model -> wrapped format for validator
compare_v2.py # Cross-scene comparison report
karman/illusion/vortex/ # SINDy output JSONs
run_pysr.py # PySR symbolic regression (niter=40)
run_pysr_deep.py # Karman deep PySR (niter=120, Re independent + joint)
run_pysr_deep_illusion.py# Illusion deep+joint PySR (niter=120)
validate/
run_closed_loop.py # Karman closed-loop validator (v23/deriv/abs modes)
run_closed_loop_illusion.py # Illusion closed-loop validator
eval_rollout.py # Offline multi-step rollout evaluation
results/ # Validation result JSONs
compare/
support_overlap.py # Support set comparison
shared_core.py # Shared core detection
run_closed_loop.py # Karman closed-loop validator
run_closed_loop_illusion.py # Illusion closed-loop validator
run_closed_loop_vortex.py # Vortex closed-loop validator
run_closed_loop_re400_si.py # Karman re400 short-SI validator
predict_pysr.py # PySR formula sympy.lambdify wrapper
eval_rollout.py # Offline multi-step rollout evaluation
launch_pysr_validation.py # Batch CFD validation launcher
batch_illusion_generalization.sh# Batch generalization CFD validation
results/ # 136 JSON files — canonical + intermediate
results/README.md # Result file reference table
results/archive/ # Archived intermediate search attempts
```
---
## Key Design Decisions
## Usage
### 1. Scene Metadata Driven
All scene parameters defined once in `configs.py`.
### 2. Feature Levels
| Level | Features | Dim | Description |
|-------|----------|:---:|-------------|
| Static | u_m, u_a, u_c, v_a, Cd_tot, Cd_rear, Cl_tot, Cl_diff | 8 | Current-step only |
| **Phase-state** | u_a, du_a/dt, Cl_tot, dCl_tot/dt, Cd_tot, Cd_rear | **6** | Oscillation phase + rate |
| Illusion-phase | Phase-state + Cd_err, Cl_err, dCd_err/dt, dCl_err/dt | **10** | Phase + error-state |
| Karman-expanded | Phase-state + u_m, u_c, v_a, Cl_diff | **10** | Phase + supplementary |
| Full-lag | Static + lag-1 | 16 | Full temporal context |
### 3. Output Modes
- **deriv**: predict `d(alpha)/dt`, then `alpha(t) = alpha(t-1) + dt_c * dalpha/dt`
- **absolute**: predict `alpha(t)` directly (no integration drift)
### 4. G-Equivariant Structure (v23)
```
Front(t) = f_front(x(t)) # no bias, odd under G
Top(t) = f_rear(x(t)) # with bias
Bottom(t) = -f_rear(G[x(t)]) # shared-head
```
---
## Current Best Results (2026-06-15)
### Illusion — New Route: Phase-state + Error-state + Absolute Action
| Scene | Closed-loop | % of PPO | Action history? | Features |
|-------|:----------:|:--------:|:---------------:|----------|
| 0.75L | **0.974** | 100.2% | **No** | ILLUSION_PHASE (10dim) |
| 1L | **0.958** | 98.5% | **No** | ILLUSION_PHASE (10dim) |
| 1.5L | N/A | — | **No** | Bang-bang regime |
### Karman re100 — Ablation
| Config | Feat | Output | R2 | Closed-loop | Note |
|--------|:----:|:-----:|:--:|:----------:|------|
| old v23 (a_lag) | 14+3 | alpha | 0.996 | **0.901** | Baseline |
| **Phase->abs** | **6** | **alpha** | **0.965** | **0.699** | Best new route |
| Phase->deriv | 6 | dalpha/dt | 0.837 | 0.656 | |
| Phase+mu->abs | 9 | alpha | 0.979 | 0.700 | mu helps cross-Re |
| Expanded->abs | 10 | alpha | 0.980 | 0.580 | Overfitting |
---
## Commands
All from repo root (`/home/frank14f/DynamisLab`).
### SINDy Fitting
### PySR Symbolic Regression (conda: sr_env)
```bash
# Illusion phase-state + absolute (recommended for 0.75L/1L)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes illusion_0.75L,illusion_1L --deriv --phase --output-mode absolute
# Illusion
conda run -n sr_env python src/SR_analysis/sindy/run_pysr_deep_illusion.py --individual
# Karman phase-state + absolute
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes karman_re100 --deriv --phase --output-mode absolute
# Karman expanded (10 dim)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes karman_re100 --deriv --karman-expand --output-mode absolute
# Karman with mu modulation
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes karman_re100 --deriv --karman-mu --output-mode absolute
# Old-style (v2, with action history)
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_all_v2.py \
--scenes karman_re50,karman_re100 --joint
# Karman deep (cross-Re independent + joint)
conda run -n sr_env python src/SR_analysis/sindy/run_pysr_deep.py --both
```
### Closed-loop Validation
### CFD Closed-Loop Validation (conda: pycuda_3_10, GPU 1 or 2)
```bash
# Karman with absolute action
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scene karman_re100 --device 0 --steps 200 --mode abs \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json
# Karman old v23
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scene karman_re100 --device 0 --steps 200 --mode v23 \
--sindy-results src/SR_analysis/sindy/karman/sindy_joint_wrapped.json
# Illusion with absolute action
# Illusion PySR formula
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop_illusion.py \
--scene illusion_1L --device 0 --steps 320 \
--sindy-results src/SR_analysis/sindy/illusion/sindy_results_deriv.json
--scene illusion_1L --device 2 --steps 320 --mode pysr \
--pysr-front validate/results/pysr_illusion_1L_front.json \
--pysr-top validate/results/pysr_illusion_1L_top.json
# Karman joint formula
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
--scene karman_re100 --device 2 --steps 200 --mode pysr \
--pysr-front validate/results/karman_joint_deep_front.json \
--pysr-top validate/results/karman_joint_deep_top.json
# Vortex (generalization test)
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop_vortex.py \
--scene vortex_lamb --device 2 --steps 150 --mode pysr \
--pysr-front validate/results/karman_joint_deep_front.json \
--pysr-top validate/results/karman_joint_deep_top.json
```
### PySR Symbolic Regression
### PPO Inference (generate controlled.npz)
```bash
conda run -n sr_env python src/SR_analysis/sindy/run_pysr.py --scene illusion_1L
```
### Offline Rollout Evaluation
```bash
python3 src/SR_analysis/validate/eval_rollout.py \
--sindy-results src/SR_analysis/sindy/karman/sindy_results_deriv.json \
--scene karman_re100
conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_karman.py --re 100 --device 2
conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_illusion.py --diameter 1.0 --device 2
conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_vortex.py --type lamb --device 2
```
---
## Important Reminders
## Critical Reminders
- `controlled.npz` actions are **normalized [-1,1]** — must convert via `(norm * scale + bias) * u0`
- **FIFO bias ≠ DRL action bias** for Illusion: FIFO=[0, -0.01, 0.01], decode=[0, -0.02, 0.02]
- "2U" in model name = S_DIM=14 (not 2x velocity), u0 always 0.01
- SAMPLE_INTERVAL: 0.75L=400, 1L=600, 1.5L/Karman=800
- Closed-loop steps auto-set: S=400→320, S=600→214, S=800→160
- One-step R² high ≠ closed-loop good — always validate
- For phase-state features, always pass `sensors_raw`/`forces_raw` to enable derivative computation
- **actions.npz are normalized [-1,1]**, not physical omega. Convert: `(action * scale + bias) * u0`
- **PySR needs `sensors_raw`/`forces_raw`** passed to `compute_features()` or derivative features are zero
- **Output target must be alpha** (non-dim): `Y = actions_phys / u0`
- **One-step R2 high != closed-loop good** -- always validate in CFD
- **Controls must propagate**: steps >= NX/u0/SI (S=400->320, S=600->214, S=800->160)
- **FIFO bias != DRL action bias** for Illusion: FIFO=[0,-U0,U0], decode=[0,-2,2]*U0
- **Joint formula must be manually reviewed** for spurious terms (e.g. `daB_dt` is constant=0 at deployment)
---
## Key Documentation
| File | Content |
|------|---------|
| `src/SR_analysis/sindy_sr_knowledge.md` | Background knowledge, bug history, known pitfalls (for coder reference) |
| `src/SR_analysis/sindy_sr_notes.md` | Task list, phase breakdown, current status |
| `docs/SR_analysis_report.md` | **Single consolidated report** — all formulas, results, methodology, structural analysis |
| `docs/illusion_joint_formula_analysis.md` | Illusion joint formula deep dive — physical interpretation, generalization curve |