# SR_analysis: Unified SINDy-SR Analysis Pipeline

## Overview

This directory consolidates the SINDy-and-symbolic-regression analysis pipeline
for the DynamisLab fluidic pinball project. It replaces the old
`src/analysis_crossre/` and `src/analysis_cloak/` directories with a unified
structure.

The pipeline fits **sparse interpretable control laws** (`obs -> act`) for all
cloak and illusion scenes, using dimensionless physical features,
G-equivariant structural constraints, and STLSQ threshold grids.

For background, see:
- `src/sindy_sr_notes.md` -- execution plan
- `src/sindy_sr_knoeledge.md` -- confirmed facts and known pitfalls

## Directory Structure

```
SR_analysis/
  configs.py                 # Unified scene metadata (all 10 scenes)
  configs/
    legacy/                  # Legacy CFD configs (config_cuda.json, config_flowfield.json)
  utils/
    __init__.py              # Selective exports (no pycuda dependency)
    feature_builder.py       # Dimensionless features + G-operator (from analysis_cloak)
    sindy_fitter.py          # STLSQ threshold grid, feature matrix builder
    cfd_interface.py         # LegacyCelerisLab wrapper (requires pycuda_3_10)
    g_operator.py            # Equivariance diagnostics
  data/
    karman/                  # Karman cloak: karman_re50, re100, re200, re400
    steady/                  # Steady cloak: steady_data.npz
    illusion/                # Illusion: illusion_0.75L, illusion_1L, illusion_1.5L
    vortex/                  # Vortex cloak: vortex_lamb, vortex_taylor
  scripts/
    infer_karman.py          # Inference: LegacyCFD + PPO -> controlled.npz
    infer_illusion.py        # Inference: for 0.75L, 1L, 1.5L diameters
    infer_vortex.py          # Inference: for Lamb dipole + Taylor monopole
  sindy/
    run_karman.py            # SINDy fitting for Karman scenes
    run_illusion.py          # SINDy fitting for Illusion scenes
    run_vortex.py            # SINDy fitting for Vortex scenes
    run_pareto.py            # Pareto-front analysis from SINDy results
    karman/                  # Output: sindy_results.json, pareto_*.json
    illusion/                # Output: sindy_results.json, pareto_*.json
    vortex/                  # Output: sindy_results.json, pareto_*.json
  validate/
    run_closed_loop.py       # Unified closed-loop validator (v23 + unstructured modes)
  compare/
    support_overlap.py       # Pairwise support set comparison
    shared_core.py           # Multi-scene shared-core detection
```

## Key Design Decisions

### 1. Scene Metadata Driven

All scene parameters (Re, action scaling, geometry, model paths) are defined
once in `configs.py`, not hard-coded in scripts. Adding a new scene means
adding one dict to `configs.py`.

### 2. Data / Features / Models Separation

- `data/` -- raw sensor/force/action arrays (.npz), one-time generation
- `sindy/` -- SINDy fitting results (JSON), reusable for comparison
- `scripts/` -- inference pipelines that produce `data/`

### 3. Unified Feature Builder

Every scene uses the same `utils/feature_builder.py`, which produces
21 dimensionless features from raw lattice-unit sensor/force data:

**Sensor features (nondim):**
- `u_m`, `u_a`, `u_c` -- streamwise: mean, antisymmetric, centre
- `v_a` -- antisymmetric cross-stream
- `sin_ua`, `cos_ua` -- phase encoding via u_a

**Force features (Cd/Cl):**
- `Cd_tot`, `Cd_rear` -- total and rear-cylinder drag
- `Cl_tot`, `Cl_diff` -- total and differential lift

**Memory features (nondim alpha):**
- `aF_lag1`, `aB_lag1`, `aT_lag1` -- lagged actions (t-1)
- `daF`, `daB`, `daT` -- action increments (t-1)-(t-2)

**Reynolds modulation:**
- `mu` (= 1/Re_D), `mu_u_a`, `mu_v_a`, `mu_Cd_tot`, `mu_Cl_diff`

### 4. G-Equivariant Structure (v23)

Default control law structure (confirmed as the best v23 model):

```
Front(t) = f_front(x(t))                     # no bias, odd under G
Top(t)   = f_rear(x(t))                      # with bias
Bottom(t) = -f_rear(G[x(t)])                 # shared-head: bottom = -top(Gx)
```

Where G is the mirror operator (y -> -y) with corrected sign rules:
- `[aF, aT, aB] -> [-aF, -aB, -aT]`
- Sensor swap: top <-> bottom, negate v
- Force swap: front unchanged, bottom <-> top, negate Cl

### 5. STLSQ Threshold Grid

Default thresholds: `[0, 0.001, 0.002, 0.005, 0.01, 0.015, 0.02, 0.03, 0.05, 0.1]`
Per-channel: front (no bias), top (shared-head), bottom (independent, for comparison)

## Scene Inventory

| Scene Name | Description | Re_code | Sample Interval | Action | U0 |
|---|---|---|---|---|---|
| karman_re50 | Karman cloak at low Re | 50 | 800 | 8x + [0,-4,4] | 0.01 |
| karman_re100 | Karman cloak (default) | 100 | 800 | 8x + [0,-4,4] | 0.01 |
| karman_re200 | Karman cloak at high Re | 200 | 800 | 8x + [0,-4,4] | 0.01 |
| karman_re400 | Karman cloak at highest Re | 400 | 800 | 8x + [0,-4,4] | 0.01 |
| steady | Open-loop constant rotation | 100 | 800 | 8x + [0,-5.1,5.1] | 0.01 |
| illusion_0.75L | Imitate 0.75D cylinder | 100 | 600 | 8x + [0,-2,2] | 0.01 |
| illusion_1L | Imitate 1.0D cylinder | 100 | 600 | 8x + [0,-2,2] | 0.01 |
| illusion_1.5L | Imitate 1.5D cylinder | 100 | 600 | 8x + [0,-2,2] | 0.02 |
| vortex_lamb | Cloak Lamb dipole | 100 | 800 | 4x + [0,-4,4] | 0.01 |
| vortex_taylor | Cloak Taylor monopole | 100 | 800 | 4x + [0,-4,4] | 0.01 |

Note: "Re_code" uses reference length 2*D (code convention).
Physical Re_D = Re_code / 2. E.g. Re_code=100 -> Re_D=50.

## Re-generation Commands

All commands run from repo root (`/home/frank14f/DynamisLab`).

### Data Generation (requires GPU, pycuda_3_10 env)

```bash
# Karman cloak -- all 4 training Re
conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_karman.py --re all --device 0

# Karman cloak -- single Re
conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_karman.py --re 100 --device 0 --steps 200

# Illusion -- all 3 diameters
conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_illusion.py --diameter all --device 0

# Vortex -- both types
conda run -n pycuda_3_10 python src/SR_analysis/scripts/infer_vortex.py --type all --device 0
```

### SINDy Fitting (no GPU needed, pycuda_3_10 env for pysindy)

```bash
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_karman.py
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_illusion.py
conda run -n pycuda_3_10 python src/SR_analysis/sindy/run_vortex.py
```

### Pareto Analysis (no GPU, no conda needed)

```bash
python3 src/SR_analysis/sindy/run_pareto.py --scene karman_re100
python3 src/SR_analysis/sindy/run_pareto.py --scene illusion_1L
```

### Closed-loop Validation (requires GPU)

```bash
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
    --scene karman_re70 --device 2 \
    --sindy-results src/SR_analysis/sindy/karman/sindy_results.json

# With custom mode
conda run -n pycuda_3_10 python src/SR_analysis/validate/run_closed_loop.py \
    --scene karman_re70 --device 2 --mode unstructured
```

### Cross-scene Comparison (no GPU)

```bash
# Pairwise support overlap
python3 src/SR_analysis/compare/support_overlap.py \
    --sindy-results src/SR_analysis/sindy/karman/sindy_results.json \
    --scenes karman_re100 illusion_1L

# Multi-scene shared core
python3 src/SR_analysis/compare/shared_core.py \
    --sindy-results src/SR_analysis/sindy/karman/sindy_results.json \
    --scenes karman_re50 karman_re100 karman_re200 karman_re400
```

## Key Results Summary

### Data Quality (similarity scores)

| Scene | PPO Similarity |
|---|---|
| karman_re50 | 0.962 |
| karman_re100 | 0.954 |
| karman_re200 | 0.884 |
| karman_re400 | 0.795 (inferred, not verified) |
| vortex_lamb | 0.942 |
| vortex_taylor | 0.916 |
| illusion_1L | ~0.55 (metric not directly comparable) |

### SINDy Fit Quality (R2 scores for one-step prediction)

| Scene | Front | Top (shared) | Bottom |
|---|---|---|---|
| karman_re50 | 0.998 | 0.989 | 0.996 |
| karman_re100 | 0.995 | 0.993 | 0.997 |
| karman_re200 | 0.957 | 0.914 | 0.918 |
| karman_re400 | 0.991 | 0.979 | 0.969 |
| illusion_0.75L | 0.991 | 0.989 | 0.990 |
| illusion_1L | 0.979 | 0.984 | 0.984 |
| illusion_1.5L | 0.959 | 0.928 | 0.932 |
| vortex_lamb | 0.904 | 0.980 | 0.933 |
| vortex_taylor | 0.960 | 0.810 | 0.643 |

### Shared Core Features

**Karman cross-Re (active in all re50/100/200):**
- Front core: `mu`, `mu_Cd_tot`, `mu_Cl_diff`, `mu_v_a` (mu-modulated terms dominate)
- Top core: `Cl_tot`, `bias`, `mu_Cd_tot`, `mu_Cl_diff`, `mu_u_a`, `mu_v_a`
- Scene-specific: lower-Re scenes have additional `Cd_tot`, `Cl_diff`, `aT_lag1` etc.

**Illusion cross-diameter (active in all 0.75L/1L/1.5L):**
- Front core: `mu`, `mu_Cd_tot`, `mu_Cl_diff` (same structure as Karman front!)
- Top core: `Cd_rear`, `Cl_tot`, `bias`, `mu_Cd_tot`, `mu_Cl_diff`
- This suggests a **shared mu-modulated feedback structure** exists across both scenes

## Known Issues and Caveats

1. **Vortex Taylor rear channels** have low R2 (0.64-0.81). The weak monopole
   produces near-zero rear action, making SINDy fitting noisy. Use Lamb as the
   primary vortex reference.

2. **Closed-loop validator** (`validate/run_closed_loop.py`) has been ported but
   NOT yet tested end-to-end. The original `validate_v23.py` verified Karman
   but the new unified version has not been run.

3. **Illusion similarity scores** use the Karman CONV_LEN=30 metric, giving
   lower raw numbers. The controlled.npz data itself is valid for SINDy.

4. **Steady cloak** is open-loop constant rotation, not PPO-derived. It serves
   as a physical consistency check, not a primary comparison scene.

5. **SINDy one-step R2 is not sufficient** -- a high R2 does not guarantee good
   closed-loop performance. Always validate via `validate/run_closed_loop.py`.

6. **Scene key naming**: keys like `illusion_1L`, `illusion_1.5L` use the short
   float format from Python (1.0 -> "1L", 1.5 -> "1.5L", 0.75 -> "0.75L").

## Next Steps (Future Work)

1. **PySR symbolic regression** -- Run PySR on the SINDy-identified active
   features (in `sr_env` conda env) to find closed-form formulas. Essential
   reading: `src/pysr.md`.

2. **Closed-loop validation of all new scenes** -- Run
   `validate/run_closed_loop.py` for illusion and vortex scenes using their
   SINDy coefficients.

3. **Cross-scene shared backbone test** -- Fit a single SINDy model on merged
   Karman + Illusion data, test if it performs on both.

4. **Time-scale explicit formulation** -- Make the sample interval an explicit
   feature to compare control laws across different frequencies.

5. **Steady as consistency check** -- Validate that Karman-derived control laws
   can reproduce the steady cloak result as a sanity check.

## File Reference

| File | Lines | Purpose |
|---|---|---|
| configs.py | ~205 | Unified scene metadata |
| utils/feature_builder.py | ~212 | Dimensionless features + G-op |
| utils/sindy_fitter.py | ~175 | STLSQ fitting, feature matrix builder |
| utils/cfd_interface.py | ~370 | LegacyCelerisLab wrapper |
| utils/g_operator.py | ~170 | Equivariance diagnostics |
| utils/__init__.py | ~10 | Selective exports |
| scripts/infer_karman.py | ~250 | Karman inference pipeline |
| scripts/infer_illusion.py | ~270 | Illusion inference pipeline |
| scripts/infer_vortex.py | ~280 | Vortex inference pipeline |
| sindy/run_karman.py | ~160 | Karman SINDy fitting |
| sindy/run_illusion.py | ~110 | Illusion SINDy fitting |
| sindy/run_vortex.py | ~110 | Vortex SINDy fitting |
| sindy/run_pareto.py | ~140 | Pareto analysis |
| validate/run_closed_loop.py | ~270 | Closed-loop validator |
| compare/support_overlap.py | ~150 | Pairwise support comparison |
| compare/shared_core.py | ~140 | Multi-scene shared core detection |
