DynamisLab/src/drl_pinball/legacy_test

Legacy Test (Track A)

Systematic validation of pre-trained PPO models using LegacyCelerisLab (the original CFD solver the models were trained with).

Quick Start

# Run all legacy tests sequentially (GPU 1, 60s delay between tests)
bash src/drl_pinball/legacy_test/run_all_legacy_tests.sh 1

# Single scene
conda run -n pycuda_3_10 python src/drl_pinball/legacy_test/test_karman_cloak_re100.py --device 1

Directory

legacy_test/
├── README.md                         # This file
├── core/
│   ├── comparator.py                 # Compare signals against SR_analysis reference
│   ├── dtw_metrics.py                # DTW/harmonics (re-exports from reproduce/core/)
│   ├── io_helpers.py                 # Save/load .npz, norm.json
│   ├── legacy_env_builder.py         # FlowField builders for all 5 scene types
│   └── model_loader.py               # PPO model loading (wraps ModelInventory)
├── test_karman_cloak_re100.py        # Flagship: Karman Cloak Re100
├── test_karman_cloak_crossre.py      # Cross-Re: re50, re200, re400
├── test_steady_cloak.py              # Steady cloak (open-loop, no DRL)
├── test_illusion_1L.py               # Illusion 1.0L (S_DIM=14)
├── test_illusion_remaining.py        # Illusion 0.75L, 1.5L
├── test_vortex_lamb.py               # Vortex Lamb dipole
├── test_vortex_taylor.py             # Vortex Taylor monopole
├── test_erase.py                     # Erase (experimental, known incomplete)
├── run_all_legacy_tests.sh           # Sequential launcher
└── output/                           # Per-scene verification outputs

Scene Coverage

Scene	S_DIM	Scale/Bias	SI	MaxSteps	Legacy Ref
Karman re100	12	8/(0,-4,4)	800	500	legacy_karman_env.py
Karman re50	12	8/(0,-4,4)	800	500	same, ν=0.008
Karman re200	12	8/(0,-4,4)	800	500	same, ν=0.002
Karman re400	12	8/(0,-4,4)	800	500	same, ν=0.001
Steady Cloak	—	open-loop	800	200	OID collect_steady_cloak.py
Illusion 0.75L	14	8/(0,-2,2)	400	500	legacy_env_imit.py
Illusion 1L	14	8/(0,-2,2)	600	500	same
Illusion 1.5L	14	8/(0,-2,2)	800	500	same
Vortex Lamb	12	4/(0,-4,4)	800	150	legacy_env_vortex.py
Vortex Taylor	12	4/(0,-4,4)	800	150	same
Erase	12	8/(0,-8,8)	600	500	legacy_env_erase.py

Design Notes

• Object ordering matches legacy EXACTLY (documented in knowledge.md Section 9):
  • Karman/Erase: dist_cyl(0) [or sensor0(0) for erase], sensors(1-3), front(4), top(5), bottom(6)
  • Steady/Illusion/Vortex: sensors(0-2), front(3), top(4), bottom(5)
• DDF checkpoint timing uses pre-bias save + test-side bias FIFO (matching legacy save_ddf() pattern)
• Action uses legacy FlowField.run() built-in EMA smoothing (weight 0.1)
• Comparison uses DTW similarity > 0.95 as primary pass criterion (phase-invariant)
• Steady cloak is open-loop — verifies lift RMS suppression, no DTW comparison
• Erase is known incomplete — no DTW threshold enforced