31 lines
3.0 KiB
Markdown
31 lines
3.0 KiB
Markdown
# Transolver-Greif Adapter (`TRANSOLVER-ADAPTER-1122`)
|
||
|
||
Code-only Transolver (Wu et al., ICML 2024) adapter aligned with the frozen grouped Greif v5 comparison protocol (`18e0e729`). No real data read, no GPU run, no push to `main`.
|
||
|
||
## Files
|
||
- `tools/track_b_transolver_greif_adapter.py`: `GreifTransolver`, `TransolverBlock`, `PhysicsAttention`, `make_greif_transolver`, and pure-array input helpers.
|
||
- `tools/test_track_b_transolver_greif.py`: CPU-only synthetic tet-mesh unit tests.
|
||
|
||
## Protocol Alignment
|
||
- **Inputs:** Exact 19 `COLUMNS` from `track_b_gino_greif_adapter.py` (no extra `xyz` channels), standardized in FP64 on each fold's learning split only (`phase2_h2_data.standardization`), cast to FP32, plus strictly positive FP32 nodal `mass` and optional `component_ids`.
|
||
- **Target & Decode:** Exact `phase2_residual_math` FP64 volume-clock prior `c * Vf`, signed unclamped FP32 `residual_target`, and `decode_seconds`. Readout head final layer is zero-initialized so epoch 0 starts at `c * Vf`.
|
||
- **Loss & Schedule:** Reuses `phase2_h2_model.{weighted_loss, optimizer, lr_epoch}` (node-equal FP32 MSE, AdamW `wd=1e-4` on >=2D non-bias, 10-epoch warmup + 190-epoch cosine, grad clip 1.0, TF32 off).
|
||
- **Folds:** Unchanged grouped folds 0/1 (`((0,0),(0,1),(1,0),(1,1))`, family isolation, `no_validation_selection=True`).
|
||
|
||
## Parameter Counts
|
||
| Model | Config | Parameters |
|
||
| --- | --- | ---: |
|
||
| `H2Geo` (reference) | `C=64, 4 GeoBlocks` | 195,077 |
|
||
| `GreifTransolver` (`default`) | `C=64, L=6, H=4, G=64, mlp_ratio=2` | 197,913 |
|
||
| `GreifTransolver` (`wide` flag) | `C=96, L=4, H=4, G=64, mlp_ratio=2` | 293,937 |
|
||
|
||
## Design Assumptions
|
||
1. **Mass-weighted slice pooling (`mass_weighted=True`, default):** Nodal mass is normalized per connected component (`m / sum(m)`) before slice pooling so attention tokens are invariant to global mass units and reflect tet volume measure. `mass_weighted=False` reproduces unweighted Transolver.
|
||
2. **Per-component isolation & chunking:** Slices are pooled per physical connected component (matching `H2Geo.global_message`) in `NODE_CHUNK=8192` chunks with optional block gradient checkpointing to bound VRAM on 30K–100K+ node tet meshes.
|
||
3. **Slice denominator & temperature stability:** Per-head temperature is clamped to `[0.1, 5.0]` and normalized slice denominators add `1e-6` (`1e-5` unweighted) to prevent FP32 division instability.
|
||
|
||
## Open Items for MeltSol / MeltAssist
|
||
1. **Execution binding & worker:** This branch provides only the model adapter and CPU synthetic tests; MeltSol should wire `GreifTransolver` into a guarded fold-0/1 worker/entry (same `first`-epoch cost gate and <= 40,000 GPU-s GPU3 cap as planned) for MeltAssist review.
|
||
2. **Checkpoint / VRAM profiling on real meshes:** Verify `NODE_CHUNK=8192` and `checkpoint_blocks=True` peak VRAM on the largest Greif learning part during the first measured epoch.
|
||
3. **Optional ablation:** Confirm whether only `default` (197.9K, `mass_weighted=True`) runs in the primary <= 40,000 GPU-s slot, leaving `wide` and `mass_weighted=False` out of scope.
|