From 17453b4822eb08e1eb438441789896ef893d53db Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 01:37:16 +0000 Subject: [PATCH] Narrow B15 lesson per MoldAstra review Co-Authored-By: Claude Opus 5.5 --- docs/claude/phase2-retrospective-2026-11-17.md | 2 +- docs/claude/research-tree.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/claude/phase2-retrospective-2026-11-17.md b/docs/claude/phase2-retrospective-2026-11-17.md index 3cc3e08a4..642332f5f 100644 --- a/docs/claude/phase2-retrospective-2026-11-17.md +++ b/docs/claude/phase2-retrospective-2026-11-17.md @@ -16,7 +16,7 @@ Single writer: Claude. This document organises the evidence; the scores themselv 1. **Tabular feature additions to A are close to exhausted.** Eleven feature/target/loss variants gave one confirmed gain of about 1.6 % relative. The B13 gain comes from regional averaging itself; *which* rule draws the regions (generic, type-organised, shuffled) makes no detectable difference (E2). 2. **Last fill is the persistent failure.** Every branch, including the tail-weighted B17 and H2-S1, fails to confirm a last-fill improvement. The B14 / B16 near-misses did not survive a dedicated recipe. -3. **Correct after solving, not before.** Learning the speed and then solving the eikonal (B15) was clearly worse; the potential has to be corrected after the solve, as A does. +3. **One learned-speed recipe failed.** Learning the speed and then solving the eikonal (B15) was clearly worse than A, which corrects after the solve. This does *not* show that pre-solve learning is generally inferior: the inverse problem, label identifiability, inputs and objective all changed at once (MoldAstra, 2026-11-18). 4. **Representation may be the lever.** H2's fold-0 field gain is larger than any feature change; whether it holds is S2's question. Not yet comparable with E1b (different statistical domain). 5. **Data quantity is not identified.** A's curve 7.70 → 7.31 → 7.08 % (25/50/100 % pools) shows modest gains with more data; for neural models the data-sufficiency test (MoldAstra e24fff7c) decides. diff --git a/docs/claude/research-tree.md b/docs/claude/research-tree.md index a74e44229..c7456a368 100644 --- a/docs/claude/research-tree.md +++ b/docs/claude/research-tree.md @@ -42,7 +42,7 @@ Log (newest first): - 2026-11-17: **B17 opened** (MoldAstra spec b5bd8a4e, accepted as written). Stage 1 = screening only on the already-used 552 parts; stage 2 = one unsealing of the fresh holdout. B14 / B16 near-misses are one hint, not two. Fresh holdout: 94 candidates sealed (a2dac8d1 / manifest a5a133a2; 72 source groups, 42 / 40 new tokens); gate extractor must first reproduce the LOG gates on the 552. The 10 GB evidence is copied to /home and hash-verified (not a remote backup). - 2026-11-17: **B16 PRUNED (fixed combination recipe).** 552 / 204 test parts, pre-score implementation review PASS (MeltAssist, 8cdd3ff4). New - E1b field +.00014, CI [-.00051, +.00079]; last fill -.039, CI [-.090, +.004]; τ -.0011, CI [-.0019, -.0004] (worse). New - A field -.00099, CI crosses 0. **Learned:** the H1-RES target does not add to B13 context. Last fill is a near-miss again (B14 too). Two near-misses on the same data are a hint, not evidence: any last-fill branch must be confirmed on *fresh* parts. **Next:** (1) v2 extra parts (94, 42 from new families) prepared as a sealed fresh holdout; (2) H2 data-sufficiency test (Mold e24fff7c) after H2-S1; (3) CAD pretraining (Mold 8114bc3c) parked behind H2 (it needs a learned model worth pretraining); (4) Mold to spec one last-fill-targeted branch, confirmation only on the fresh holdout. Literature (MeltAssist 1-3/3): video-model OOD physics failure has limited support (Kang et al. ICML 2025, controlled 2D); PDE pretraining sample efficiency has evidence (Poseidon, CoDA-NO); generation on irregular meshes has precedents, but there is none for 3D volume filling or calibrated uncertainty; CAD sets give geometry only. - 2026-11-17: **B13-E2 PRUNED (fixed type-context recipe).** 552 / 204 test parts, implementation review post hoc PASS (MeltAssist). Raw T-G field +.00028, CI [-.00075, +.00118]; last fill +.019, CI crosses 0; τ CI crosses 0. Field means G .06970, T .06998, C .06973. The adjusted T-G interval is unavailable (13.7 % of replicates have the zero point outside support, above the 5 % rule); the T-C and C-G adjusted intervals cross 0. **Learned:** at this recipe the B13 gain comes from regional averaging of F / D_w* itself, and *which* rule draws the 8 regions (generic, type-organised or shuffled type) makes no detectable difference. Not an equivalence claim. It does not close O1 with other readouts or connectivity. Cost 6,631 / 14,400. -- 2026-11-17: **B15 PRUNED (fixed recipe).** Fold 0 validation, 111 / 35. Field .0918 vs A .0649 (+.0270, CI [+.0160, +.0388]); last fill +.250, CI [+.111, +.408]. Both clearly worse, so learned speed + eikonal solve + volume clock lands near the training-free EIK level. **Learned:** pointwise speed errors integrate along paths; the potential must be corrected *after* the solve (as A does), not only its input speed. It does not close learned potentials or a learned moving interface. Cost 6,354 / 8,000. **B16 opened:** B13 context + H1-RES target, five folds, primary vs E1b. +- 2026-11-17: **B15 PRUNED (fixed recipe).** Fold 0 validation, 111 / 35. Field .0918 vs A .0649 (+.0270, CI [+.0160, +.0388]); last fill +.250, CI [+.111, +.408]. Both clearly worse, so learned speed + eikonal solve + volume clock lands near the training-free EIK level. **Learned (narrowed 2026-11-18):** this recipe failed; a plausible reading is that pointwise speed errors integrate along paths, but several factors changed at once, so it does not show pre-solve learning is generally inferior. It does not close learned potentials or a learned moving interface. Cost 6,354 / 8,000. **B16 opened:** B13 context + H1-RES target, five folds, primary vs E1b. - 2026-11-17: **B14 PRUNED (fixed recipe).** Fold 0 validation, 111 parts / 35 components, coverage 131/132 learning and 35/35 validation components. Field (new - A) -0.000091, CI [-0.000328, +0.000153]; last fill -0.0364, CI [-0.0738, +0.0019]. Both cross 0, so it fails the B12-B15 rule. Cost 13,951 / 16,000 CPU-s. **Learned:** path-family features do not move the field; the last-fill CI only just crosses 0, a weak hint at most and not a finding. It does not refute the channel/topology hypothesis in general. Any reuse must be a new, pre-registered last-fill-targeted branch (e.g. combined with B13 context), not a rescoring of this one. E2 geometry is complete (552/552, 3,868 CPU-s); postfit is being implemented. B15 prediction continuation running. - 2026-11-17: **B13-E1b CONFIRMED** (Melt, local commit 044b99eb; review ZIP 04a9fc8e). All 552 test parts / 204 components, each scored once; the 42 G-invalid parts kept as native NaN. Field relative L2 (new - A) is -0.001128, CI [-0.001999, -0.000260] (A .07083 -> .06970, about 1.6 % relative). τ-b +0.00146, CI [+0.00031, +0.00277]. Last fill +0.0109, CI crosses 0. Four folds improve on the mean; fold 1 is slightly worse with its CI crossing 0. This is the first five-fold-confirmed increment over A: small, and a *generic* region-context effect only. It is not evidence for type organisation (O1) or a physical mechanism. E2 (T-G deployment gate, cap 14,400 CPU-s) is released. A combination of B13 and H1-RES is a later candidate. B1 erratum filed by Melt: the field condition passed, and PARK came from last fill and τ. - 2026-11-17: **B13-E1 PARK_G_COVERAGE** (no fit and no score). The fold-1 test (31/41 components = .756) and fold-3 learning (101/128 = .789) roles fell below the 80 % complete-component gate. This is not a performance failure and not a five-fold confirmation. **B13-E1b** (separately named) is opened: the same frozen recipe on all 552 parts, with native NaN for the two G columns where G is invalid (the fold-0 B13 recipe already kept its 4 G-invalid parts this way), no coverage gate, and the same decision rule. A deployed surrogate must score every new part. The E1 PARK receipt stays. B14 per-part cap raised to 300 s, with one rerun for indices 16, 21 and 22.