Decision record s10: public-data stage close, learning curves, gap to goal, recommendation

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ClaudeandClaude Opus 5.5 committed 2026-11-17 00:54:54 +00:00
1 parent 7b8f0d81a8
commit 09382ec363
1 file changed
+29
@@ -105,3 +105,32 @@ S3-A runs as approved. In parallel, the route-A falsification tests (O-vol oracl
- The faithful Gre24 GBRT (A, 7.08 %) is about the same as the historical R2 (7.30 %).
- **Reading:** the distance-feature GBRT family has plateaued around 7 % on our grouped protocol. Further gains need new information, not more tuning. H3 (the lowest point estimate) is not adopted as a base, to avoid selecting on the test set.
- **Decision:** the next experiment is the minimal O1 test (MoldAstra specification 14Drirmj…; arms B/T/G/C; T vs G primary; 50 % pool; delta = 1 %) on the frozen A base, chosen for fidelity rather than score. Execution contract by MeltAstra; it runs after the R4bc queue finishes.
## 10. Stage close: public Greif data (2026-11-17)
Final R4(b)/(c) results: MeltMind 036290bb. EXACT_MATCH with INTERIM_100, and an independent BITWISE_MATCH re-prediction of the 35 models at 100 %.
**Learning curves (552/204, component-mean relative L2, pools 100 / 50 / 25 %):**
- EIK (training-free): 9.59 % at every pool.
- A, faithful Gre24 GBRT: 7.08 / 7.31 / 7.70 %.
- H3: 7.00 / 7.12 / 7.43 %.
- EIK minus A: +2.50 / +2.28 / +1.88 points (all intervals exclude 0).
**Reading (descriptive; the pools are nested, not independent):**
1. Learned geometric-feature models beat the training-free volume map clearly at every pool. Even the 25 % pool (about a quarter of the training components) keeps most of the gain. This is real **sample efficiency at small n** for this family.
2. The curve is **flat**: four times the data reduces A's error by only about 8 % relative. Within this feature family and these factors, more of the same data buys little. That is not a general floor.
3. **The training-free EIK locates the last-filled region better than A at every pool** (0.342 Rg against 0.377 to 0.420). Field-L2 training gives up end-of-fill location. Which output counts as "useful" therefore decides which model is better.
4. O1: not testable on this data with size-matched K = 8 partitions (eligibility 23 %). The design lesson: cut area and region size belong in the analysis as covariates, not as matching constraints.
**What is still missing between the evidence and the goal (a practical surrogate for new parts, sample-efficient):**
- (a) **An acceptance definition.** Point 3 shows the ranking of models flips between field L2 and last-fill location.
- (b) **Interventions.** Greif v5 has one random gate and one automatic process per part, so gate selection and process generalisation can't be tested.
- (c) **New information beyond distance features**, because the learning curve is flat. The most promising candidate is a **multi-fidelity route**: a cheap low-fidelity simulation (2.5D / Hele-Shaw / coarse-mesh quick analysis from the CAE itself) as input, and a learned correction to full 3D. Our own graph solver was infeasible, but a CAE-provided low-fidelity solve avoids that.
**Recommendation to Yanqi:**
1. Fix the acceptance outputs and their priority, for example last-filled region, weld lines, fill-time field, gate ranking.
2. In-house CAE data request:
- (i) parts with two or more candidate gates, solved in pairs;
- (ii) process variation on the same parts;
- (iii) low/high-fidelity pairs on the same geometry and gate.
3. Stop further incremental work on public Greif fill time; keep the reproduced Gre24 GBRT as the public anchor.
4. Bring O1 back inside the multi-fidelity correction, with covariate-controlled partitions, once (2) exists.