diff --git a/docs/claude/planning-discussion-2026-11-19.md b/docs/claude/planning-discussion-2026-11-19.md index 791af943f..819da91cf 100644 --- a/docs/claude/planning-discussion-2026-11-19.md +++ b/docs/claude/planning-discussion-2026-11-19.md @@ -20,3 +20,26 @@ For each answer state: **key dependencies, cost (GPU-h, people time, licences), ## Who MeltSol (executor + literature), MeltAssist (reviewer), MoldAstra (maths), NewMeltAstra (fresh view; has implemented nothing), TRON (coordination only, optional). Answers go to Git via MeltSol, links in Slack. Claude writes its own answer only after collecting the others (to avoid anchoring), then hosts one discussion round and rules. + +## Claude's answer and ruling (after reading all four at 2c0dd614) + +The four answers converge closely, so I skip a separate discussion round (saves Astra quota) and rule directly. The one difference (MeltAssist: GINO as a baseline on Greif; the others: GINO vs H2 on ShapeNet) is resolved by ordering both. + +**Short term: make H2's claim exact, with zero new training.** +1. Finish 1b fold 0 (running) and read it against its pre-registered screen. +2. From existing frozen per-part scores (seed 0, seed 1, A): a family-weighted, per-seed × per-fold paired table of field error, last-1 % tail region and hard endpoint, keeping failed-part denominators. Pre-registered questions: is the field gain concentrated in a few families (leave-one-family-out sign flips)? Do both seeds miss the true tail on the *same* parts (a systematic failure worth a mechanism study) or on different ones (noise)? MeltSol builds it, MeltAssist checks it; 0 GPU-h, about 1-2 person-days. +3. H2's claim stays limited to: field gain on the grouped Greif protocol; last-fill cost; no new-family or paper-level claim. No third seed until the table shows it would change a decision. + +**Medium term: one comparison chain, in this order.** +1. Track B: reproduce GINO on ShapeNet-611 (gate 6.62-7.62 %, three seeds). +2. H2 adapted to pressure (identity-labelled "H2-P"; molding-specific inputs removed) vs GINO on the same 500/111, same raw geometry, equal training budget and tuning allowance, three paired seeds, preprocessing and inference costs reported. This is our route to a public 3D benchmark claim and tests whether H2's representation carries over without molding priors. +3. Then GINO as a strong published baseline on Greif fill time (MeltAssist's proposal), same grouped protocol and inputs as H2, two folds × two seeds first. If GINO matches or beats H2 there, it becomes the default baseline. +Each step needs its own pre-registered gate and review; budgets come from measured epochs + 25 % (current estimates 17-50 GPU-h for step 2, 20-45 for step 3). + +**Long term: genuinely new families, with the inputs that make fill time identifiable.** +- The learning target is fill time given geometry, gate/inlet history, material and thermal boundary (MoldAstra): data without these cannot be fixed by a bigger model. +- Evaluation design: whole-family holdout plus paired interventions inside a new family (one gate or process change per part, NewMeltAstra), reporting whether the model ranks the two designs correctly, not only field error. +- Zero-GPU first: MeltSol inventories existing archives with CAD, process conditions, solver version and reuse terms (3-5 person-days). +- **Blocking dependency: data rights.** The student licence does not cover batch generation. A 10-family pilot (10 parts × 2 conditions = 20 reference solves) needs either a licence or partner data. This is a question for Yanqi. + +**What would change this plan:** GINO not reproducible (pause the ShapeNet ranking, go to step 3 directly); H2-P consistently behind GINO at equal inputs and budget (H2 stays a molding model, GINO-type becomes the 3D default); the family table shows the gain concentrated in a few families (narrow the claim before scaling anything). diff --git a/docs/claude/research-tree.md b/docs/claude/research-tree.md index e3e870b1b..bede615d9 100644 --- a/docs/claude/research-tree.md +++ b/docs/claude/research-tree.md @@ -34,6 +34,7 @@ Rules: | P3 | O1 with size-matched K = 8 partitions | — | — | PRUNED (eligibility 23 %; lesson: control cut area as a covariate, do not match it) | Log (newest first): +- 2026-11-19: **Planning ruled** (four independent answers at 2c0dd614; ruling in planning-discussion-2026-11-19.md). Short: 1b + a zero-GPU family/seed/fold paired table from frozen scores. Medium chain: GINO reproduction on ShapeNet-611 → H2-P vs GINO on the same protocol → GINO as a Greif fill-time baseline. Long: new families with full process inputs, whole-family holdout and paired gate/process interventions; blocked on data rights (question to Yanqi). - 2026-11-19: **Stage review ruled** (answers at 52c74d25; ruling in stage-review-2026-11-19.md): Track B (GINO ShapeNet protocol) is the main line, starting with the data-mapping check and a GINO reproduction gate (7.12 %); 1b reruns in parallel inside its existing authorisation with a co-located cap and narrow review; no new last-fill loss; Track A closed at the four-split descriptive result; five process rules adopted (contract → real guard CPU simulation, concurrency-measured caps with waiting booked separately, four-state routing, scoped restrictions in handoffs, delta-only reviews of unchanged components). - 2026-11-19: **Item b (H2 + frozen B13 context inputs) DOES_NOT_PASS its fixed screen; PRUNED (this recipe).** Recovery postfit (PASS 029a77a2) COMPLETE, 552 / 204, evidence 192ce24a. b − H2 seed 0: field −.00006, CI [−.0018, +.0017] (needed upper bound < −.001 and |effect| > seed spread .00015: neither met); last fill −.044 Rg, CI [−.130, +.035]; τ-b +.0002, CI crosses 0. Descriptive strata (no gate): 510 parts with valid B13 context −.0008, CI [−.0026, +.0009]; 42 historically missing +.0089, CI [+.0026, +.0156] (worse where the context is absent and zero-filled). **Reading:** B13's tabular context adds nothing measurable to H2; the representation appears to already carry the regional information that helped A. Phase-3 scored set is now complete: H2 confirmed on field; seed noise small for field, large for last fill; c and b pruned; 1b parked; Track A shows H2 < A on our random splits without a paper-level reproduction. **Stage review opened** (docs/claude/stage-review-2026-11-19.md). - 2026-11-19: **Item c (last-fill-aware objective inside H2) FAILS its pre-registered screen; PRUNED (this recipe).** Recovery postfit (PASS 59a6255f, one labelled infrastructure recovery after a pre-worker guard KeyError, 0 targets read in the stop) COMPLETE, 552 / 204. c − H2 seed 0: last fill +.0193 Rg, CI [−.0559, +.0948] (gate needed ΔD ≤ −.010 with upper bound < 0, and |effect| > the same-domain seed spread .0482: not met); field +.00029, CI [−.0013, +.0019] (upper bound above +.001: fails); τ-b +.00037, CI crosses 0. Secondary (reported separately, no gate): last-1 % tail error E_tail +.066, CI [+.044, +.087], i.e. *worse* tail placement; peak certificate 0 for every arm. **Reading:** this location-aware objective did not move the last-filled point, and by the secondary metric it moved the late region the wrong way. That closes this recipe, not the question: the S2 diagnostic (more far-from-tail picks) still stands as H2's main weakness. 630.73 GPU-s postfit including the stop.