Files
MeltMind/docs/claude/stage-review-2026-11-19.md
T

5.2 KiB

Stage review after phase 3 (Claude, 2026-11-19)

Yanqi: at big stage points, everyone summarises, discusses and re-researches. This file is the brief. Claude rules afterwards and sends Yanqi one short stage report.

Scored facts (details and CIs in research-tree.md)

Item Result
H2 (learned representation), grouped 552/204 Field 7.08 → 6.58 % vs A, confirmed; last fill worse (+.080 Rg)
seed 1 Field seed spread ~.00015 (tiny); last-fill seed spread ~.048 Rg (large)
Item 1 data sufficiency Unidentified / overfit signs; more data helps on validation only
c (last-fill objective in H2) Fails; secondary tail error worse
b (H2 + B13 context) Fails; worse on the 42 parts without context
Track A (four random splits) H2 3.25 % vs A 3.99 % (paired CI excludes 0); A does not reproduce Gre24's 3.31-3.38 %, so no paper claim
1b (100-epoch schedule) Parked
Fresh holdout Blocked (gate calibration v2 failed)
Track B (GINO ShapeNet) Not started

Questions for each member (independent answers first, then discussion)

  1. What do these results say about where H2's remaining error comes from (field vs last-fill location), and what is the single most promising next branch? Justify against the evidence above.
  2. Track A: what would make a Gre24 comparison legitimate (reproduce A at 3.31-3.38 % on their protocol, or argue the gap)? Is it worth the cost now, or should Track B (GINO public benchmark, cleaner protocol) take priority for the SOTA goal?
  3. Last fill: given the seed spread (.048 Rg) and two failed recipes (B17, c), is last fill a target to keep attacking, to measure differently (set-based / multi-seed), or to park?
  4. Process: what went well and what cost us time in phase 3 (routing outage, budget stops, missing contract field, review rounds)? One concrete rule each.

Who

  • MeltSol: executor view: costs, failures, what infrastructure would make the next phase cheaper. Also the bulk literature view on Track B SOTA.
  • MeltAssist: reviewer view: where review caught or missed defects.
  • MoldAstra: maths view on Q1 and Q3 (short).
  • TRON: routing and coordination lessons (Q4 only).
  • NewMeltAstra, MeltAstra: not asked (save Astra quota); MeltAstra retires now that c, b and Track A are scored.

Each answer: at most one page, posted to Git (MeltSol publishes), link in Slack. No new compute during the review.

Ruling (Claude, 2026-11-19, after the four answers at 52c74d25)

Answers: MeltSol, MeltAssist, MoldAstra (Q1/Q3), TRON (Q4). Disagreement: MoldAstra puts 1b (short full schedule) first; MeltSol and MeltAssist put Track B first.

  1. Track B is the main line. The goal is SOTA on public 3D benchmarks with same task, same data and fair splits. The GINO protocol gives that; Greif does not without a reproduction we cannot currently make. Order: (a) MeltSol checks the ShapeNet-611 sample, vertex and pressure mapping (Zenodo 13936501 v2 against the official GINO implementation) and freezes split, inputs and de-normalised relative L2; (b) reproduce the GINO decoder with the paper recipe; the gate is the reported 7.12 % within its reported error; (c) only then H2-type models on the same protocol. Targets on record: GINO 7.12 %, OTNO 6.70 % (ShapeNet); Ahmed (GINO 8.31 %, PCNO 6.82 %, MNO 4.68 %) stays on hold until the original 551 identities and metric equivalence are confirmed.
  2. 1b runs in parallel, because it is cheap and already authorised. MoldAstra's point holds: 1b stopped on a cost projection, not a result, and it directly tests the overfitting signal from item 1. It reruns as a new labelled entry (budget stop, not infrastructure) inside Yanqi's existing 1b authorisation (≤ 25 GPU-h, 5.35 already spent): cap from measured co-located epochs + 25 %. If that projection exceeds the remaining authorisation, it is not launched and the number is reported. Narrow review only (schedule, cap and guard; science fields unchanged), then a fold-0 screen against the 200-epoch baseline with the seed-1 reference.
  3. Last fill: no new dedicated loss. Hard endpoint and last-1 % tail error stay as reported metrics, by seed, paired. A dedicated last-fill objective comes back only with a cross-seed repeatable regional gain or an identified, fixable failure mechanism (MoldAstra).
  4. Track A: closed at the four-split descriptive result. H2 3.25 % vs A 3.99 % on our random splits; no claim against Gre24's table. Reproducing 3.31-3.38 % is deferred, not scheduled.
  5. Process rules adopted (all four answers converged):
    • every new dispatch entry runs its real frozen contract through the real guard on CPU (all fields, one empty-card dispatch, one busy-card timeout) before review;
    • caps from measured epochs at the target concurrency + 25 %; waiting, compute and remaining projection booked separately; waiting counts once in the total, never in a per-fold runaway check; a stop receipt names its trigger;
    • routing: send by session ID + host; record sent / acknowledged / started / terminal; a missing route is a blocker (try reconnecting first), never a Slack substitute or a change of executor;
    • handoff text states the scope and end of every restriction (the read-only episode);
    • reviews of unchanged shared components check only the new delta and interface receipts.