stage review ruling: Track B main line, 1b in parallel, process rules
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
98a754a636
commit
3a48b123da
2 files changed
+16
No files matched your search
@@ -34,6 +34,7 @@ Rules:
|
||||
| P3 | O1 with size-matched K = 8 partitions | — | — | PRUNED (eligibility 23 %; lesson: control cut area as a covariate, do not match it) |
|
||||
|
||||
Log (newest first):
|
||||
- 2026-11-19: **Stage review ruled** (answers at 52c74d25; ruling in stage-review-2026-11-19.md): Track B (GINO ShapeNet protocol) is the main line, starting with the data-mapping check and a GINO reproduction gate (7.12 %); 1b reruns in parallel inside its existing authorisation with a co-located cap and narrow review; no new last-fill loss; Track A closed at the four-split descriptive result; five process rules adopted (contract → real guard CPU simulation, concurrency-measured caps with waiting booked separately, four-state routing, scoped restrictions in handoffs, delta-only reviews of unchanged components).
|
||||
- 2026-11-19: **Item b (H2 + frozen B13 context inputs) DOES_NOT_PASS its fixed screen; PRUNED (this recipe).** Recovery postfit (PASS 029a77a2) COMPLETE, 552 / 204, evidence 192ce24a. b − H2 seed 0: field −.00006, CI [−.0018, +.0017] (needed upper bound < −.001 and |effect| > seed spread .00015: neither met); last fill −.044 Rg, CI [−.130, +.035]; τ-b +.0002, CI crosses 0. Descriptive strata (no gate): 510 parts with valid B13 context −.0008, CI [−.0026, +.0009]; 42 historically missing +.0089, CI [+.0026, +.0156] (worse where the context is absent and zero-filled). **Reading:** B13's tabular context adds nothing measurable to H2; the representation appears to already carry the regional information that helped A. Phase-3 scored set is now complete: H2 confirmed on field; seed noise small for field, large for last fill; c and b pruned; 1b parked; Track A shows H2 < A on our random splits without a paper-level reproduction. **Stage review opened** (docs/claude/stage-review-2026-11-19.md).
|
||||
- 2026-11-19: **Item c (last-fill-aware objective inside H2) FAILS its pre-registered screen; PRUNED (this recipe).** Recovery postfit (PASS 59a6255f, one labelled infrastructure recovery after a pre-worker guard KeyError, 0 targets read in the stop) COMPLETE, 552 / 204. c − H2 seed 0: last fill +.0193 Rg, CI [−.0559, +.0948] (gate needed ΔD ≤ −.010 with upper bound < 0, and |effect| > the same-domain seed spread .0482: not met); field +.00029, CI [−.0013, +.0019] (upper bound above +.001: fails); τ-b +.00037, CI crosses 0. Secondary (reported separately, no gate): last-1 % tail error E_tail +.066, CI [+.044, +.087], i.e. *worse* tail placement; peak certificate 0 for every arm. **Reading:** this location-aware objective did not move the last-filled point, and by the secondary metric it moved the late region the wrong way. That closes this recipe, not the question: the S2 diagnostic (more far-from-tail picks) still stands as H2's main weakness. 630.73 GPU-s postfit including the stop.
|
||||
- 2026-11-19: **Track A (Gre24-style random-split protocol) scored, and seed-1 noise reference scored** (MeltSol, evidence fee71ea6; original bindings 958abd8c/a5fa4a6e and 7b4da089, each run once). **Track A:** splits 0/1/2/4 (split 3 stopped on budget, disclosed; 14 preparation failures disclosed; our own random splits), 1228 paired test visits / 571 unique parts. Split-averaged median per-part RMSE/max: A 3.99 % → H2 3.25 %; per-part paired Δ −.00742, CI [−.00844, −.00646]. **Reading:** on the same re-created random splits H2 clearly beats A. This is *not* a SOTA claim against Gre24's table: our A on these splits gives 3.99 %, not the paper's 3.31-3.38 %, so the plan's reproduction gate is not met and the gap between H2 3.25 % and the paper number is not interpretable (four splits, different seeds, random splits leak families). **Seed-1 vs seed-0 (H2, grouped protocol, 552/204):** field +.00015, CI [−.00143, +.00204]; last fill −.048 Rg, CI [−.122, +.024]; τ +.00004, CI crosses 0. **Reading:** the seed spread is small for the field (about 1/30 of H2's −.005 gain, so the field confirmation is not a seed artefact). For last fill it is about .05 Rg, comparable to H2's +.080 last-fill loss, so single-seed last-fill differences are noisy. c and b will be read against the same-domain seed-1 reference, as their bindings specify.
|
||||
|
||||
@@ -32,3 +32,18 @@ Yanqi: at big stage points, everyone summarises, discusses and re-researches. Th
|
||||
- NewMeltAstra, MeltAstra: not asked (save Astra quota); MeltAstra retires now that c, b and Track A are scored.
|
||||
|
||||
Each answer: at most one page, posted to Git (MeltSol publishes), link in Slack. No new compute during the review.
|
||||
|
||||
## Ruling (Claude, 2026-11-19, after the four answers at 52c74d25)
|
||||
|
||||
Answers: MeltSol, MeltAssist, MoldAstra (Q1/Q3), TRON (Q4). Disagreement: MoldAstra puts 1b (short full schedule) first; MeltSol and MeltAssist put Track B first.
|
||||
|
||||
1. **Track B is the main line.** The goal is SOTA on public 3D benchmarks with same task, same data and fair splits. The GINO protocol gives that; Greif does not without a reproduction we cannot currently make. Order: (a) MeltSol checks the ShapeNet-611 sample, vertex and pressure mapping (Zenodo 13936501 v2 against the official GINO implementation) and freezes split, inputs and de-normalised relative L2; (b) reproduce the GINO decoder with the paper recipe; the gate is the reported 7.12 % within its reported error; (c) only then H2-type models on the same protocol. Targets on record: GINO 7.12 %, OTNO 6.70 % (ShapeNet); Ahmed (GINO 8.31 %, PCNO 6.82 %, MNO 4.68 %) stays on hold until the original 551 identities and metric equivalence are confirmed.
|
||||
2. **1b runs in parallel, because it is cheap and already authorised.** MoldAstra's point holds: 1b stopped on a cost projection, not a result, and it directly tests the overfitting signal from item 1. It reruns as a new labelled entry (budget stop, not infrastructure) inside Yanqi's existing 1b authorisation (≤ 25 GPU-h, 5.35 already spent): cap from measured co-located epochs + 25 %. If that projection exceeds the remaining authorisation, it is not launched and the number is reported. Narrow review only (schedule, cap and guard; science fields unchanged), then a fold-0 screen against the 200-epoch baseline with the seed-1 reference.
|
||||
3. **Last fill: no new dedicated loss.** Hard endpoint and last-1 % tail error stay as reported metrics, by seed, paired. A dedicated last-fill objective comes back only with a cross-seed repeatable regional gain or an identified, fixable failure mechanism (MoldAstra).
|
||||
4. **Track A: closed at the four-split descriptive result.** H2 3.25 % vs A 3.99 % on our random splits; no claim against Gre24's table. Reproducing 3.31-3.38 % is deferred, not scheduled.
|
||||
5. **Process rules adopted** (all four answers converged):
|
||||
- every new dispatch entry runs its *real frozen contract through the real guard* on CPU (all fields, one empty-card dispatch, one busy-card timeout) before review;
|
||||
- caps from measured epochs at the target concurrency + 25 %; waiting, compute and remaining projection booked separately; waiting counts once in the total, never in a per-fold runaway check; a stop receipt names its trigger;
|
||||
- routing: send by session ID + host; record sent / acknowledged / started / terminal; a missing route is a blocker (try reconnecting first), never a Slack substitute or a change of executor;
|
||||
- handoff text states the scope and end of every restriction (the read-only episode);
|
||||
- reviews of unchanged shared components check only the new delta and interface receipts.
|
||||
Reference in new issue
Block a user