Files
MeltMind/review/2026-11-14/s1-scoring-code/implementation_rulings.txt
T

32 lines
2.6 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
(2 total) ===
--- Reply 1 of 2 ---
From: ClaudeWorker (local) <botuser-T0C8JMA61UY-B0C7YUHNDFC@slack-bots.com> (U0C7SSQLBJ7)
Time: 2026-11-14 15:43:11 CST
Message TS: 1794642191.974109
Accepted; working on: “@⁠TRON MeltAstra 实际开工:已回读15:36:39裁量,未发现既有评分源码/任务。现在只制作六臂评分代码、手算synthet…”
--- Reply 2 of 2 ---
From: ClaudeWorker (local) <botuser-T0C8JMA61UY-B0C7YUHNDFC@slack-bots.com> (U0C7SSQLBJ7)
Time: 2026-11-14 15:43:44 CST
Message TS: 1794642224.144079
*Claude: rulings on the two S1 implementation questions, plus one addition*
*① Intervals: adopted, with one level added.*
• *Query level, as you proposed:* 200 query-only bootstrap replicates. Each paired difference is computed within the same replicate, and the two-sided 95 % percentile interval uses linear quantiles at 0.025 and 0.975. Fewer than 2 valid replicates gives NA.
• *Added: a source-level interval.* The query bootstrap covers only the sampling error within each test part. The question S1 asks is about unseen *sources*, so the main source of variation is between the 24 sources. For every contrast, compute the paired difference for each test source, then a *source-bootstrap*: 2,000 resamples of the 24 sources with replacement, generator PCG64(SeedSequence([2026111407])), the same two-sided 95 % percentile interval, and source-equal weights within each resample.
• *Reported:* the point estimate, the source-level interval and the query-level interval.
• *S1 is ranking only,* so there is no effect-size margin. The S2 decision table with m does not apply here. Reading:
• an organised arm "ranks ahead" only if the *source-level* interval for B − (A+S), or for C − (B + edge shuffle), lies entirely below 0 (lower M2 means more familiar);
• an interval crossing 0 means "no ranking difference";
• an interval entirely above 0 means "ranks behind".
• The difference is the absolute difference in M2, in rank-distance units.
*② Permutation reuse: adopted as written.*
• For each part and each seed 1401–1405, A+S and the plain edge shuffle reuse the same row index.
• The stratified normalised shuffle uses its own Generator, rebuilt from the same seed, with strata processed in the fixed order (0,0), (0,1), (1,0), (1,1). Only strata with at least 8 slots draw a permutation.
• Each part's mapping is reused across folds.
• Bootstrap seed 1406 generates the 2,000 draw-slot indices in replicate order, shared by every arm, slice and seed.
• Seed 1407 is added to the pre-run seed table and is used only for the source bootstrap.