Files
MeltMind/docs/s1-real-scoring-results-2026-11-14.md
T

8.5 KiB
Raw Blame History

S1 M2 v1.2 — 24 supervised LOPO folds

All 24 folds completed and passed verification. The two primary contrasts have opposite directions: B ranks behind its shuffled node/gate control, while C ranks ahead of the ordinary edge-shuffle control. This does not meet the proposed joint interpretation that both organized blocks make inputs more familiar. No prediction model was trained.

Primary result table

Decisions use the five-permutation-seed mean and the 固定 LOPO 配对差的来源重加权区间 (source-reweighting interval over fixed LOPO paired differences). Differences are absolute M2 rank-distance units; smaller is more familiar. These are two-sided 95% linear-percentile intervals from seed 1407 with 2,000 resamples. They do not refit ranks or reference pools and do not guarantee 95% coverage for new sources. Overlapping LOPO training folds remain fixed.

Contrast Query slice Five-seed mean difference Fixed-LOPO source-reweighting interval Ranking Seeds with same point direction / interval classification
B − (A+S) Whole +0.00262105 [+0.00137530, +0.00408964] B behind 5/5 / 5/5
B − (A+S) Near wall +0.00171341 [+0.00039639, +0.00318377] B behind 5/5 / 5/5
C − (B+edge shuffle) Whole −0.01495009 [−0.01620100, −0.01378732] C ahead 5/5 / 5/5
C − (B+edge shuffle) Near wall −0.01582829 [−0.01691546, −0.01481519] C ahead 5/5 / 5/5

Per-seed results are stability diagnostics only; they do not vote on the primary decision. An interval including or touching zero would mean 未分先后 / not separated, never equivalent. decision_table.csv contains full precision, both source and query intervals, valid replicate counts, source-equal and part-equal differences, and the two separately labeled seed counts. Every contrast/slice has all 2,000 valid source replicates and 200 valid query replicates.

The secondary normalized edge-shuffle contrast is much smaller: C minus the stratified control is −0.00055111, interval [−0.00091199, −0.00023184] for whole queries, and −0.00055050, interval [−0.00095840, −0.00015434] for near-wall queries. Both point directions and interval classifications agree across 5/5 seeds. It remains a secondary diagnostic, not a replacement for a failed primary comparison.

B−A and C−B are included as descriptive cross-dimension differences. RMS normalization does not remove dimension/redundancy effects. These coverage measurements establish neither filling prediction accuracy nor improved sample efficiency.

Complete evidence

All six arms, all five seeds, both slices and all 24 sources are retained. arm_summary.csv gives aggregate scores and intervals; part_scores.csv gives per-part scores, coverage and query-bootstrap diagnostics; seed_stability.csv reports each contrast/seed without assigning it primary decision status. Source-equal and part-equal totals agree to floating precision because this panel has one part per source.

query_clipping.csv, control_clipping_summary.csv, neighbor_diagnostics.csv and shuffle_audits.json expose numerical and control behavior. Full per-reference-part clipping details, rank maps/weights, physical reference IDs, eight selected neighbors and distances, node scores, query draw-slot bootstrap counts, and replicate estimates are retained in each fold's structure.json plus arrays.npz inside the evidence archive. Nothing was dropped from failed/rejected intermediates; there were no failed real folds.

Across 24 distinct parts × 5 permutations (218,280 eligible slots, counted once per part/seed, not 23 times for repeated training folds):

  • Ordinary edge shuffle produces 187,525 nodes with raw-bound excess above 1e-10 and 106,156 normalized cross-channel order violations above 1e-10. This control deliberately breaks the association with receiver gaps. Its large contrast alone does not isolate physically feasible local structure.
  • Stratified normalized shuffle has zero bound or cross-channel order violations above 1e-10. Exact comparisons find 325 bound and 339 order violations, with maximum excess 8.881784197001252e-16 in either category; these are retained without repair.
  • Query clipping occurs in all ten columns somewhere in the panel. The largest combined tail fraction for one part/column/slice is 0.1865. No training column is constant in these folds. Complete ranges, sides and denominators are published.

Execution and verification

  • Frozen scorer is unchanged from e425419c8012127400e8945baa8d04e2953646bd. Original O2/draw inputs remain those released at a86a74cbd943e0687ee69fb1cccdc003250113cc / 94011fd8efe0512e6579c7c541ccd585dc710c2e.
  • Claude main review and conditional release, Grok independent seed/formula PASS, interval label agreement, actual continuation receipt. Claude reviewed source and hashes but did not rerun synthetic tests in his environment; Grok independently replayed the vectors and formula, not the whole scorer.
  • Execution contract SHA256: c12c54f825c7c1c958a6951e3ea62c6339404bb1328511482790f4dfff528210. Both presentation pins were fixed before launch. The supervisor adapter reuses the existing whole-process memory-tree implementation, 20 ms polling target, TERM/KILL handling and final peak check; only job invocation and receipt handling were adapted to S1.
  • 24 folds ran sequentially with 1,800 s / 1.25 GiB limits per complete child process, including input load, scoring and serialization. No budget was enlarged. Total supervised fold time 270.5283 s, maximum 13.2551 s, maximum peak RSS 146.0078 MiB. One child process was observed at a time.
  • Polling was a 20 ms target, not a timing guarantee: the largest observed gap was 0.586656 s. Per-process high-water marks and final ru_maxrss supplement polling. All raw traces and supervisor receipts are retained.
  • Exactly 785,808 logical 8NN queries, 785,808 radius searches, 432 tree builds, and 6,286,464 direct candidate distances. Every fold matches query_cost.csv; candidate count is exactly eight per logical query on this dataset. Near-wall slicing and bootstrap add no kNN queries.
  • Four synthetic execution/storage tests pass, including normal completion, timeout, memory-stop and byte-preserving NPZ roundtrip. Three additional reporting tests prove that only the mean interval decides, seed agreement is separate, zero endpoints mean not separated, and the secondary control is not promoted to a primary decision. The original 14 scorer tests and their source are unchanged.
  • Verification checks every saved file against the supervision receipts; recomputes every stored point/query-replicate ratio; checks eight distinct physical neighbors, score/distance means, source isolation, rank/weight invariants, baseline reuse and part permutations across folds. Aggregation used no new kNN queries or rank fits. Its separate runtime was 10.7197 s and peak RSS 125.9844 MiB.

Reading and replaying the archive

The archive includes every real-run artifact, exact execution/reporting source, frozen input metadata needed for verification, original approvals, test logs and a file-by-file SHA256 manifest. It uses workspace-relative paths. structure.json refers to numbered arrays in arrays.npz; run_s1_fold.unpack_tree restores numeric arrays and tuple-key diagnostics with allow_pickle=False. Nonfinite scalar tags are explicit; array bit patterns are preserved.

For a report-only replay after extraction, keep the extracted workspace as the current directory. Create a new empty report directory, copy execution_contract.json and completion.json into it, and symlink its folds to the extracted real-run folds. Run tools/report_s1_scoring.py NEW_REPORT_DIRECTORY using Python 3.10 / NumPy 1.26.4 / SciPy 1.15.3 and OMP_NUM_THREADS=2 OPENBLAS_NUM_THREADS=2. This validates and aggregates fixed stored scores without loading real descriptors for new scoring. Compare its eight CSV tables and aggregate.json with the originals. Re-running the real folds would require the earlier full O2/draw evidence packages and a separate authorized execution scope.

The package is submitted to Claude for main result review. Grok's required targeted check is complete; no duplicate full review is requested.