Add requested threshold fractions and case-paired differences from stored scores

This commit is contained in:
Frank14f committed 2026-11-16 06:43:26 +08:00
1 parent fb77657e3f
commit 7f611ab7b3
4 files changed
+28317

No files matched your search

@@ -0,0 +1,64 @@
{
"unix": 1794782466.574412,
"commit": "9bf495ae0b1badd08565d42dcb22e4eeeb722df0",
"method": "Independent HTTPS raw commit readback, compared to local exact bytes",
"records": [
{
"path": "docs/gre24-v5-reproduction-results-2026-11-16.md",
"http_status": 200,
"bytes": 8270,
"local_sha256": "59151094c1d12d40f5ba587304fb6033ecbf4637a41ecff8381b67b729ea4be7",
"remote_sha256": "59151094c1d12d40f5ba587304fb6033ecbf4637a41ecff8381b67b729ea4be7",
"match": true
},
{
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/final-terminal-audit-2026-11-16.json",
"http_status": 200,
"bytes": 18041,
"local_sha256": "3dcba59b459d87717e5b2f883d29eb2ae62c4fb3c1eff2cf2c0d5005f4a2fdc7",
"remote_sha256": "3dcba59b459d87717e5b2f883d29eb2ae62c4fb3c1eff2cf2c0d5005f4a2fdc7",
"match": true
},
{
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/reproduction-complete.json",
"http_status": 200,
"bytes": 9095,
"local_sha256": "ce4023152b969dfd60b8d6ead58aed9e7a29ff354def1b7e61e7955837e307fa",
"remote_sha256": "ce4023152b969dfd60b8d6ead58aed9e7a29ff354def1b7e61e7955837e307fa",
"match": true
},
{
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/score-summary.json",
"http_status": 200,
"bytes": 15935,
"local_sha256": "ebfe3314147d56bda9a356797be5fb9650b48c0cd04c07bb70d4a913143e3884",
"remote_sha256": "ebfe3314147d56bda9a356797be5fb9650b48c0cd04c07bb70d4a913143e3884",
"match": true
},
{
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/models-freeze.json",
"http_status": 200,
"bytes": 7040,
"local_sha256": "2ce1e3ffd4ffcf4f2c5a79ec734c76e220093febb988cd334347fbfef1663419",
"remote_sha256": "2ce1e3ffd4ffcf4f2c5a79ec734c76e220093febb988cd334347fbfef1663419",
"match": true
},
{
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/predictions-freeze.json",
"http_status": 200,
"bytes": 1939755,
"local_sha256": "8104a960cf32c4f5f57336c71369ad47bb1c808145b7d15bf9e6c2fda23d1169",
"remote_sha256": "8104a960cf32c4f5f57336c71369ad47bb1c808145b7d15bf9e6c2fda23d1169",
"match": true
},
{
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/scores.json",
"http_status": 200,
"bytes": 2147188,
"local_sha256": "af2aa9fdb207c86c4ff91a75a33e84734e3995f37fd316ae394bfd231f57a674",
"remote_sha256": "af2aa9fdb207c86c4ff91a75a33e84734e3995f37fd316ae394bfd231f57a674",
"match": true
}
],
"status": "PASS"
}
@@ -0,0 +1,11 @@
{
"unix": 1794782533.6830049,
"worker_cpu_seconds": 0.165208997,
"wall_seconds": 0.16521130001638085,
"source_scores_sha256": "af2aa9fdb207c86c4ff91a75a33e84734e3995f37fd316ae394bfd231f57a674",
"original_scores_unchanged": true,
"scope": "Read stored scalar scores and summarize only; no model/array/label loading or rescoring",
"original_completed_CPU_seconds": 136620.98104499982,
"cumulative_with_display_CPU_seconds": 136621.14625399682,
"GPU_seconds_added": 0
}
@@ -0,0 +1,39 @@
# Gre24 v5:已有评分的阈值及配对差展示补充
按Claude原串1794782386.838149要求,仅从固定 `scores.json` 读取既有逐件标量并汇总。原评分SHA `af2aa9fdb207c86c4ff91a75a33e84734e3995f37fd316ae394bfd231f57a674` 前后不变,原summary、预测、模型不改写;无新训练、预测、评分、bootstrap或标签读取。三分母min/median/max原表见已发布 `docs/gre24-v5-reproduction-results-2026-11-16.md`(9bf495a)。
## RRMSE严格小于0.1的比例
每格为通过件数/该split有效评分件数及比例;614成功/10准备超时的固定排除不变,五split分别有效307/308/306/305/308,缺5/4/6/7/4。分母是各split有效评分件数,不是312或节点数。五split比例等权平均,不先池化各split样本;无undefined评分。全部数值均来自原标量,严格小于0.1,不是小于等于。
| 分母 | 模型 | split0 | split1 | split2 | split3 | split4 | 五split比例平均 |
| --- | --- | --- | --- | --- | --- | --- | ---: |
| RMS(y) | naive | 174/307 (56.6775%) | 190/308 (61.6883%) | 173/306 (56.5359%) | 180/305 (59.0164%) | 181/308 (58.7662%) | 58.536882% |
| RMS(y) | GBRT | 238/307 (77.5244%) | 235/308 (76.2987%) | 226/306 (73.8562%) | 235/305 (77.0492%) | 232/308 (75.3247%) | 76.010639% |
| RMS(y) | FNN | 222/307 (72.3127%) | 220/308 (71.4286%) | 214/306 (69.9346%) | 207/305 (67.8689%) | 231/308 (75.0000%) | 71.308954% |
| max(y) | naive | 272/307 (88.5993%) | 272/308 (88.3117%) | 261/306 (85.2941%) | 267/305 (87.5410%) | 262/308 (85.0649%) | 86.962215% |
| max(y) | GBRT | 293/307 (95.4397%) | 291/308 (94.4805%) | 291/306 (95.0980%) | 292/305 (95.7377%) | 295/308 (95.7792%) | 95.307045% |
| max(y) | FNN | 288/307 (93.8111%) | 287/308 (93.1818%) | 285/306 (93.1373%) | 281/305 (92.1311%) | 286/308 (92.8571%) | 93.023688% |
| mean(y) | naive | 145/307 (47.2313%) | 158/308 (51.2987%) | 150/306 (49.0196%) | 157/305 (51.4754%) | 157/308 (50.9740%) | 49.999803% |
| mean(y) | GBRT | 207/307 (67.4267%) | 206/308 (66.8831%) | 203/306 (66.3399%) | 209/305 (68.5246%) | 210/308 (68.1818%) | 67.471221% |
| mean(y) | FNN | 193/307 (62.8664%) | 197/308 (63.9610%) | 187/306 (61.1111%) | 170/305 (55.7377%) | 200/308 (64.9351%) | 61.722274% |
## 同件配对的naive与GBRT差
每个split按相同case配对,先算 **Δ = RRMSE_naive − RRMSE_GBRT**,再对该split的Δ取中位数。正值表示GBRT误差较低;这不是两个模型各自中位数相减。五split配对数依次307/308/306/305/308,原失败排除不补替;逐对原数与Δ也在补充JSON内。数值为比例,非百分比。
| 分母 | split0 median(Δ) | split1 median(Δ) | split2 median(Δ) | split3 median(Δ) | split4 median(Δ) |
| --- | ---: | ---: | ---: | ---: | ---: |
| RMS(y) | 0.018514102312 | 0.017794192390 | 0.019783545436 | 0.017345627252 | 0.020289906179 |
| max(y) | 0.011086873306 | 0.010534568802 | 0.011531321491 | 0.010271705431 | 0.011972139328 |
| mean(y) | 0.021249401762 | 0.020732402248 | 0.023199691516 | 0.019992931523 | 0.023283182183 |
## 参考事实、实现选择与未能确认的内容
论文参考统计:Claude1794782386.838149报告原文300/314约95.5%,并说明原文没有定义RRMSE分母。本补充引用其团队原文核读记录,不声称本轮重新阅读原论文或确认其分母。
本次实现选择:明确列出三分母,RMS(y)为原固定主指标;仍为v5、t_set时间尺度、fill-only早停FNN及R1补充离散化。max(y)下GBRT阈值比例平均95.307045%,数值接近上述参考;主RMS(y)下为76.010639%。这说明同一批既有误差采用不同分母会产生很不同的展示,不能据接近某参考值反向选择主指标。
未能确认:原论文实际采用何种分母、v4/v5差异、t_set与V/Q、FNN多输出及离散化的独立影响均不能由这张表识别,10较大件超时排除与近重复跨split亦未消除。相近百分比不是忠实复现认证;本补充没有把总体差距归因给某一个候选因素。
新增展示汇总实际CPU **0.165208997秒**,GPU新增0。原完成账本136620.981045秒不改写;单列展示费用后累计 **136621.146253997秒/216000秒**。原GPU876.330852390秒/7200秒不变。费用回执 `stored-score-display-receipt-2026-11-16.json`,完整结果 `stored-score-display-supplement-2026-11-16.json` 均位于v5-models-continuation结果根。