Add requested threshold fractions and case-paired differences from stored scores
This commit is contained in:
1 parent
fb77657e3f
commit
7f611ab7b3
4 files changed
+28317
No files matched your search
+64
@@ -0,0 +1,64 @@
|
||||
{
|
||||
"unix": 1794782466.574412,
|
||||
"commit": "9bf495ae0b1badd08565d42dcb22e4eeeb722df0",
|
||||
"method": "Independent HTTPS raw commit readback, compared to local exact bytes",
|
||||
"records": [
|
||||
{
|
||||
"path": "docs/gre24-v5-reproduction-results-2026-11-16.md",
|
||||
"http_status": 200,
|
||||
"bytes": 8270,
|
||||
"local_sha256": "59151094c1d12d40f5ba587304fb6033ecbf4637a41ecff8381b67b729ea4be7",
|
||||
"remote_sha256": "59151094c1d12d40f5ba587304fb6033ecbf4637a41ecff8381b67b729ea4be7",
|
||||
"match": true
|
||||
},
|
||||
{
|
||||
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/final-terminal-audit-2026-11-16.json",
|
||||
"http_status": 200,
|
||||
"bytes": 18041,
|
||||
"local_sha256": "3dcba59b459d87717e5b2f883d29eb2ae62c4fb3c1eff2cf2c0d5005f4a2fdc7",
|
||||
"remote_sha256": "3dcba59b459d87717e5b2f883d29eb2ae62c4fb3c1eff2cf2c0d5005f4a2fdc7",
|
||||
"match": true
|
||||
},
|
||||
{
|
||||
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/reproduction-complete.json",
|
||||
"http_status": 200,
|
||||
"bytes": 9095,
|
||||
"local_sha256": "ce4023152b969dfd60b8d6ead58aed9e7a29ff354def1b7e61e7955837e307fa",
|
||||
"remote_sha256": "ce4023152b969dfd60b8d6ead58aed9e7a29ff354def1b7e61e7955837e307fa",
|
||||
"match": true
|
||||
},
|
||||
{
|
||||
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/score-summary.json",
|
||||
"http_status": 200,
|
||||
"bytes": 15935,
|
||||
"local_sha256": "ebfe3314147d56bda9a356797be5fb9650b48c0cd04c07bb70d4a913143e3884",
|
||||
"remote_sha256": "ebfe3314147d56bda9a356797be5fb9650b48c0cd04c07bb70d4a913143e3884",
|
||||
"match": true
|
||||
},
|
||||
{
|
||||
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/models-freeze.json",
|
||||
"http_status": 200,
|
||||
"bytes": 7040,
|
||||
"local_sha256": "2ce1e3ffd4ffcf4f2c5a79ec734c76e220093febb988cd334347fbfef1663419",
|
||||
"remote_sha256": "2ce1e3ffd4ffcf4f2c5a79ec734c76e220093febb988cd334347fbfef1663419",
|
||||
"match": true
|
||||
},
|
||||
{
|
||||
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/predictions-freeze.json",
|
||||
"http_status": 200,
|
||||
"bytes": 1939755,
|
||||
"local_sha256": "8104a960cf32c4f5f57336c71369ad47bb1c808145b7d15bf9e6c2fda23d1169",
|
||||
"remote_sha256": "8104a960cf32c4f5f57336c71369ad47bb1c808145b7d15bf9e6c2fda23d1169",
|
||||
"match": true
|
||||
},
|
||||
{
|
||||
"path": "artifacts/gre24-reproduction-2026-11-15/v5-models-continuation-2026-11-16/scores.json",
|
||||
"http_status": 200,
|
||||
"bytes": 2147188,
|
||||
"local_sha256": "af2aa9fdb207c86c4ff91a75a33e84734e3995f37fd316ae394bfd231f57a674",
|
||||
"remote_sha256": "af2aa9fdb207c86c4ff91a75a33e84734e3995f37fd316ae394bfd231f57a674",
|
||||
"match": true
|
||||
}
|
||||
],
|
||||
"status": "PASS"
|
||||
}
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"unix": 1794782533.6830049,
|
||||
"worker_cpu_seconds": 0.165208997,
|
||||
"wall_seconds": 0.16521130001638085,
|
||||
"source_scores_sha256": "af2aa9fdb207c86c4ff91a75a33e84734e3995f37fd316ae394bfd231f57a674",
|
||||
"original_scores_unchanged": true,
|
||||
"scope": "Read stored scalar scores and summarize only; no model/array/label loading or rescoring",
|
||||
"original_completed_CPU_seconds": 136620.98104499982,
|
||||
"cumulative_with_display_CPU_seconds": 136621.14625399682,
|
||||
"GPU_seconds_added": 0
|
||||
}
|
||||
+28203
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,39 @@
|
||||
# Gre24 v5:已有评分的阈值及配对差展示补充
|
||||
|
||||
按Claude原串1794782386.838149要求,仅从固定 `scores.json` 读取既有逐件标量并汇总。原评分SHA `af2aa9fdb207c86c4ff91a75a33e84734e3995f37fd316ae394bfd231f57a674` 前后不变,原summary、预测、模型不改写;无新训练、预测、评分、bootstrap或标签读取。三分母min/median/max原表见已发布 `docs/gre24-v5-reproduction-results-2026-11-16.md`(9bf495a)。
|
||||
|
||||
## RRMSE严格小于0.1的比例
|
||||
|
||||
每格为通过件数/该split有效评分件数及比例;614成功/10准备超时的固定排除不变,五split分别有效307/308/306/305/308,缺5/4/6/7/4。分母是各split有效评分件数,不是312或节点数。五split比例等权平均,不先池化各split样本;无undefined评分。全部数值均来自原标量,严格小于0.1,不是小于等于。
|
||||
|
||||
| 分母 | 模型 | split0 | split1 | split2 | split3 | split4 | 五split比例平均 |
|
||||
| --- | --- | --- | --- | --- | --- | --- | ---: |
|
||||
| RMS(y) | naive | 174/307 (56.6775%) | 190/308 (61.6883%) | 173/306 (56.5359%) | 180/305 (59.0164%) | 181/308 (58.7662%) | 58.536882% |
|
||||
| RMS(y) | GBRT | 238/307 (77.5244%) | 235/308 (76.2987%) | 226/306 (73.8562%) | 235/305 (77.0492%) | 232/308 (75.3247%) | 76.010639% |
|
||||
| RMS(y) | FNN | 222/307 (72.3127%) | 220/308 (71.4286%) | 214/306 (69.9346%) | 207/305 (67.8689%) | 231/308 (75.0000%) | 71.308954% |
|
||||
| max(y) | naive | 272/307 (88.5993%) | 272/308 (88.3117%) | 261/306 (85.2941%) | 267/305 (87.5410%) | 262/308 (85.0649%) | 86.962215% |
|
||||
| max(y) | GBRT | 293/307 (95.4397%) | 291/308 (94.4805%) | 291/306 (95.0980%) | 292/305 (95.7377%) | 295/308 (95.7792%) | 95.307045% |
|
||||
| max(y) | FNN | 288/307 (93.8111%) | 287/308 (93.1818%) | 285/306 (93.1373%) | 281/305 (92.1311%) | 286/308 (92.8571%) | 93.023688% |
|
||||
| mean(y) | naive | 145/307 (47.2313%) | 158/308 (51.2987%) | 150/306 (49.0196%) | 157/305 (51.4754%) | 157/308 (50.9740%) | 49.999803% |
|
||||
| mean(y) | GBRT | 207/307 (67.4267%) | 206/308 (66.8831%) | 203/306 (66.3399%) | 209/305 (68.5246%) | 210/308 (68.1818%) | 67.471221% |
|
||||
| mean(y) | FNN | 193/307 (62.8664%) | 197/308 (63.9610%) | 187/306 (61.1111%) | 170/305 (55.7377%) | 200/308 (64.9351%) | 61.722274% |
|
||||
|
||||
## 同件配对的naive与GBRT差
|
||||
|
||||
每个split按相同case配对,先算 **Δ = RRMSE_naive − RRMSE_GBRT**,再对该split的Δ取中位数。正值表示GBRT误差较低;这不是两个模型各自中位数相减。五split配对数依次307/308/306/305/308,原失败排除不补替;逐对原数与Δ也在补充JSON内。数值为比例,非百分比。
|
||||
|
||||
| 分母 | split0 median(Δ) | split1 median(Δ) | split2 median(Δ) | split3 median(Δ) | split4 median(Δ) |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| RMS(y) | 0.018514102312 | 0.017794192390 | 0.019783545436 | 0.017345627252 | 0.020289906179 |
|
||||
| max(y) | 0.011086873306 | 0.010534568802 | 0.011531321491 | 0.010271705431 | 0.011972139328 |
|
||||
| mean(y) | 0.021249401762 | 0.020732402248 | 0.023199691516 | 0.019992931523 | 0.023283182183 |
|
||||
|
||||
## 参考事实、实现选择与未能确认的内容
|
||||
|
||||
论文参考统计:Claude1794782386.838149报告原文300/314约95.5%,并说明原文没有定义RRMSE分母。本补充引用其团队原文核读记录,不声称本轮重新阅读原论文或确认其分母。
|
||||
|
||||
本次实现选择:明确列出三分母,RMS(y)为原固定主指标;仍为v5、t_set时间尺度、fill-only早停FNN及R1补充离散化。max(y)下GBRT阈值比例平均95.307045%,数值接近上述参考;主RMS(y)下为76.010639%。这说明同一批既有误差采用不同分母会产生很不同的展示,不能据接近某参考值反向选择主指标。
|
||||
|
||||
未能确认:原论文实际采用何种分母、v4/v5差异、t_set与V/Q、FNN多输出及离散化的独立影响均不能由这张表识别,10较大件超时排除与近重复跨split亦未消除。相近百分比不是忠实复现认证;本补充没有把总体差距归因给某一个候选因素。
|
||||
|
||||
新增展示汇总实际CPU **0.165208997秒**,GPU新增0。原完成账本136620.981045秒不改写;单列展示费用后累计 **136621.146253997秒/216000秒**。原GPU876.330852390秒/7200秒不变。费用回执 `stored-score-display-receipt-2026-11-16.json`,完整结果 `stored-score-display-supplement-2026-11-16.json` 均位于v5-models-continuation结果根。
|
||||
Reference in new issue
Block a user