Compare commits

...
Author SHA1 Message Date
ClaudeandClaude Opus 5.5 36ffc8b1b2 test: gate V1 side files and V3 legacy receipts; early output rejection
- analyze_jet_cv_continuation: samples are analyzed only if the sibling
  re*_s*.json receipt is the same case with run_status COMPLETE and
  source_recheck UNCHANGED; receipts without run_status (pre-recheck
  runners) are refused unless --legacy-receipts, and labelled
  LEGACY_NO_FINAL_SOURCE_RECHECK. Results carry receipt_gate.
- V3: receipts are schema 2 and need source_recheck UNCHANGED; a missing
  recheck is accepted only for frozen-mode schema 1 receipts whose
  run_case.py hash is one of the pre-recheck runners (9805383, 2cf9853,
  db3d098; hashes checked against git history in a test).
- V1/V3 refuse --output inside the checkout before source checks or CUDA
  (shared source_provenance.reject_output_inside).
- A failing recheck is recorded as RECHECK_FAILED (still invalid); the
  in-run V1 receipt write is tmp+replace.
Negative host tests for each gate. No threshold, solver or evidence edits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:13:59 +00:00
ClaudeandClaude Opus 5.5 df0f586331 test: final source recheck for V1 jet CV and V3 Taylor-Green runs
Both runners now re-derive the recorded source identity after the run
(source_provenance.final_source_check) and only then mark the receipt
complete. A changed source, or a recheck that fails, rewrites the receipt
as invalid and raises:
- V1: in-run receipts carry run_status=INCOMPLETE; finish_receipt sets
  COMPLETE or INVALID_SOURCE_CHANGED (+ source_recheck/source_after).
- V3: completed stays False, invalid=SOURCE_CHANGED_DURING_RUN, no
  screening_pass; the assessor requires source_recheck UNCHANGED (frozen
  mode still accepts receipts from before the field existed).
Interrupted runs keep INCOMPLETE / completed=False. Host tests cover
unchanged, changed and failed rechecks and incomplete runs. No threshold
or solver change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:44:23 +00:00
ClaudeandClaude Opus 5.5 8cae58cdf1 test: fix V4 dirty mock, reject in-checkout output, typed blob errors
- test_dfg_re20: the dirty-checkout test now supplies all three git
  responses (HEAD:src, HEAD, status) and asserts the call order, so it
  reaches the clean-checkout rejection instead of StopIteration.
- run_dfg_re20: --output inside the source checkout is refused before
  provenance, plan writing or any CUDA work (receipts would dirty the
  recorded source); tested with the checkout root and nested paths.
- source_provenance.historical_files: validates all paths first and
  reports an unreadable blob as HistoryUnavailable, not a raw
  CalledProcessError; tested with a failing git show.

No physics thresholds changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:36:28 +00:00
ClaudeandClaude Opus 5.5 34b30c434f test: restore into the same body scene; V3 mock asserts current=False
The checkpoint loader requires matching body count and geometry, so the
round-trip test rebuilds the three-circle scene before loading instead of
an empty simulation (loader unchanged). run_case.main now calls
verify_source(..., current=False); the two dry-run/run mock assertions
follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:33:48 +00:00
ClaudeandClaude Opus 5.5 c33af7a66d test: versioned reader for DFG Re20 result.json (schema 1 and 2)
read_result() returns cases/provenance for the old bare-list receipts and
the new schema-2 object, and rejects unknown schemas. Existing evidence
(e.g. 70f6617) and its digest scripts stay valid for schema 1; queue and
digest scripts must use read_result(...)['cases'] for new receipts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:29:52 +00:00
ClaudeandClaude Opus 5.5 2220cdd157 test: DFG Re20 provenance refuses dirty trees; versioned receipts
Per review of aceea70e: a dirty checkout is refused (HEAD would not
describe the code that runs, and a dirty-to-dirty edit would pass the
end-of-run check). On a clean tree HEAD pins the imported helpers too.
plan.json and result.json carry schema=2; schema-1 receipts (bare list,
e.g. evidence 70f6617) are documented. Adds a dirty-refusal test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:28:25 +00:00
ClaudeandClaude Opus 5.5 8064834c37 test: tighten historical-pin checks per review of 941eb28b
- test_refined_dfg::test_source_pin runs its mismatch negative controls
  first (now including window mode) so missing history cannot skip them.
- test_re100: renamed to test_manifest_matches_its_pinned_commit, plus an
  explicit record that today's src differs from the pin.
- Structural manifest checks run before any history-dependent check.
- Real tamper test: a flipped digest raises via verify_manifest, and a
  path absent at the commit raises ValueError.
- A blobless clone (tree present, blob missing) is reported as unavailable
  history, not as a missing path.
- All history skips go through history_unavailable(): fatal when
  CELERIS_REQUIRE_HISTORY=1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:28:09 +00:00
ClaudeandClaude Opus 5.5 f02355a2b7 test: LES a135252 guard tests use the pinned tree, not today's src
The a135252 manifest is historical qualification: it pins the committed
corrected tree at a135252. The guard is correct; the four tests used the
current working tree as the 'pinned' fixture, so 87c3a31's jets.py change
broke them. Fixtures are now materialized from a135252 with git; the
corrected acceptance, baseline rejection, four tamper rejections and the
reversed-header baseline control are unchanged. A new test records that
the guard refuses today's src where it differs from the pin.

Missing history skips with the reason, or fails when
CELERIS_REQUIRE_HISTORY=1 (release gating) via history_unavailable().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:27:13 +00:00
ClaudeandClaude Opus 5.5 f626cbcda2 test: bitwise checkpoint round trip for a three-cylinder body scene
Save -> load into a fresh Simulation must restore populations (bitwise),
flags, body and step counts, and both copies must stay bitwise identical
after 40 further steps. Jets remain non-checkpointable (documented).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:25:24 +00:00
ClaudeandClaude Opus 5.5 db3d098f43 test: current-checkout mode for the projected-pair TG runner and assessor
run_case --current-checkout records the clean checkout (HEAD, src
fingerprint, harness hashes, pinned build-capture helper) labelled
current_checkout instead of requiring the frozen a135252 tree; receipts
carry numerical_revision=None. assess --current-checkout applies the same
gates to such receipts. Frozen mode is unchanged and now rejects any
receipt with source_mode, so the two never cross-qualify. A missing a135252
object is reported as unavailable history instead of a source diff.
check_projected_pairs.py is untouched, so its pinned hash and existing
receipts keep their meaning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:24:44 +00:00
ClaudeandClaude Opus 5.5 aceea70e5f test: record checkout provenance in DFG Re20 receipts
plan.json and result.json now carry HEAD, the dirty flag, the src tree
hash and the runner hash, labelled current_checkout; result.json is
refused if the checkout changed during the run. Numerics, sampling and
the frozen steady rule are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:16:21 +00:00
ClaudeandClaude Opus 5.5 941eb28b79 test: check pinned source manifests against their own commits
Historical-evidence pins (dfg_re100 / jet_circle_cv / refined_dfg manifests)
now compare each recorded digest with the file at the recorded commit, read
with git, instead of with today's working tree. Missing history (no git,
shallow clone, installed copy) raises HistoryUnavailable and the test is
reported as skipped with that reason, never passed. Receipt/plan logic tests
take the selected manifest directly, so their assertions run again; the
pinned modes' mismatch rejections stay as negative controls.

V1 (run_jet_circle_cv) keeps its pinned mode and gains --current-checkout:
it records HEAD, the src tree and per-file hashes labelled as current
checkout, never as the frozen qualification, and requires a clean tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 18:15:50 +00:00
ClaudeandCodex 750e8a483e test: DFG Re20 steady rule as agreed (10 % range, 2 % slope, window-mean verdict)
Supersedes 71221837. Samples every t* 0.5 (interval-mean forces), trailing
t* 50..60 window (21 samples); range <= 10 % and |slope*window| <= 2 % of
each official band width. Verdict on the window mean with the distance to
the nearest band edge. Our engineering rule, frozen before any run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 869d60d78e)
2026-10-20 21:59:15 +08:00
ClaudeandCodex 6a03f463bc test: DFG Re20 steady rule per review (1 % band, range and slope, 0.5 s samples)
Samples every 0.5 s at synchronized root steps; trailing 5 s window (11
samples); range and |slope*window| each at most 1 % of the official band
width. States the fluid-on-cylinder force sign, reports cd_positive,
defaults to the refined case and uses the review's label.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 71221837f9)
2026-10-20 21:59:15 +08:00
ClaudeandCodex 46dc6ba086 test: add DFG 2D-1 Re20 runner against official reference intervals
Reuses the pinned Re100 runner and helpers unchanged; sets only the
lattice viscosity for Re 20. Reports Cd, Cl and p_diff (dp * 0.04 for
U = 0.2 m/s), a steady rule frozen before any run (window ranges at most
a tenth of each official interval width) and no interval verdict when
not steady. Declares the pressure-NEQ outlet and Zou-He inlet deviations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 9b8ff90609)
2026-10-20 21:59:15 +08:00
ClaudeandCodex e1efc18274 test: follow the recursive _advance in three-level feed checks
baba5e0 replaced _advance_child with _advance(index, parent_buffer) driven
from _hop; the two feed-order tests still patched the removed method.
Patch _advance and record only index 2 (L1 hop feeding L2); assertions,
expected order and the stale-feed negative control are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 3f47436073)
2026-10-20 21:57:57 +08:00
ClaudeandClaude Opus 5.5 9660894a1b docs: checkpoint save/load scope and read_surface_exchange wording
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 13:38:30 +00:00
ClaudeandClaude Opus 5.5 ba3772e3b8 docs: record cut-link ownership and read_surface_exchange changes from 87c3a31
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 13:29:44 +00:00
CodexandFrank14f b4692e6ec4 test: select single-case resolution and compact refinement 2026-10-19 23:55:26 +08:00
CodexandFrank14f 5731c5171b test: add optional public macro CV observations 2026-10-19 22:43:22 +08:00
CodexandFrank14f 05de02ee60 test: parameterize matched public MRT Jet diagnostics 2026-10-19 22:43:22 +08:00
CodexandFrank14f 87c3a3165e fix: validate passive boundary ownership and expose Jet exchange 2026-10-19 22:43:22 +08:00
26 changed files with 1995 additions and 164 deletions
+35
View File
@@ -0,0 +1,35 @@
# Compatibility and API changes
Changes that can make a previously accepted setup or call behave differently.
Each entry names the commit, the affected entry points and the observable result.
## 87c3a31: cut-link ownership is validated for every body
`validate_cut_link_ownership` (`body/preprocess/flag_overlay.py`) now runs for
passive bodies as well as Jet/aperture bodies. Every cut link must have an
active fluid receiver (FLUID set, SOLID clear), and each `(receiver, direction)`
population may have only one writer.
Rejected with `ValueError`, before flags, populations or device buffers change:
1. Overlapping or touching passive solids whose cut links write the same
population ("Boundary requires unique cut-link writers").
2. A body whose cut-link receiver is a non-fluid cell, e.g. a channel wall row
or the inlet/outlet column ("Cut links must have active fluid receivers").
Both were previously accepted silently for passive bodies; the interpolated
boundary then had undefined results. The rules apply to the cut links of all
bodies together, so a passive body overlapping a Jet body is rejected as well.
The check runs in `Simulation.initialize`, runtime body sync (`sync_bodies`),
checkpoint save and checkpoint load (`common/checkpoint.py`). Load validates
before restoring anything, so a checkpoint whose saved geometry violates either
rule no longer restores; rebuild the geometry with separated bodies.
## 87c3a31: `Simulation.read_surface_exchange` accepts Jet-driven bodies
`read_surface_exchange(id)` previously required `SurfaceActuation` on the body.
It now also accepts bodies that own Jet aperture links, including Jets with zero
or negative amplitude (the aperture test is geometric). Bodies with neither
`SurfaceActuation` nor aperture links still raise
`ValueError("Body has no surface actuation")`. The value is unchanged: the latest
step's wall/aperture exchange, not total nozzle force.
+54
View File
@@ -0,0 +1,54 @@
# DFG 2D-1 (Re 20) against the official reference intervals
`tests/validation/run_dfg_re20.py` runs the steady DFG 2D-1 case on the refined
channel root (D40 root, D80 wide fine patch) and on the uniform D80 grid. Geometry,
inflow, initial state, layout and observation are those of the pinned Re100 runner
(`run_refined_dfg.py`, `re100_helpers.py`, both unchanged); only the lattice
viscosity is set for Re 20 (root tau 0.56, fine 0.62, uniform 0.62, lattice speed 0.01).
Label: "compared against the official reference bands", not a full benchmark
certification: the outlet is pressure-NEQ (`neq_extrap`), not do-nothing, and the
inlet is local Zou-He.
## Quantities
- Cd = 2 Fx / (rho U^2 D), Cl = 2 Fy / (rho U^2 D), interval means over each sample.
- p_diff = p(0.15, 0.2) - p(0.25, 0.2), linear extrapolation to the exact surface
points (`re100_helpers.pressure_probes`). The helper returns dp / (rho U^2) in
scaled units; p_diff = dp * 0.04 for U = 0.2 m/s, rho = 1. Both are reported.
- Official intervals (Schaefer & Turek 1996): Cd [5.57, 5.59], Cl [0.0104, 0.0110],
p_diff [0.1172, 0.1176].
## Bounds and steady rule (frozen before any run)
- End time t* = 60 (30 s physical): 240,000 refined root steps, 480,000 uniform steps.
- One sample every t* = 0.5 (0.25 s) at the end of a synchronized root step; Cd and
Cl are interval means. Trailing window t* 50..60 inclusive (21 samples).
- Steady rule (our engineering choice, not official): in the window, for each of Cd,
Cl, p_diff, max - min <= 10 % and |least-squares slope x window| <= 2 % of its
official interval width (range 2e-3 / 6e-5 / 4e-5; slope 4e-4 / 1.2e-5 / 8e-6).
Absolute limits, so the small Cl is never a divisor. Never relaxed after a run.
- The verdict uses the window mean; the distance to the nearest interval edge is
reported for each quantity (negative outside).
- Force sign: fluid on cylinder, x along the inflow, y up from the bottom wall; Cd > 0
and the official Cl > 0 (`cd_positive` is reported).
- Not steady at t* = 60: status `NOT_STEADY`, no interval verdict. Health limits
(Mach <= 0.2, rho in [0.5, 1.5]) stop the run with `UNHEALTHY`.
- Steady: `WITHIN_INTERVALS` or `OUTSIDE_INTERVALS`, with the per-quantity verdict.
## Run
```
python tests/validation/run_dfg_re20.py --output <new-dir> --plan
CUDA_VISIBLE_DEVICES=<n> python tests/validation/run_dfg_re20.py --case both \
--collision SRT --output <new-dir> --max-wall-seconds <cap>
```
The uniform D80 control separates method error from refinement error. `--plan` writes `plan.json` only. A GPU run writes `plan.json` and `result.json`
(per case: lattice parameters, every sample, window ranges, status, verdict).
Receipts: `plan.json` and `result.json` carry `schema` 2 and `provenance` (HEAD, src tree,
runner hash; clean checkout required, refused if it changes during the run). Receipts
written before this (schema 1, e.g. evidence 70f6617) are a bare list of cases. Read both
with `run_dfg_re20.read_result(path)` (`["cases"][0]` replaces the old `[0]`); existing
evidence scripts stay valid for schema-1 receipts and need this reader for new ones.
Host contract tests: `tests/unit/test_dfg_re20.py`.
+59
View File
@@ -56,3 +56,62 @@ Root must independently review and admit any execution. An admitted paired pilot
the existing watchdog, device selection, `--gpu-gates-approved`, positive
`--max-wall-seconds` and a new output directory, adding only `--exchange-window` to the
selected collision's pilot command. No GPU or remote action was performed for this patch.
## Public Jet diagnostic option
`--collision MRT --exchange-window --jet-scenario zero|fixed|reverse` selects
unshifted FP32, no LES, constant transfer on both existing public backends. Default
`none` and the historical manifests remain unchanged. `zero` registers both Jets at
zero amplitude and retains every existing DFG verdict, including pressure. `fixed`
uses outward radial parabolic sectors at pi/2 and 3pi/2, half-width pi/12, peaks
(+.001,-.001). Both use the existing D40/D80 geometry and U=.01. Sector origins are
(80.5,81) in root edge coordinates and (160,160.5) in uniform node coordinates;
radial extent is [0,1.005R]. `reverse` keeps those peaks through t*=300, updates both
public setters at that completed boundary, and continues to 305 with opposite signs.
Fixed diagnostic windows are 250–260, 260–270, 270–280, 280–290 and 290–300; reversal
adds 300–305. Histories retain exact interval endpoints and root/uniform lattice
impulses. The refined window uses total exchange; uniform uses public `read_force`
batch averages. Driven windows report mean/min/max/RMS Cd/Cl without applying DFG
reference or periodicity gates. Exit zero means healthy execution, not qualification;
`matched_case_pass` remains false. No independent CV or stress-load claim is made.
This opt-in records current source/helper hashes with the existing manifest format,
checkout revision/dirty state and runner hash. Pilot/full must match that snapshot
and the exact scenario. It does not relabel current source as historical evidence.
Use the existing bounded plan/pilot/full commands with the three option flags above;
full requires the matching completed t*=5 paired pilot. `step_seconds` includes force
readback; observation/overhead and total wall time are separate, not kernel throughput.
Add `--observe-cv` to record independent scalar control-volume samples from the
already downloaded public macros. The pilot exercises this path at t*=.02..5;
full records 501 endpoints at 290..300. Each `cv_history.jsonl` row contains actual
root time, preceding interval start/duration/impulse, and two inventories/outward
fluxes ordered [mass,Px,Py]. Exclude the first row's preceding impulse: that row
supplies the initial inventory. Uniform native impulse converts by .5² to root
units; refined ledger impulse is already in root units. Explicit native-aligned
contours differ by .25 root cells in x and are recorded in the plan. Cropping
retains the two-cell face-gradient stencils. No fields or per-link arrays are saved.
`cv_observation_seconds` is the CV portion of observation/overhead time. Helper hash
and the observation flag must match pilot/full. Time integration, temporal
coarsening and any physical assessment remain separate offline work.
## Bounded resolution and placement controls
Defaults remain `--case both --uniform-diameter 80 --placement wide`. Single-case
runs use `--case uniform` or `--case refined` and never report a matched-case pass.
Their own reference/periodicity or driven-diagnostic results remain available.
A single zero-Jet full run exits successfully only when its periodicity and the
unchanged complete DFG reference verdict pass; driven completion is not qualification.
The D160 option requires `--case uniform --uniform-diameter 160` and an explicit
public Jet scenario. U and Jet speeds stay .01 and ±.001; viscosity is .016,
grid 3521×658, center (320,320.5), radius80. CV uses root spacing/time factor .25,
origin (.375,.75), impulse factor .0625, root viscosity .004, and x-face shift .125.
`--case refined --placement compact` selects root bounds (48,48,128,128), fine
168×168 and origin (46,46), with the same D40/fineD80 body/Jet resolution. Its CV
contours are (52,52,112,112) and (50,50,120,120); complete face-gradient stencils
are clear of both solids and the transfer footprint. Compact is refined-only.
CV metadata gives each grid's spacing/origin/contours explicitly. Case, uniform
diameter and placement must match the newly completed pilot in every receipt.
+2 -5
View File
@@ -216,11 +216,8 @@ def surface_link_velocity(objects, cfg, flags, fluid_idx, dir, body_id, rx, ry,
enabled = [obj for obj in objects if obj.surface_actuation is not None]
if not enabled and not jets:
return None, None
from ..lbm.descriptors import FLUID, SOLID
if np.any((flags[fluid_idx] & FLUID) == 0) or np.any(flags[fluid_idx] & SOLID):
raise ValueError("Surface cut links must have active fluid receivers")
if len(np.unique(fluid_idx.astype(np.uint64)*cfg.nq+dir)) != len(dir):
raise ValueError("Surface boundary requires unique cut-link writers")
from .preprocess.flag_overlay import validate_cut_link_ownership
validate_cut_link_ownership(flags, fluid_idx, dir, cfg.nq)
surface_velocity = np.zeros((len(dir), 2), np.float32)
surface_aperture = np.zeros(len(dir), bool)
for obj in enabled:
+17 -12
View File
@@ -27,7 +27,7 @@ import pycuda.driver as cuda
from .objects import SimObject
from .registry import BodyRegistry
from .preprocess.flag_overlay import merge_flag_masks
from .preprocess.flag_overlay import merge_flag_masks, validate_cut_link_ownership
from ..cuda.compiler_v2 import obs_layout
from ..cuda.obs_layout import ObsLayout
from ..lbm.descriptors import (
@@ -369,9 +369,9 @@ class ObjectManager:
Args:
objects: List of SimObject instances to process.
domain_flags: Optional uint16 flags for donor-cell classification.
domain_flags: Optional uint16 flags for receiver/donor classification.
sensor_cell_counts_out: Int array to fill with per-object sensor
cell counts. If provided, ``fill(0)`` is called first.
cell counts after the full geometry has passed validation.
hits_out: List that receives one FP64 (n, 2) array of true hit
points row-aligned with the cut-link columns (for Jets).
@@ -394,8 +394,8 @@ class ObjectManager:
active_fluid = ((domain_flags & FLUID) != 0) & ((domain_flags & SOLID) == 0)
force_owners: set[int] = set()
if sensor_cell_counts_out is not None:
sensor_cell_counts_out.fill(0)
sensor_counts = (np.zeros_like(sensor_cell_counts_out)
if sensor_cell_counts_out is not None else None)
ez = None
if self.cfg and self.cfg.is_d3q19:
@@ -430,9 +430,9 @@ class ObjectManager:
if len(cells) > 0:
s_cells.append(cells)
s_ids.append(ids)
if (sensor_cell_counts_out is not None
and 0 <= obj.obj_id < sensor_cell_counts_out.size):
sensor_cell_counts_out[obj.obj_id] = int(len(cells))
if (sensor_counts is not None
and 0 <= obj.obj_id < sensor_counts.size):
sensor_counts[obj.obj_id] = int(len(cells))
if obj.is_force_region and hasattr(obj, 'get_sensor_list'):
cells, ids = obj.get_sensor_list(self.nx, self.ny, self.nz)
keep = active_fluid[cells]
@@ -461,8 +461,9 @@ class ObjectManager:
r_fr_ids = np.concatenate(fr_ids) if fr_ids else np.zeros(0, dtype=np.int32)
r_hit = np.concatenate(cl_hit) if cl_hit else np.zeros((0, 2))
if hits_out is not None:
hits_out.append(r_hit)
# Passive walls need the same ownership guarantees as driven walls.
# Overlapping solids are not a supported union-boundary operation.
validate_cut_link_ownership(domain_flags, r_idx, r_dir, self.nq)
if self.jets or any(obj.surface_actuation is not None for obj in objects):
# Reject unresolved apertures/ownership before sync or restart mutates state.
from ..lbm.curved_links import CurvedLinkSoA
@@ -470,6 +471,11 @@ class ObjectManager:
links.assign_host(r_idx, r_dir, r_q, r_bid, r_rx, r_ry, r_rz, r_fallback)
links.assign_surface(objects, self.cfg, domain_flags, jets=self.jets, hits=r_hit)
if sensor_cell_counts_out is not None:
sensor_cell_counts_out[:] = sensor_counts
if hits_out is not None:
hits_out.append(r_hit)
return (
r_idx, r_dir, r_q, r_bid,
r_rx, r_ry, r_rz, r_fallback,
@@ -487,8 +493,6 @@ class ObjectManager:
the caller has already prepared and uploaded the final flag field
for this geometry, such as during initialization.
"""
self._telemetry_field = field
next_flags = self.build_flags(field.build_channel_flags()) if rebuild_flags else field.flag
hits = []
@@ -499,6 +503,7 @@ class ObjectManager:
fr_cells, fr_obj_id,
) = self.build_compact_lists(domain_flags=next_flags, hits_out=hits)
self._telemetry_field = field
if rebuild_flags:
field.flag = next_flags
field.upload_flags()
+10 -1
View File
@@ -1,6 +1,6 @@
# CelerisLab/body/preprocess/flag_overlay.py
"""
Flag overlay: merge per-object flag masks onto a clean domain base.
Flag overlay and cut-link ownership checks against the merged domain.
"""
from __future__ import annotations
@@ -12,6 +12,15 @@ import numpy as np
from ...lbm.descriptors import FLUID, SOLID, SENSOR_FLAG, FRC_REGION
def validate_cut_link_ownership(flags, fluid_idx, directions, nq):
"""Reject solid receivers and multiple writers to one population."""
if np.any((flags[fluid_idx] & FLUID) == 0) or np.any(flags[fluid_idx] & SOLID):
raise ValueError("Cut links must have active fluid receivers")
writers = fluid_idx.astype(np.uint64) * nq + directions
if len(np.unique(writers)) != len(directions):
raise ValueError("Boundary requires unique cut-link writers")
def merge_flag_masks(
base: np.ndarray,
masks: List[np.ndarray],
+17 -8
View File
@@ -336,13 +336,19 @@ class Simulation:
def read_surface_exchange(self, id: int):
"""Latest step's wall/aperture (Fx,Fy,Tz) exchange, not total nozzle force.
Supports SurfaceActuation and Jet-driven bodies, including zero-amplitude
apertures. Passive bodies without an aperture are rejected.
Unlike read_body this is not accumulated/time-averaged. It resets after
geometry sync or checkpoint load; advance before interpreting it.
"""
if self.bodies.get(id).surface_actuation is None:
obj = self.bodies.get(id)
curved = self.field.curved
has_aperture = (curved.aperture is not None
and np.any(curved.aperture & (curved.body_id == id)))
if obj.surface_actuation is None and not has_aperture:
raise ValueError("Body has no surface actuation")
self.stream.synchronize()
return self.field.curved.surface_exchange(id)
return curved.surface_exchange(id)
@context_bound
def read_body(self, id: int, *,
@@ -451,18 +457,21 @@ class Simulation:
@context_bound
def initialize(self):
"""Initialize flow field and sync objects to GPU."""
# Recompile if objects were added after construction.
self._assert_allocation_contract()
# Validate the complete geometry before flags, populations or modules
# change. Compact-list construction publishes no outputs on failure.
next_flags = self.bodies.build_flags(self.field.build_channel_flags())
if self.bodies.jets and self.bodies.count == 0:
raise ValueError("Jets need bodies with cut links")
self.bodies.build_compact_lists_for(self.bodies.objects, next_flags)
# Recompile if objects were added after construction.
if self._read_generated_n_objects() != self.bodies.count:
self.recompile()
self.field.flag = self.field.build_channel_flags()
if self.bodies.count > 0:
self.field.flag = self.bodies.build_flags(self.field.flag)
self.field.flag = next_flags
self.field.upload_flags()
if self.bodies.jets and self.bodies.count == 0:
raise ValueError("Jets need bodies with cut links")
self.stepper.initialize()
if self.bodies.count > 0:
self.bodies.sync_to_gpu(self.field, rebuild_flags=False)
@@ -0,0 +1,66 @@
"""Checkpoint save -> load round trip for a scene with passive bodies. Requires GPU.
A fresh Simulation with the same scene, restored from the checkpoint must hold the same populations,
flags, body count and step count, and must stay bitwise identical to the original
after both continue for the same number of steps. Checkpoints with Jets are
refused (documented limitation) and are not covered here.
"""
import os
import tempfile
import unittest
import numpy as np
from ._cases import make_sim
# Three separated cylinders (pinball-like); cut-link receivers stay in bulk fluid.
CIRCLES = (((40, 32), 6), ((56, 22), 6), ((56, 42), 6))
def scene():
sim = make_sim()
sim.initialize()
for center, radius in CIRCLES:
sim.add_body("circle", center=center, radius=radius)
sim.sync_bodies()
return sim
class CheckpointBodiesGPU(unittest.TestCase):
def test_round_trip_is_bitwise_and_continues_identically(self):
sim = scene()
sim.run(60)
with tempfile.NamedTemporaryFile(suffix=".h5", delete=False) as f:
path = f.name
restored = None
try:
saved = sim.save_checkpoint(path)
# The loader requires the same body count and geometry: rebuild the scene.
restored = scene()
restored.load_checkpoint(saved)
self.assertEqual(restored.bodies.count, len(CIRCLES))
self.assertEqual(restored.stepper.step_count, sim.stepper.step_count)
np.testing.assert_array_equal(restored.get_flags(), sim.get_flags())
a, b = sim.get_ddf(), restored.get_ddf()
self.assertEqual(a.dtype, b.dtype)
self.assertEqual(a.tobytes(), b.tobytes())
sim.run(40)
restored.run(40)
self.assertEqual(restored.stepper.step_count, sim.stepper.step_count)
a, b = sim.get_ddf(), restored.get_ddf()
self.assertTrue(np.isfinite(a).all())
self.assertEqual(a.tobytes(), b.tobytes())
for i in range(len(CIRCLES)):
self.assertTrue(np.all(np.isfinite(restored.read_force(i))))
finally:
if restored is not None:
restored.close()
os.unlink(path)
sim.close()
if __name__ == "__main__":
unittest.main()
+133
View File
@@ -0,0 +1,133 @@
"""CPU-only contract tests for run_dfg_re20. Not CUDA or DFG acceptance evidence."""
import sys
from pathlib import Path
import unittest
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "validation"))
import run_dfg_re20 as v # noqa: E402
import run_refined_dfg as rr # noqa: E402
h, _ = rr.helpers(rr.DEFAULT_SOURCE)
def rows(values, tstar_end=v.TSTAR_MAX):
n = int(round(tstar_end / v.SAMPLE_TSTAR))
return [dict(tstar=(i + 1) * v.SAMPLE_TSTAR, **values(i)) for i in range(n)]
class Re20Contract(unittest.TestCase):
def test_lattice_parameters_and_bounds(self):
root, uniform = v.lattice_parameters(40), v.lattice_parameters(80)
self.assertAlmostEqual(root["tau"], 0.56, places=12)
self.assertAlmostEqual(uniform["tau"], 0.62, places=12)
self.assertEqual((root["steps"], root["interval"]), (240000, 2000))
self.assertEqual((uniform["steps"], uniform["interval"]), (480000, 4000))
self.assertAlmostEqual(v.plan()["refined_fine_tau"], 0.62, places=12)
def test_config_only_changes_viscosity(self):
base = v._re100_config(h, 40, "SRT", "none")
cfg, body, p = v.config(h, 40, "SRT")
self.assertAlmostEqual(cfg["physics"]["viscosity"], 0.01 * 40 / 20, places=15)
self.assertEqual((p["re"], p["viscosity"]), (20.0, cfg["physics"]["viscosity"]))
base[0]["physics"]["viscosity"] = cfg["physics"]["viscosity"]
self.assertEqual(base[0], cfg)
self.assertEqual(base[1], body)
with self.assertRaisesRegex(ValueError, "no Jets"):
v.config(h, 40, "MRT", "zero")
def test_units(self):
s = v.sample([2.0, -0.5], 4, 40, 3.0)
den = rr.SPEED**2 * 40 / 2
self.assertAlmostEqual(s["cd"], 2.0 / 4 / den)
self.assertAlmostEqual(s["cl"], -0.5 / 4 / den)
self.assertAlmostEqual(s["p_diff"], 3.0 * 0.04)
self.assertEqual(s["dp_scaled"], 3.0)
def test_steady_and_verdict(self):
inside = dict(cd=5.58, cl=0.0106, p_diff=0.1174)
out = v.summarize(rows(lambda i: inside))
self.assertEqual((out["status"], out["steady"], out["window_samples"]),
("WITHIN_INTERVALS", True, 21))
self.assertTrue(out["cd_positive"])
self.assertAlmostEqual(out["edge_distance"]["cl"], 0.0002)
self.assertTrue(all(out["verdict"].values()))
low = v.summarize(rows(lambda i: dict(inside, cl=0.0100)))
self.assertEqual(low["status"], "OUTSIDE_INTERVALS")
self.assertFalse(low["verdict"]["cl"])
def test_slow_drift_gives_no_verdict(self):
# Range 2e-5 passes (limit 6e-5); slope * window 2e-5 exceeds 1.2e-5.
out = v.summarize(rows(lambda i: dict(cd=5.58, cl=0.0106 + 1e-6 * i, p_diff=0.1174)))
self.assertEqual((out["status"], out["verdict"]), ("NOT_STEADY", None))
self.assertLessEqual(out["ranges"]["cl"], out["range_limits"]["cl"])
self.assertGreater(out["drifts"]["cl"], out["slope_limits"]["cl"])
def test_noise_range(self):
quiet = v.summarize(rows(lambda i: dict(cd=5.58, cl=0.0106 + 2e-5 * (i % 2), p_diff=0.1174)))
self.assertTrue(quiet["steady"]) # within the range limit, no trend
noisy = v.summarize(rows(lambda i: dict(cd=5.58, cl=0.0106 + 1e-4 * (i % 2), p_diff=0.1174)))
self.assertEqual(noisy["status"], "NOT_STEADY")
self.assertGreater(noisy["ranges"]["cl"], noisy["range_limits"]["cl"])
def test_incomplete_run_gives_no_verdict(self):
out = v.summarize(rows(lambda i: dict(cd=5.58, cl=0.0106, p_diff=0.1174), tstar_end=30))
self.assertEqual((out["status"], out["verdict"]), ("INCOMPLETE", None))
class Re20Provenance(unittest.TestCase):
def test_records_current_checkout(self):
record = v.provenance(rr.DEFAULT_SOURCE)
self.assertEqual(record["source_mode"], "current_checkout")
self.assertEqual(len(record["checkout_revision"]), 40)
self.assertEqual(len(record["src_tree"]), 40)
self.assertEqual(len(record["runner_sha256"]), 64)
self.assertFalse(record["checkout_dirty"])
self.assertEqual(v.RESULT_SCHEMA, 2)
def test_reader_accepts_both_schemas_and_rejects_unknown(self):
import json
import tempfile
from pathlib import Path
case = dict(case="refined", status="NOT_STEADY", verdict=None, rows=[])
with tempfile.TemporaryDirectory() as d:
old, new, bad = Path(d, "old.json"), Path(d, "new.json"), Path(d, "bad.json")
old.write_text(json.dumps([case]))
new.write_text(json.dumps(dict(schema=2, provenance=dict(src_tree="a" * 40),
cases=[case])))
bad.write_text(json.dumps(dict(schema=3, cases=[case])))
self.assertEqual(v.read_result(old), dict(schema=1, provenance=None, cases=[case]))
self.assertEqual(v.read_result(new)["cases"][0], case)
self.assertEqual(v.read_result(new)["provenance"]["src_tree"], "a" * 40)
with self.assertRaisesRegex(ValueError, "unknown DFG Re20 result schema"):
v.read_result(bad)
def test_dirty_checkout_is_refused(self):
from unittest.mock import patch
# Three git calls in order: HEAD:src, HEAD, status --porcelain.
with patch("subprocess.check_output",
side_effect=["e" * 40 + "\n", "f" * 40 + "\n", " M src/x.py\n"]) as git:
with self.assertRaisesRegex(RuntimeError, "clean checkout"):
v.provenance(rr.DEFAULT_SOURCE)
self.assertEqual(git.call_count, 3)
self.assertEqual([c.args[0][-1] for c in git.call_args_list[:2]], ["HEAD:src", "HEAD"])
self.assertIn("--porcelain", git.call_args_list[2].args[0])
def test_output_inside_checkout_is_refused_before_any_work(self):
import tempfile
from pathlib import Path
from unittest.mock import patch
source = rr.DEFAULT_SOURCE.resolve()
for output in (source, source / "re20-out", source / "src" / "x"):
argv = ["run_dfg_re20.py", "--plan", "--source", str(source), "--output", str(output)]
with patch("sys.argv", argv), \
patch.object(v, "provenance", side_effect=AssertionError("work started")):
with self.assertRaisesRegex(SystemExit, "outside the source checkout"):
v.main()
self.assertFalse((source / "re20-out").exists())
with tempfile.TemporaryDirectory() as d:
v.reject_output_inside(Path(d) / "out", source) # outside: accepted
if __name__ == "__main__":
unittest.main()
+108 -1
View File
@@ -16,6 +16,7 @@ from run_jet_circle_cv import (
case,
state,
cv,
cv_from_macros,
aperture_probe,
measured_aperture_flux,
verify,
@@ -28,6 +29,39 @@ def eq(rho, u, v):
class Host(unittest.TestCase):
def test_public_macros_pressure_advection_and_cropped_frames(self):
for h, origin, shape, rect in (
(.5, (0., 0.), (48, 64), (5., 4., 25., 20.)),
(.5, (.25, .5), (48, 64), (5.25, 4., 25.25, 20.)),
(.25, (.375, .75), (96, 128), (5.125, 4., 25.125, 20.)),
):
y, x = np.indices(shape)
x = origin[0] + h * (x + .5)
k, u, v = 1e-5, .02, -.01
rho = 1 + 3 * k * x # p=(rho-1)/3=k*x
ux, uy, fluid = np.full(shape, u), np.full(shape, v), np.ones(shape, bool)
result = cv_from_macros(rho, ux, uy, fluid, origin, rect, .004, h)
expected = 320 * np.array([3*k*u, k + 3*k*u*u, 3*k*u*v])
np.testing.assert_allclose(result[1], expected, rtol=1e-11, atol=1e-13)
np.testing.assert_allclose(result, cv(eq(rho, u, v), fluid, origin, rect, .004, h), atol=1e-12)
x0, y0, x1, y1 = np.rint((np.array(rect)-np.tile(origin, 2))/h).astype(int)
selection = np.s_[y0-2:y1+2, x0-2:x1+2]
cropped_origin = np.asarray(origin) + h*np.array([x0-2, y0-2])
cropped = cv_from_macros(rho[selection], ux[selection], uy[selection],
fluid[selection], cropped_origin, rect, .004, h)
np.testing.assert_array_equal(cropped, result)
def test_public_macros_full_normal_viscous_stress(self):
y, x = np.indices((48, 64))
x = (x + .5) * .5
rho, ux, uy, fluid = np.ones_like(x), 1e-4*x*x, np.zeros_like(x), np.ones(x.shape, bool)
args = (rho, ux, uy, fluid, (0., 0.), (5., 4., 25., 20.))
_, viscous = cv_from_macros(*args, .004)
_, inviscid = cv_from_macros(*args, 0.)
np.testing.assert_allclose(viscous-inviscid, [0, -4*.004*1e-4*320, 0], atol=1e-13)
with self.assertRaisesRegex(ValueError, "matching 2D"):
cv_from_macros(rho, ux, uy[:-1], fluid, (0, 0), (5, 4, 25, 20), .004)
def test_descriptor_and_uniform_closed_flux(self):
from CelerisLab.lbm.descriptors import D2Q9_EX, D2Q9_EY
@@ -132,7 +166,80 @@ class Host(unittest.TestCase):
self.assertEqual(a["steps"] * 4, b["steps"])
def test_source_pin(self):
verify(REPO)
"""Historical integrity: the manifest matches its pinned commit, read with git."""
from run_jet_circle_cv import verify_historical
from source_provenance import HistoryUnavailable, history_unavailable
try:
verify_historical(REPO)
except HistoryUnavailable as error:
history_unavailable(self, error)
def test_pinned_mode_still_rejects_changed_source(self):
"""Negative control: the frozen-source guard refuses a tree that differs from PIN."""
import json
import run_jet_circle_cv as v
manifest = json.loads((v.HERE / "data/jet_circle_cv/source_manifest.json").read_text())
import hashlib
current = {f: hashlib.sha256((REPO / f).read_bytes()).hexdigest()
for f in manifest["files"] if (REPO / f).exists()}
if current == manifest["files"]:
self.skipTest("working tree equals PIN; no changed source to reject")
with self.assertRaisesRegex(RuntimeError, "Source hash mismatch"):
verify(REPO)
def test_current_checkout_provenance(self):
from run_jet_circle_cv import current_checkout
record = current_checkout(REPO)
self.assertEqual(record["source_mode"], "current_checkout")
self.assertIn("not the frozen", record["source_capture"])
self.assertEqual(len(record["src_tree"]), 40)
self.assertTrue(record["src_sha256"])
def test_final_source_recheck_completes_or_invalidates_receipt(self):
import json
import tempfile
import run_jet_circle_cv as v
recorded = dict(source_mode="current_checkout", checkout_revision="a" * 40,
checkout_dirty=False, src_tree="b" * 40)
payload = dict(status="PILOT_DIAGNOSTIC_ONLY", run_status="INCOMPLETE", windows=[1, 2])
with tempfile.TemporaryDirectory() as d:
path = Path(d) / "re100_s1.json"
path.write_text(json.dumps(payload))
done = v.finish_receipt(path, payload, recorded, lambda: dict(recorded))
self.assertEqual(json.loads(path.read_text()), done)
self.assertEqual((done["run_status"], done["source_recheck"]), ("COMPLETE", "UNCHANGED"))
for recheck, state in ((lambda: dict(recorded, checkout_dirty=True), "CHANGED"),
(lambda: v.verify(Path(d)), "RECHECK_FAILED")):
path.write_text(json.dumps(payload))
with self.assertRaisesRegex(RuntimeError, "Source changed during the run"):
v.finish_receipt(path, payload, recorded, recheck)
bad = json.loads(path.read_text())
self.assertEqual((bad["run_status"], bad["source_recheck"]),
("INVALID_SOURCE_CHANGED", state))
self.assertEqual(bad["source_after"] is None, state == "RECHECK_FAILED")
self.assertEqual(bad["windows"], [1, 2])
self.assertFalse(Path(str(path) + ".tmp").exists())
def test_receipts_written_during_the_run_are_incomplete(self):
"""Static check (run_case needs CUDA): the in-run payload is INCOMPLETE and only
main() -> finish_receipt can mark it COMPLETE."""
import inspect
import run_jet_circle_cv as v
self.assertIn('run_status="INCOMPLETE"', inspect.getsource(v.run_case))
self.assertNotIn("COMPLETE\"", inspect.getsource(v.run_case).replace("INCOMPLETE\"", ""))
self.assertIn("finish_receipt(", inspect.getsource(v.main))
def test_output_inside_checkout_is_refused_before_source_or_cuda(self):
from unittest import mock
import run_jet_circle_cv as v
argv = ["run_jet_circle_cv.py", "--repo", str(REPO), "--output", str(REPO / "v1-out")]
with mock.patch("sys.argv", argv), mock.patch.dict("os.environ", CUDA_VISIBLE_DEVICES="0"), \
mock.patch.object(v, "source_identity", side_effect=AssertionError("work started")):
with self.assertRaisesRegex(SystemExit, "outside the source checkout"):
v.main()
self.assertFalse((REPO / "v1-out").exists())
if __name__ == "__main__":
+34
View File
@@ -151,6 +151,8 @@ class Measurements(unittest.TestCase):
(path / "samples_s2.json").write_text(
json.dumps(dict(case=c, sample_file="samples_s2.npz"))
)
(path / f"re{c['re_d']}_s2.json").write_text(json.dumps(
dict(case=c, run_status="COMPLETE", source_recheck="UNCHANGED")))
np.savez(
path / "samples_s2.npz",
t_reference=t,
@@ -163,6 +165,7 @@ class Measurements(unittest.TestCase):
velocity_error=np.zeros(n),
)
result = analyze(path)
self.assertEqual(result["receipt_gate"], "COMPLETE_SOURCE_UNCHANGED")
self.assertEqual(len(result["windows"]), 32)
np.testing.assert_allclose(
result["windows"][0]["normalized_cv_residual"], 0, atol=1e-14
@@ -178,5 +181,36 @@ class Measurements(unittest.TestCase):
np.testing.assert_allclose(integrate(t, y), [112, 8], atol=1e-15)
def test_receipt_gate_refuses_incomplete_invalid_and_unmarked_legacy(self):
import json
from analyze_jet_cv_continuation import receipt_gate
c = case(2, steps=3200)
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp)
receipt = path / f"re{c['re_d']}_s2.json"
with self.assertRaisesRegex(ValueError, "No sibling run receipt"):
receipt_gate(path, c)
for fields, message in (
(dict(run_status="INCOMPLETE"), "INCOMPLETE"),
(dict(run_status="INVALID_SOURCE_CHANGED", source_recheck="CHANGED"), "INVALID"),
(dict(run_status="COMPLETE", source_recheck="RECHECK_FAILED"), "RECHECK_FAILED"),
(dict(run_status="COMPLETE"), "source_recheck=None"),
(dict(), "Legacy run receipt"),
):
receipt.write_text(json.dumps(dict(case=c, **fields)))
with self.assertRaisesRegex(ValueError, message):
receipt_gate(path, c)
receipt.write_text(json.dumps(dict(case=dict(c, scale=1), run_status="COMPLETE",
source_recheck="UNCHANGED")))
with self.assertRaisesRegex(ValueError, "different case"):
receipt_gate(path, c)
receipt.write_text(json.dumps(dict(case=c)))
self.assertEqual(receipt_gate(path, c, legacy=True), "LEGACY_NO_FINAL_SOURCE_RECHECK")
receipt.write_text(json.dumps(dict(case=c, run_status="INCOMPLETE")))
with self.assertRaisesRegex(ValueError, "INCOMPLETE"):
receipt_gate(path, c, legacy=True) # legacy opt-in never admits new statuses
if __name__ == "__main__":
unittest.main()
+49
View File
@@ -188,6 +188,55 @@ def jet_case(device, jets):
return field, bodies
def test_public_surface_exchange_for_jet_preserves_aperture_at_zero_and_reverse(device):
from CelerisLab.simulation import Simulation
jet = Jet(Sector(CENTER, 0., .6, 3.), Radial(CENTER, inward=False), "quartic", 2e-3)
field, bodies = jet_case(device, [jet])
sim = SimpleNamespace(bodies=bodies, field=field, _initialized=True,
stream=SimpleNamespace(synchronize=device.synchronize))
cl = field.curved
assert bodies.get(0).surface_actuation is None
aperture, velocity = cl.aperture, cl.surface_velocity.copy()
assert aperture.any() and (~aperture).any()
# Synthetic per-link exchanges test readback/splitting, not kernel physics.
values = np.arange(cl.count * 3, dtype=np.float32).reshape(-1, 3)
device.htod(cl.link_exchange_gpu, values)
for amplitude in (2e-3, 0., -1e-3):
Simulation.set_jet_amplitude(sim, 0, amplitude)
result = Simulation.read_surface_exchange(sim, 0)
assert cl.aperture is aperture
np.testing.assert_array_equal(aperture, cl.jet_id >= 0)
np.testing.assert_allclose(cl.surface_velocity, velocity * (amplitude / 2e-3))
for name, mask in (("wall", ~aperture), ("aperture", aperture)):
np.testing.assert_array_equal(result[name], values[mask].sum(axis=0, dtype=np.float64))
def test_public_surface_exchange_keeps_legacy_and_passive_guards(device):
from CelerisLab.simulation import Simulation
field = make_field(device)
bodies = manager(field, [circle(SurfaceActuation(0., .6, .002), radius=2.)])
bodies.sync_to_gpu(field)
sim = SimpleNamespace(bodies=bodies, field=field,
stream=SimpleNamespace(synchronize=device.synchronize))
for value in Simulation.read_surface_exchange(sim, 0).values():
np.testing.assert_array_equal(value, 0.)
field.curved.free_gpu_columns()
with pytest.raises(RuntimeError, match="diagnostic is allocated"):
Simulation.read_surface_exchange(sim, 0)
# An exchange buffer elsewhere in the scene must not admit a passive body.
for jets in ([], [Jet(Sector(CENTER, 0., .6, 3.), Radial(CENTER))]):
field, bodies = jet_case(device, jets)
passive = bodies.add(circle(center=(2., 3.), radius=.6))
bodies.sync_to_gpu(field)
sim.bodies, sim.field = bodies, field
with pytest.raises(ValueError, match="Body has no surface actuation"):
Simulation.read_surface_exchange(sim, passive)
if not jets:
with pytest.raises(ValueError, match="Body has no surface actuation"):
Simulation.read_surface_exchange(sim, 0)
def test_manager_passes_true_hits_and_reuploads_only_velocity(device):
# CPU memory double only: checks host/device bookkeeping, not GPU numerics.
jet = Jet(Sector(CENTER, 0., .6, 3.), Radial(CENTER, inward=False), "quartic", 2e-3)
+44 -22
View File
@@ -13,8 +13,15 @@ from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "validation"))
from run_uniform_les_flow import source_guard, decode_macros, HERE, HEADER
from source_provenance import (HistoryUnavailable, historical_files, historical_sha256,
historical_tree, history_unavailable)
ROOT = HERE.parents[1]
A135252 = "a135252ae78b47b57d7e77d19001700b3bedac8b"
def a135252_pins():
return json.loads((HERE / "data/les_flow/source_pins_a135252.json").read_text())
class GuardTests(unittest.TestCase):
@@ -41,20 +48,35 @@ class GuardTests(unittest.TestCase):
with self.assertRaisesRegex(RuntimeError, "velocity/profile"):
decode_macros(invalid, w, cx, cy)
def test_current_corrected_pins(self):
self.assertEqual(
source_guard(ROOT, "corrected", "a135252")["base_commit"],
"a135252ae78b47b57d7e77d19001700b3bedac8b",
)
def materialize(self, root):
"""The pinned a135252 tree, read with git (the guard's own qualification)."""
try:
historical_files(ROOT, A135252, a135252_pins()["sources"], root)
except HistoryUnavailable as error:
history_unavailable(self, error)
def test_corrected_and_tamper_guard(self):
pins = json.loads((HERE / "data/les_flow/source_pins_a135252.json").read_text())
def test_a135252_pins_qualify_their_own_commit(self):
with TemporaryDirectory() as directory:
root = Path(directory)
for name in pins["sources"]:
dest = root / name
dest.parent.mkdir(parents=True, exist_ok=True)
shutil.copyfile(ROOT / name, dest)
self.materialize(root)
with patch("run_uniform_les_flow.subprocess.check_output",
return_value="isolated-fixture\n"):
self.assertEqual(source_guard(root, "corrected", "a135252")["base_commit"], A135252)
def test_current_tree_is_not_the_a135252_qualification(self):
"""Explicit record: the guard refuses today's src wherever it differs from the pin."""
pins = a135252_pins()["sources"]
changed = sorted(n for n, sha in pins.items() if (ROOT / n).exists()
and hashlib.sha256((ROOT / n).read_bytes()).hexdigest() != sha)
if not changed:
self.skipTest("current src equals the a135252 pin")
with self.assertRaisesRegex(RuntimeError, "Source pin mismatch"):
source_guard(ROOT, "corrected", "a135252")
def test_corrected_and_tamper_guard(self):
with TemporaryDirectory() as directory:
root = Path(directory)
self.materialize(root)
with patch(
"run_uniform_les_flow.subprocess.check_output",
return_value="isolated-fixture\n",
@@ -85,25 +107,25 @@ class GuardTests(unittest.TestCase):
with self.assertRaisesRegex(RuntimeError, "Source pin mismatch"):
source_guard(ROOT, "corrected") # Omission still uses the historical profile.
def test_current_manifest_pins_actual_corrected_tree(self):
pins = json.loads((HERE / "data/les_flow/source_pins_a135252.json").read_text())
tracked = subprocess.check_output(["git", "-C", str(ROOT), "ls-files", "src"], text=True)
self.assertEqual(set(pins["sources"]), set(tracked.splitlines()))
for name, expected in pins["sources"].items():
self.assertEqual(hashlib.sha256((ROOT / name).read_bytes()).hexdigest(), expected)
def test_manifest_pins_the_a135252_corrected_tree(self):
pins = a135252_pins()
self.assertEqual(pins["base_commit"], A135252)
self.assertEqual(pins["sources"][HEADER], pins["candidate_header_sha256"])
self.assertNotEqual(pins["sources"][HEADER], pins["baseline_header_sha256"])
with self.assertRaisesRegex(RuntimeError, "Source pin mismatch"):
source_guard(ROOT, "baseline", "a135252")
try:
names = historical_tree(ROOT, A135252)
actual = historical_sha256(ROOT, A135252, pins["sources"])
except HistoryUnavailable as error:
history_unavailable(self, error)
self.assertEqual(set(pins["sources"]), set(names))
self.assertEqual(actual, pins["sources"])
def test_named_baseline_accepts_only_reversed_header(self):
pins = json.loads((HERE / "data/les_flow/source_pins_a135252.json").read_text())
with TemporaryDirectory() as directory:
root = Path(directory)
for name in pins["sources"]:
dest = root / name
dest.parent.mkdir(parents=True, exist_ok=True)
shutil.copyfile(ROOT / name, dest)
self.materialize(root)
subprocess.run(["git", "apply", "--reverse", str(HERE / "les/data/smagorinsky_normalization.patch")], cwd=root, check=True)
with patch("run_uniform_les_flow.subprocess.check_output", return_value="baseline-fixture\n"):
source_guard(root, "baseline", "a135252")
+21 -3
View File
@@ -62,13 +62,31 @@ def test_periodic_synthetic():
assert not p.assess(rows()[:100])["periodic"]
def test_production_unmodified():
def test_manifest_matches_its_pinned_commit():
"""Historical integrity only: the manifest describes the h.PIN tree (read with git)."""
m = json.loads(
(ROOT / "tests/validation/data/dfg_re100/source_manifest.json").read_text()
)
assert m["commit"] == h.PIN
for name, digest in m["production_sha256"].items():
assert hashlib.sha256((ROOT / name).read_bytes()).hexdigest() == digest, name
import unittest
from source_provenance import HistoryUnavailable, history_unavailable, verify_manifest
try:
verify_manifest(ROOT, m["commit"], m["production_sha256"])
except HistoryUnavailable as error:
history_unavailable(unittest.TestCase(), error)
def test_current_src_is_not_the_pinned_qualification():
"""Explicit record: today's src differs from h.PIN, so its qualification does not carry over."""
m = json.loads(
(ROOT / "tests/validation/data/dfg_re100/source_manifest.json").read_text()
)
changed = sorted(n for n, d in m["production_sha256"].items()
if not (ROOT / n).exists()
or hashlib.sha256((ROOT / n).read_bytes()).hexdigest() != d)
if not changed:
pytest.skip("current src equals the pinned tree")
assert changed # e.g. src/CelerisLab/lbm/channel.py since 47b05bc
def test_reference():
+202 -14
View File
@@ -12,16 +12,33 @@ import run_refined_dfg as r
h, periodic = r.helpers(r.DEFAULT_SOURCE)
def historical_manifest(collision="SRT", observation_mode="last_base_step"):
"""The manifest verify_source would select, without hashing the working tree.
Receipt/plan logic tests use it; source integrity is checked once in test_source_pin.
"""
name = ("source_manifest_window.json" if observation_mode == "window" else
"source_manifest.json" if collision == "SRT" else "source_manifest_mrt.json")
return json.loads((r.HERE / "data/refined_dfg" / name).read_text())
def test_source_pin():
historical = json.loads((r.HERE / "data/refined_dfg/source_manifest.json").read_text())
historical = historical_manifest("SRT")
assert historical["commit"].startswith("cd3a657")
mrt_history = json.loads((r.HERE / "data/refined_dfg/source_manifest_mrt.json").read_text())
mrt_history = historical_manifest("MRT")
assert mrt_history["commit"].startswith("3ea131e")
assert r.verify_source(r.DEFAULT_SOURCE, "MRT", "window")["commit"].startswith("a233c90")
with unittest.TestCase().assertRaisesRegex(RuntimeError, "Source mismatch"):
r.verify_source(r.DEFAULT_SOURCE, "MRT")
with unittest.TestCase().assertRaisesRegex(RuntimeError, "Source mismatch"):
r.verify_source(r.DEFAULT_SOURCE, "SRT")
assert historical_manifest("MRT", "window")["commit"].startswith("a233c90")
# Negative controls first, so missing history can never hide them: every pinned
# mode refuses a working tree that differs from its pin.
for collision, mode in (("MRT", "last_base_step"), ("SRT", "last_base_step"), ("MRT", "window")):
with unittest.TestCase().assertRaisesRegex(RuntimeError, "Source mismatch"):
r.verify_source(r.DEFAULT_SOURCE, collision, mode)
from source_provenance import HistoryUnavailable, history_unavailable, verify_manifest
for manifest in (historical, mrt_history, historical_manifest("MRT", "window")):
try: # historical integrity: each manifest matches the tree of its own commit
verify_manifest(r.DEFAULT_SOURCE, manifest["commit"], manifest["sha256"])
except HistoryUnavailable as error:
history_unavailable(unittest.TestCase(), error)
def test_matched_units_and_planes():
@@ -143,26 +160,30 @@ def test_axis_specific_sampling_phase():
def test_full_requires_completed_pilot():
with unittest.TestCase().assertRaisesRegex(ValueError, "pilot-results"):
r.verify_pilot(None, r.verify_source(r.DEFAULT_SOURCE, "MRT", "window"))
r.verify_pilot(None, historical_manifest("MRT", "window"))
def pilot_fixture(path, observation_mode="window"):
def pilot_fixture(path, observation_mode="window", jet_scenario="none", case_selection="both", uniform_diameter=80, placement="wide"):
"""Synthetic contract fixture only; never a physical result or run receipt."""
manifest = (r.verify_source(r.DEFAULT_SOURCE, "MRT", "window")
manifest = ((historical_manifest("MRT", "window") if jet_scenario == "none" else
r.verify_source(r.DEFAULT_SOURCE, "MRT", "window", jet_scenario))
if observation_mode == "window" else
json.loads((r.HERE / "data/refined_dfg/source_manifest_mrt.json").read_text()))
selection = dict(case_selection=case_selection, uniform_diameter=uniform_diameter, placement=placement)
(path / "plan.json").write_text(
json.dumps(
dict(
source_manifest=manifest,
**selection,
collision="MRT",
jet_scenario=jet_scenario,
observation_mode=observation_mode,
bounds_root_edge=list(r.BOUNDS),
bounds_root_edge=list(r.PLACEMENTS[placement]),
runner_sha256=r.runner_hash(),
)
)
)
for case, diameter in (("uniform", 80), ("refined", 40)):
for case, diameter in r.selected_cases(case_selection, uniform_diameter):
dest = path / case
dest.mkdir()
interval = round(0.02 * diameter / r.SPEED)
@@ -189,7 +210,9 @@ def pilot_fixture(path, observation_mode="window"):
json.dumps(
dict(
case=case,
**selection,
collision="MRT",
jet_scenario=jet_scenario,
observation_mode=observation_mode,
mode="pilot",
status="pilot_complete",
@@ -206,7 +229,9 @@ def pilot_fixture(path, observation_mode="window"):
json.dumps(
dict(
case=case,
**selection,
collision="MRT",
jet_scenario=jet_scenario,
observation_mode=observation_mode,
mode="pilot",
step=interval,
@@ -341,7 +366,7 @@ def test_record_actual_checkout_separately():
actual = checkout_provenance(r.DEFAULT_SOURCE)
assert len(actual["checkout_revision"]) == 40
assert isinstance(actual["checkout_dirty"], bool)
assert r.verify_source(r.DEFAULT_SOURCE, "MRT", "window")["production_origin_commit"].startswith("d930")
assert historical_manifest("MRT", "window")["production_origin_commit"].startswith("d930")
def test_window_plan_preserves_physics():
@@ -351,7 +376,7 @@ def test_window_plan_preserves_physics():
assert baseline.pop("observation_mode") == "last_base_step"
assert window.pop("observation_mode") == "window"
assert baseline == window
assert r.verify_source(r.DEFAULT_SOURCE, collision, "window")["observation_mode"] == "window"
assert historical_manifest(collision, "window")["observation_mode"] == "window"
with unittest.TestCase().assertRaises(ValueError):
r.verify_source(r.DEFAULT_SOURCE, "MRT", "unknown")
@@ -451,6 +476,169 @@ def test_window_contract_rejection():
assert not backend.events
def test_public_jet_configuration_and_geometry():
for diameter, center, radius in ((40, (80.5, 81), 20), (80, (160, 160.5), 40)):
cfg, _, _ = r.config(h, diameter, "MRT", "fixed")
assert not cfg["method"]["ddf_shifting"] and not cfg["method"]["les"]["enabled"]
jets = r.public_jets(center, radius, "fixed")
assert [j.amplitude for j in jets] == [.001, -.001]
assert all(j.field.inward is False and j.profile == "parabolic" for j in jets)
assert all(j.region.r_max == radius * 1.005 for j in jets)
assert all(j.amplitude == 0 for j in r.public_jets(center, radius, "zero"))
cfg, _, _, host = r.host_geometry(r.DEFAULT_SOURCE, h, "MRT", "fixed")
assert not cfg.ddf_shifting and len(host.boundary_cells) > 0
assert r.public_jets((0, 0), 1, "none") == ()
with unittest.TestCase().assertRaises(ValueError):
r.config(h, 40, "SRT", "fixed")
def test_public_jet_reversal_boundary_and_windows():
from types import SimpleNamespace
calls = []
handle = SimpleNamespace(set_jet_amplitude=lambda i, a: calls.append((i, a)))
for diameter in (40, 80):
boundary = round(250 * diameter / r.SPEED)
assert r.jet_interval(handle, "fixed", boundary - 3, 80, diameter) == (3, None)
switch = round(300 * diameter / r.SPEED)
assert r.jet_interval(handle, "reverse", switch - 3, 80, diameter) == (3, None)
assert not calls
n, event = r.jet_interval(handle, "reverse", switch, 80, diameter)
assert n == 80 and event["step"] == switch and event["tstar"] == 300
assert calls == [(0, -.001), (1, .001)]
calls.clear()
assert r.jet_interval(handle, "reverse", switch + 80, 80, diameter) == (80, None)
assert r.jet_interval(handle, "fixed", switch, 80, diameter) == (80, None)
assert not calls
rows = [dict(tstar=t, tstar_force=t - .5, cd=2.0, cl=-.1) for t in range(251, 306)]
windows = r.jet_diagnostics(rows, "reverse")
assert len(windows) == 6 and all(w["complete"] for w in windows)
assert all(w["cd"]["mean"] == 2 and np.isclose(w["cl"]["mean"], -.1) for w in windows)
assert windows[-1]["covered_tstar"] == 5
with unittest.TestCase().assertRaisesRegex(ValueError, "crosses"):
r.jet_diagnostics([dict(tstar=250.5, tstar_force=250, cd=1, cl=0)], "fixed")
assert r.plan(h, "MRT", "window", "reverse")["modes"]["full_tstar"] == 305
def test_public_jet_source_and_pilot_separation():
from tempfile import TemporaryDirectory
with TemporaryDirectory() as tmp:
path = Path(tmp)
manifest = pilot_fixture(path, jet_scenario="fixed")
assert manifest["source_capture"].startswith("current checkout")
assert len(manifest["checkout_revision"]) == 40
assert r.verify_pilot(path, manifest, "MRT", "window", "fixed") == 3606.0
for scenario in ("none", "zero", "reverse"):
with unittest.TestCase().assertRaisesRegex(ValueError, "Jet scenario"):
r.verify_pilot(path, manifest, "MRT", "window", scenario)
for relative in ("uniform/summary.json", "refined/ready.json"):
receipt = path / relative
record = json.loads(receipt.read_text())
receipt.write_text(json.dumps(dict(record, jet_scenario="zero")))
with unittest.TestCase().assertRaises(ValueError):
r.verify_pilot(path, manifest, "MRT", "window", "fixed")
receipt.write_text(json.dumps(record))
with unittest.TestCase().assertRaises(ValueError):
r.verify_source(r.DEFAULT_SOURCE, "MRT", "last_base_step", "fixed")
def test_cv_snapshot_common_units_and_window_endpoints():
records = []
for refined, shape, origin, diameter, impulse in (
(True, (240, 568), (38., 22.), 40, [4., -8.]),
(False, (330, 1761), (.25, .5), 80, [16., -32.]),
(False, (658, 3521), (.375, .75), 160, [64., -128.]),
):
rho, ux, uy, fluid = np.ones(shape), np.full(shape, .01), np.zeros(shape), np.ones(shape, bool)
begin, n = round(290*diameter/r.SPEED), round(.02*diameter/r.SPEED)
record = r.cv_snapshot(rho, ux, uy, fluid, origin, refined, begin, n, diameter, impulse)
records.append(record)
np.testing.assert_allclose(record["outward_flux"], 0, atol=1e-12)
assert r.cv_selected(begin, diameter, "full") and not r.cv_selected(begin-n, diameter, "full")
end = round(300*diameter/r.SPEED)
assert r.cv_selected(end, diameter, "full") and not r.cv_selected(end+n, diameter, "full")
assert r.cv_selected(n, diameter, "pilot")
assert sum(r.cv_selected(step, diameter, "full") for step in range(begin-n, end+n+1, n)) == 501
for key in ("time_root", "interval_start_root", "interval_dt_root", "interval_impulse_root", "inventory"):
for record in records[1:]:
np.testing.assert_allclose(records[0][key], record[key], rtol=0, atol=1e-10)
assert records[0]["interval_dt_root"] == 80 and np.isclose(records[0]["tstar"], 290.02)
np.testing.assert_array_equal(records[0]["interval_impulse_root"], [4, -8])
def test_cv_pilot_mode_and_helper_provenance():
from tempfile import TemporaryDirectory
manifest = r.verify_source(r.DEFAULT_SOURCE, "MRT", "window", "fixed", True)
assert "tests/validation/run_jet_circle_cv.py" in manifest["sha256"]
with TemporaryDirectory() as tmp:
path = Path(tmp)
old = pilot_fixture(path, jet_scenario="fixed")
with unittest.TestCase().assertRaisesRegex(ValueError, "CV observation mode"):
r.verify_pilot(path, old, "MRT", "window", "fixed", True)
for relative in ("plan.json", "uniform/summary.json", "refined/summary.json",
"uniform/ready.json", "refined/ready.json"):
file = path / relative
record = json.loads(file.read_text())
record.update(observe_cv=True, cv_samples=250)
if relative == "plan.json":
record["source_manifest"] = manifest
file.write_text(json.dumps(record))
assert r.verify_pilot(path, manifest, "MRT", "window", "fixed", True) == 3606
def test_compact_geometry_and_cv_stencils():
from CelerisLab.lbm.refinement import ACTIVE
from CelerisLab.lbm.descriptors import SOLID
_, pair, _, host = r.host_geometry(r.DEFAULT_SOURCE, h, "MRT", "fixed", "compact")
assert pair.fine_shape == (168, 168) and tuple(pair.fine_origin) == (46, 46)
fluid = (pair.fine_kind == ACTIVE) & ((host.flags.reshape(pair.fine_shape) & SOLID) == 0)
rho, ux, uy = np.ones(pair.fine_shape), np.full(pair.fine_shape, .01), np.zeros(pair.fine_shape)
record = r.cv_snapshot(rho, ux, uy, fluid, pair.fine_origin, True, 1160000, 80, 40, [1, 2], "compact")
np.testing.assert_allclose(record["outward_flux"], 0, atol=1e-12)
stencil = np.zeros(pair.fine_shape, bool)
for rect in r.cv_contours(True, 40, "compact"):
x0, y0, x1, y1 = np.rint((rect - np.tile(pair.fine_origin, 2))/.5).astype(int)
stencil[y0-2:y1+2, x0-2:x0+2] = stencil[y0-2:y1+2, x1-2:x1+2] = True
stencil[y0-2:y0+2, x0-2:x1+2] = stencil[y1-2:y1+2, x0-2:x1+2] = True
assert fluid[stencil].all() and not stencil.ravel()[host.transfer_fine_cells].any()
def test_d160_config_and_single_case_receipts():
from tempfile import TemporaryDirectory
from unittest.mock import patch
cfg, _, p = r.config(h, 160, "MRT", "fixed")
assert (p["nx"], p["ny"], p["center"], p["radius"]) == (3521, 658, [320, 320.5], 80)
assert p["viscosity"] == .016 and p["mean_speed"] == .01 and not cfg["method"]["ddf_shifting"]
assert r.uniform_origin(160) == (.375, .75)
np.testing.assert_array_equal(r.cv_contours(False, 160), r.CV_CONTOURS + [.125, 0, .125, 0])
for scenario, expected_exit in (("fixed", 0), ("zero", 2)):
with TemporaryDirectory() as tmp:
path = Path(tmp)
manifest = pilot_fixture(path, jet_scenario=scenario, case_selection="uniform", uniform_diameter=160)
assert r.verify_pilot(path, manifest, "MRT", "window", scenario, False, "uniform", 160) == 1803
for selection, diameter, placement in (("both", 160, "wide"), ("uniform", 80, "wide"), ("uniform", 160, "compact")):
with unittest.TestCase().assertRaises(ValueError):
r.verify_pilot(path, manifest, "MRT", "window", scenario, False, selection, diameter, placement)
# Synthetic dispatch/result check: no CUDA imports or execution.
output = path / "full"
argv = ["runner", "--collision", "MRT", "--exchange-window", "--jet-scenario", scenario,
"--case", "uniform", "--uniform-diameter", "160", "--mode", "full",
"--pilot-results", str(path), "--output", str(output), "--max-wall-seconds", "10", "--gpu-gates-approved"]
result = dict(case="uniform", healthy=True, periodic=scenario == "zero", jet_diagnostic_windows=[],
status="driven_diagnostics_complete" if scenario == "fixed" else "periodic",
reference_comparison=dict(reference_agreement=False))
with patch.object(sys, "argv", argv), patch.dict(r.os.environ, CUDA_VISIBLE_DEVICES="mock-no-gpu"), \
patch.object(r, "run_case", return_value=result) as run:
assert r.main() == expected_exit and run.call_count == 1
comparison = json.loads((output / "comparison.json").read_text())
assert comparison["matched_case_pass"] is False and "refined_vs_uniform" not in comparison
assert comparison["status"].startswith("single_case_")
if __name__ == "__main__":
suite = unittest.TestSuite(
unittest.FunctionTestCase(fn)
+75
View File
@@ -238,6 +238,81 @@ def test_overlapping_force_regions_rejected(device):
bodies.build_compact_lists(bodies.build_flags(field.flag))
@pytest.mark.parametrize("second_x,error", [(14., "unique cut-link writers"),
(17., "active fluid receivers")])
def test_passive_overlap_rejected_before_publishing_lists(second_x, error):
cfg = LBMConfig(nx=32, ny=24)
cfg.validate()
bodies = ObjectManager(cfg.nx, cfg.ny, cfg.nz, cfg.nq, cfg)
for x in (14., second_x):
bodies.add(obj(x=x, y=12., r=4.3))
bodies.add(obj(x=4., y=4., r=2., is_sensor=True))
counts = np.full(bodies.count, 91, np.int32)
hits = []
flags = bodies.build_flags(np.full(cfg.nx * cfg.ny, FLUID, np.uint16))
for domain_flags in (flags, None):
with pytest.raises(ValueError, match=error):
bodies.build_compact_lists_for(bodies.objects, domain_flags,
sensor_cell_counts_out=counts, hits_out=hits)
np.testing.assert_array_equal(counts, 91)
assert hits == []
@pytest.mark.parametrize("centers", [((10., 12.), (22., 12.)),
((10.5, 12.), (19.5, 12.))])
def test_separated_passive_bodies_keep_valid_writers(centers):
# The second case has a subcell physical gap with a resolved fluid column.
cfg = LBMConfig(nx=32, ny=24)
cfg.validate()
bodies = ObjectManager(cfg.nx, cfg.ny, cfg.nz, cfg.nq, cfg)
for x, y in centers:
bodies.add(obj(x=x, y=y, r=4.3))
flags = bodies.build_flags(np.full(cfg.nx * cfg.ny, FLUID, np.uint16))
indices, directions, _, ids, *_ = bodies.build_compact_lists(flags)
assert set(ids) == {0, 1}
assert np.all(flags[indices] & FLUID) and not np.any(flags[indices] & SOLID)
assert len(np.unique(indices.astype(np.uint64) * cfg.nq + directions)) == len(indices)
@pytest.mark.parametrize("entry", ["sync", "initialize", "runtime_edit"])
@pytest.mark.parametrize("second_x", [5., 6.])
def test_passive_overlap_failure_keeps_live_state(device, entry, second_x):
from CelerisLab.simulation import Simulation
field = make_field(device)
bodies = manager(field, [obj(), obj(x=9., y=3., r=1., is_sensor=True)])
bodies.sync_to_gpu(field)
overlapping = obj(x=second_x)
if entry == "runtime_edit":
bodies.stage_add(overlapping)
else:
bodies.add(overlapping)
objects = bodies.objects
flags, curved, counts = field.flag.copy(), field.curved.fluid_idx.copy(), bodies.sensor_cell_counts.copy()
action = bodies.action.copy()
buffers = [(block, bytes(block.data), block.freed) for block in device.blocks]
writes = device.writes
sim = SimpleNamespace(bodies=bodies, field=field, lbm_cfg=field.cfg,
_initialized=entry == "runtime_edit",
_assert_allocation_contract=lambda: None,
stream=SimpleNamespace(synchronize=device.synchronize))
with pytest.raises(ValueError, match="active fluid receivers|unique cut-link writers"):
if entry == "sync":
bodies.sync_to_gpu(field)
elif entry == "initialize":
Simulation.initialize(sim)
else:
Simulation.sync_bodies(sim)
assert bodies.objects == objects
assert bodies.has_pending_edit() == (entry == "runtime_edit")
assert sim._initialized == (entry == "runtime_edit")
assert device.writes == writes and len(device.blocks) == len(buffers)
assert all(bytes(block.data) == data and block.freed == freed for block, data, freed in buffers)
np.testing.assert_array_equal(field.flag, flags)
np.testing.assert_array_equal(field.curved.fluid_idx, curved)
np.testing.assert_array_equal(bodies.sensor_cell_counts, counts)
np.testing.assert_array_equal(bodies.action, action)
def test_action_upload_and_scratch_are_pinned(device):
field = make_field(device, streaming="esopull")
bodies = manager(field, [obj(is_force_region=True)])
+75 -10
View File
@@ -8,17 +8,19 @@ import sys
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "tests/validation"))
from source_provenance import checkout_provenance
import pytest
from source_provenance import (HistoryUnavailable, checkout_provenance, historical_sha256,
historical_tree, history_unavailable, verify_manifest)
import unittest
import re100_helpers
import run_jet_circle_cv
ORIGIN = "47b05bc8abfc957fe876b0c63bdb386e3c6cbaa5"
def test_manifests_cover_current_source_and_preserve_origin():
names = subprocess.check_output(
["git", "-C", str(ROOT), "ls-files", "src"], text=True
).splitlines()
def test_manifests_match_their_recorded_source_and_preserve_origin():
"""Historical evidence integrity: each manifest describes the tree of its own commit."""
manifests = []
for group, hashes_key, revision_key in (
("dfg_re100", "production_sha256", "commit"),
("jet_circle_cv", "files", "revision"),
@@ -28,18 +30,22 @@ def test_manifests_cover_current_source_and_preserve_origin():
ROOT / "tests/validation/data" / group / "source_manifest.json"
).read_text()
)
# Structural checks need no history and always run.
assert manifest[revision_key] == ORIGIN
assert set(manifest[hashes_key]) == set(names)
assert manifest["source_origin_commits"][-1] == ORIGIN
assert (
manifest["historical_checkpoints"]["production"]
== "d79e3f6f0a424f5bcc3a1a6d610313e49daf97a2"
)
assert "not inherited from d79" in manifest["experiment"]
for name, expected in manifest[hashes_key].items():
assert (
hashlib.sha256((ROOT / name).read_bytes()).hexdigest() == expected
), name
manifests.append(manifest[hashes_key])
try:
names = historical_tree(ROOT, ORIGIN)
for digests in manifests:
assert set(digests) == set(names)
verify_manifest(ROOT, ORIGIN, digests)
except HistoryUnavailable as error:
history_unavailable(unittest.TestCase(), error)
assert re100_helpers.PIN == run_jet_circle_cv.PIN == ORIGIN
@@ -72,3 +78,62 @@ def test_runtime_checkout_identity_is_separate_from_source_origin(tmp_path):
assert checkout_provenance(tmp_path) == dict(
checkout_revision=expected, checkout_dirty=True
)
def test_historical_check_rejects_tampering_and_reports_missing_history(tmp_path):
with pytest.raises(HistoryUnavailable):
historical_sha256(tmp_path, ORIGIN, ["src/x.py"]) # not a repository
with pytest.raises(HistoryUnavailable):
historical_sha256(ROOT, "0" * 40, ["src/x.py"]) # commit absent
manifest = json.loads(
(ROOT / "tests/validation/data/dfg_re100/source_manifest.json").read_text())
digests = dict(manifest["production_sha256"])
name = sorted(digests)[0]
tampered = dict(digests, **{name: ("0" if digests[name][0] != "0" else "1") + digests[name][1:]})
try:
historical_tree(ROOT, ORIGIN)
except HistoryUnavailable as error:
history_unavailable(unittest.TestCase(), error)
with pytest.raises(RuntimeError, match="Historical manifest mismatch"):
verify_manifest(ROOT, ORIGIN, tampered) # one flipped digest is caught
with pytest.raises(ValueError, match="not part of"):
historical_sha256(ROOT, ORIGIN, ["src/CelerisLab/does_not_exist.py"])
def test_historical_files_reports_unreadable_blob_as_history_unavailable(tmp_path):
"""A blob missing from the clone is HistoryUnavailable, not a raw CalledProcessError."""
import source_provenance
from unittest.mock import patch
missing = subprocess.CompletedProcess(["git", "show"], 128, b"", b"fatal: missing blob")
with patch.object(source_provenance, "historical_sha256", return_value={"src/a.py": "0" * 64}), \
patch.object(source_provenance.subprocess, "run", return_value=missing):
with pytest.raises(HistoryUnavailable, match="blob not in this clone"):
source_provenance.historical_files(ROOT, ORIGIN, ["src/a.py"], tmp_path)
assert not (tmp_path / "src/a.py").exists()
def test_final_source_check_unchanged_changed_and_failed_recheck():
from source_provenance import final_source_check
recorded = dict(revision="a" * 40, dirty=False)
assert final_source_check(recorded, lambda: dict(recorded)) == dict(source_recheck="UNCHANGED")
changed = final_source_check(recorded, lambda: dict(recorded, dirty=True))
assert changed == dict(source_recheck="CHANGED", source_after=dict(recorded, dirty=True),
source_recheck_error=None)
def fails():
raise RuntimeError("Source hash mismatch: src/x.py")
failed = final_source_check(recorded, fails)
assert (failed["source_recheck"], failed["source_after"]) == ("RECHECK_FAILED", None)
assert "Source hash mismatch" in failed["source_recheck_error"]
def test_reject_output_inside_checkout(tmp_path):
from source_provenance import reject_output_inside
for output in (ROOT, ROOT / "out", ROOT / "src" / "x.json"):
with pytest.raises(SystemExit, match="outside the source checkout"):
reject_output_inside(output, ROOT)
reject_output_inside(tmp_path / "out", ROOT)
@@ -14,11 +14,35 @@ def integrate(t, y):
)
def analyze(directory):
def receipt_gate(directory, c, legacy=False):
"""Side files are used only if the sibling run receipt finished with an unchanged source.
Receipts from runners before the final source recheck have no run_status; they are
refused unless legacy=True (--legacy-receipts), and the result is labelled as such.
"""
path = Path(directory) / f"re{c['re_d']}_s{c['scale']}.json"
if not path.exists():
raise ValueError(f"No sibling run receipt {path.name}")
receipt = json.loads(path.read_text())
if receipt.get("case") != c:
raise ValueError("Sibling run receipt describes a different case")
if "run_status" not in receipt:
if not legacy:
raise ValueError("Legacy run receipt without run_status/final source recheck; "
"pass --legacy-receipts to analyze it as such")
return "LEGACY_NO_FINAL_SOURCE_RECHECK"
if receipt["run_status"] != "COMPLETE" or receipt.get("source_recheck") != "UNCHANGED":
raise ValueError(f"Run receipt is {receipt['run_status']} "
f"(source_recheck={receipt.get('source_recheck')})")
return "COMPLETE_SOURCE_UNCHANGED"
def analyze(directory, legacy=False):
meta_path = next(Path(directory).glob("samples_s*.json"))
meta = json.loads(meta_path.read_text())
c = meta["case"]
s = c["scale"]
gate = receipt_gate(directory, c, legacy)
sample_path = meta_path.parent / meta["sample_file"]
checksum_verified = False
if "sample_sha256" in meta:
@@ -97,6 +121,7 @@ def analyze(directory):
)
return dict(
scale=s,
receipt_gate=gate,
complete_schedule=bool(complete_schedule),
sample_checksum_verified=checksum_verified,
windows=records,
@@ -115,8 +140,10 @@ def main():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("directories", nargs="+", type=Path)
p.add_argument("--output", type=Path, required=True)
p.add_argument("--legacy-receipts", action="store_true",
help="Accept receipts written before the final source recheck (labelled)")
a = p.parse_args()
results = [analyze(d) for d in a.directories]
results = [analyze(d, a.legacy_receipts) for d in a.directories]
payload = dict(
status="TRANSIENT_DIAGNOSTIC_COMPARISON",
results=results,
@@ -80,11 +80,12 @@ class ThreeLevelCPU(unittest.TestCase):
f, g = run._levels[1:]
b = g.boundary
events, feeds = [], []
advance, gather = run._advance_child, run._gather_halo
def record_feed(buffer, **kw):
self.assertIs(buffer, (f.current, f.spare)[len(feeds)])
feeds.append(buffer)
advance(buffer, **kw)
advance, gather = run._advance, run._gather_halo
def record_feed(index, buffer, **kw): # index 2: L1 hop feeds L2
if index == 2:
self.assertIs(buffer, (f.current, f.spare)[len(feeds)])
feeds.append(buffer)
advance(index, buffer, **kw)
def record_halo(level, buffer, **kw):
if level is g:
events.append(("halo", buffer))
@@ -95,7 +96,7 @@ class ThreeLevelCPU(unittest.TestCase):
self.assertIs(kind, g.kind)
events.append(("step", src))
g.kernel.side_effect = record_step
with patch.object(run, "_advance_child", side_effect=record_feed), \
with patch.object(run, "_advance", side_effect=record_feed), \
patch.object(run, "_gather_halo", side_effect=record_halo):
run.step()
self.assertEqual(feeds, [f.current, f.spare])
@@ -105,8 +106,8 @@ class ThreeLevelCPU(unittest.TestCase):
self.assertGreater(len(np.unique(b.tables["block"])), 1)
# Negative control: reusing L1's old source at the midpoint must fail.
feeds.clear()
with patch.object(run, "_advance_child", side_effect=lambda buffer, **kw:
record_feed(f.current, **kw)):
with patch.object(run, "_advance", side_effect=lambda index, buffer, **kw:
record_feed(index, f.current if index == 2 else buffer, **kw)):
with self.assertRaises(AssertionError):
run.step()
run.close()
@@ -172,11 +173,12 @@ class ThreeLevelGPU(unittest.TestCase):
p1, p2 = run.patch, run.child_patch
run.initialize(state(16, 16, 1), state(*p1.fine_shape, 2), state(*p2.fine_shape, 3))
f = run._levels[1]
feeds, original = [], run._advance_child
def record(buffer, **kw):
feeds.append(int(buffer))
original(buffer, **kw)
with patch.object(run, "_advance_child", side_effect=record):
feeds, original = [], run._advance
def record(index, buffer, **kw): # index 2: L1 hop feeds L2
if index == 2:
feeds.append(int(buffer))
original(index, buffer, **kw)
with patch.object(run, "_advance", side_effect=record):
cur, spare = int(f.current), int(f.spare)
run.step(1)
self.assertEqual(feeds, [cur, spare]) # current G, then the new G of hop 1
+267
View File
@@ -0,0 +1,267 @@
"""Steady DFG 2D-1 (Re 20): refined and uniform runs against the official reference intervals.
Not a full benchmark certification. Declared deviations from the official setup:
pressure-NEQ outlet (``neq_extrap``) instead of do-nothing, local Zou-He inlet.
Geometry, inflow profile, initial state, refined layout and observation reuse the
pinned Re100 runner (``run_refined_dfg``) and ``re100_helpers`` unchanged; only the
lattice viscosity is set for Re 20 (nu = U_lattice * D / 20 at the root level).
Units. The helpers use scaled DFG units with mean inflow speed 1, so
Cd = 2 Fx / (rho U^2 D) and Cl = 2 Fy / (rho U^2 D) are scale-free, and
``pressure_probes`` returns dp / (rho U^2). The official p_diff = p(0.15, 0.2) -
p(0.25, 0.2) uses U = 0.2 m/s, rho = 1, so p_diff = dp * 0.04. Both are reported;
only p_diff is compared with the official interval. Forces use the existing sign
convention: F is the force of the fluid on the cylinder, i.e. the refined raw
exchange impulse (window ledger, ``total.impulse``) or the uniform ``read_force``,
with x along the inflow and y up from the bottom wall. Cd must be positive, and the
official Cl is positive (centre 0.20 below the centreline 0.205), so a flipped sign
shows up as Cl outside its interval.
Sampling. One sample every SAMPLE_TSTAR (t* = U t / D; 0.25 s), taken at the end of
a completed root step with all levels synchronized; the step index is recorded. Cd
and Cl are means over the sample interval (window exchange ledger refined, per-batch
forces uniform), which filters acoustic and step-parity oscillation; p_diff is the
end-of-interval value at the exact surface points.
Steady rule (our engineering choice, frozen before any run; not official). Over the
trailing WINDOW_TSTAR (t* 50..60 inclusive, 21 samples), for each of Cd, Cl and
p_diff: range (max - min) <= 10 % and |least-squares slope * window| <= 2 % of the
width of its official interval. Absolute limits, so the small Cl is never a divisor;
the slope term bounds slow monotone drift that the range alone would pass. If the
rule fails at TSTAR_MAX the run reports "not steady" and gives no interval verdict;
limits are never relaxed after a run. The verdict uses the window mean, and the
distance of each mean to its nearest interval edge is reported (negative outside).
"""
import argparse
import json
import math
import os
from pathlib import Path
import sys
import time
from tempfile import TemporaryDirectory
import numpy as np
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
import run_refined_dfg as rr # noqa: E402
RE = 20.0
U_PHYS = 0.2 # m/s mean inflow; rho = 1
TSTAR_MAX = 60.0 # 30 s physical
SAMPLE_TSTAR = 0.5 # 0.25 s
WINDOW_TSTAR = 10.0 # trailing 5 s, 21 samples
# Schaefer & Turek (1996), 2D-1 lower/upper bounds.
BANDS = dict(cd=(5.57, 5.59), cl=(0.0104, 0.0110), p_diff=(0.1172, 0.1176))
STEADY_RANGE = {k: (hi - lo) / 10 for k, (lo, hi) in BANDS.items()}
STEADY_SLOPE = {k: (hi - lo) / 50 for k, (lo, hi) in BANDS.items()}
# 2: {schema, provenance, cases}; the earlier receipts (schema 1) are a bare list of cases.
RESULT_SCHEMA = 2
DEVIATIONS = ("pressure-NEQ outlet (neq_extrap), not do-nothing", "local Zou-He inlet")
_re100_config = rr.config
def config(h, diameter, collision="SRT", jet_scenario="none"):
"""The pinned Re100 configuration with the lattice viscosity set for Re 20."""
if jet_scenario != "none":
raise ValueError("DFG 2D-1 runs have no Jets")
cfg, body, p = _re100_config(h, diameter, collision, jet_scenario)
nu = p["mean_speed"] * p["diameter"] / RE
cfg["physics"]["viscosity"] = nu
return cfg, body, dict(p, viscosity=nu, re=RE)
def lattice_parameters(diameter, speed=rr.SPEED):
nu = speed * diameter / RE
return dict(viscosity=nu, tau=0.5 + 3 * nu, steps=math.ceil(TSTAR_MAX * diameter / speed),
interval=round(SAMPLE_TSTAR * diameter / speed))
def sample(impulse, n, diameter, dp):
den = rr.SPEED**2 * diameter / 2
return dict(cd=float(impulse[0] / n / den), cl=float(impulse[1] / n / den),
dp_scaled=float(dp), p_diff=float(dp) * U_PHYS**2)
def summarize(rows):
"""Steady rule over the trailing window, then the interval verdict (or none)."""
if not rows or rows[-1]["tstar"] < TSTAR_MAX - 1e-9:
return dict(status="INCOMPLETE", steady=False, verdict=None)
window = [r for r in rows if r["tstar"] >= TSTAR_MAX - WINDOW_TSTAR - 1e-9]
t = np.array([r["tstar"] for r in window])
ranges = {k: max(r[k] for r in window) - min(r[k] for r in window) for k in BANDS}
drifts = {k: abs(np.polyfit(t, [r[k] for r in window], 1)[0]) * WINDOW_TSTAR for k in BANDS}
steady = all(ranges[k] <= STEADY_RANGE[k] and drifts[k] <= STEADY_SLOPE[k] for k in BANDS)
final = {k: float(np.mean([r[k] for r in window])) for k in BANDS}
edge = {k: min(final[k] - BANDS[k][0], BANDS[k][1] - final[k]) for k in BANDS}
result = dict(window_samples=len(window), ranges=ranges, drifts=drifts,
range_limits=STEADY_RANGE, slope_limits=STEADY_SLOPE, steady=steady,
window_mean=final, edge_distance=edge, bands=BANDS,
deviations=DEVIATIONS, cd_positive=final["cd"] > 0)
if not steady:
return dict(result, status="NOT_STEADY", verdict=None)
inside = {k: BANDS[k][0] <= final[k] <= BANDS[k][1] for k in BANDS}
return dict(result, status="WITHIN_INTERVALS" if all(inside.values()) else "OUTSIDE_INTERVALS",
verdict=inside)
def read_result(path):
"""Cases and provenance from a result.json of either schema.
Schema 1 (receipts before RESULT_SCHEMA=2, e.g. evidence 70f6617) is a bare list of
cases without provenance; schema 2 is {schema, provenance, cases}. Readers that
indexed [0] on schema 1 must use read_result(path)["cases"][0].
"""
data = json.loads(Path(path).read_text())
if isinstance(data, list):
return dict(schema=1, provenance=None, cases=data)
if not isinstance(data, dict) or data.get("schema") != RESULT_SCHEMA:
raise ValueError(f"unknown DFG Re20 result schema: {data.get('schema') if isinstance(data, dict) else type(data).__name__}")
return dict(schema=RESULT_SCHEMA, provenance=data["provenance"], cases=data["cases"])
def healthy(row):
return row["max_mach"] <= 0.2 and row["rho_min"] >= 0.5 and row["rho_max"] <= 1.5
def run_case(a, h, case):
from contextlib import ExitStack
from CelerisLab.cuda.context import CudaContext
from CelerisLab.lbm.descriptors import FLUID
from CelerisLab.lbm.refinement_stepper import FixedPatchStepper
from CelerisLab.simulation import Simulation
refined = case == "refined"
cfgdict, bodydict, p = config(h, 40 if refined else 80, a.collision)
lp = lattice_parameters(p["diameter"])
rows = []
with ExitStack() as stack:
if refined:
cfg, pair, body, host = rr.host_geometry(a.source, h, a.collision)
context = stack.enter_context(CudaContext(0))
backend = stack.enter_context(FixedPatchStepper(
cfg, rr.PLACEMENTS["wide"], context=context, coarse_boundary="channel",
base_block=rr.BASE_BLOCK, fine_block=rr.FINE_BLOCK, fine_objects=(body,),
observe_raw_exchange=True, observe_window_exchange=True))
from CelerisLab.lbm.channel import channel_masks
from CelerisLab.lbm.refinement import ACTIVE
from CelerisLab.lbm.descriptors import SOLID
masks = [channel_masks(pair.coarse_shape)[0] & ~pair.covered,
(pair.fine_kind == ACTIVE)
& ((host.flags.reshape(pair.fine_shape) & SOLID) == 0)]
backend.initialize(rr.initial_state(pair.coarse_shape, (0.0, 0.0), 1.0, root=True),
rr.initial_state(pair.fine_shape, pair.fine_origin, 0.5))
probe_p = dict(p, center=[(80.5 - pair.fine_origin[0]) / 0.5 - 0.5,
(81.0 - pair.fine_origin[1]) / 0.5 - 0.5], radius=40.0)
else:
tmp = Path(stack.enter_context(TemporaryDirectory()))
(tmp / "lbm.json").write_text(json.dumps(cfgdict))
(tmp / "body.json").write_text(json.dumps(bodydict))
sim = stack.enter_context(Simulation(str(tmp / "lbm.json"), str(tmp / "body.json"),
device_id=0))
sim.initialize()
probe_p = p
fluid = (sim.field.flag.reshape(p["ny"], p["nx"]) & FLUID) != 0
begin = 0
while begin < lp["steps"]:
rr.check_deadline(a.deadline)
n = min(lp["interval"], lp["steps"] - begin)
if refined:
impulse = rr.advance_refined_interval(backend, n, a.deadline, "window")
fields = [rr.macros(g) for g in backend.download()]
rho, ux, uy = fields[1]
fluid = masks[1]
vals = [tuple(v[m] for v in f) for m, f in zip(masks, fields)]
else:
impulse, done = np.zeros(2), 0
while done < n:
batch = min(100, n - done)
sim.run(batch)
impulse += sim.read_force(0).astype(np.float64) * batch
done += batch
m = sim.get_macroscopic()
rho, ux, uy = [np.asarray(m[k], np.float64).reshape(p["ny"], p["nx"])
for k in ("rho", "ux", "uy")]
vals = [(rho[fluid], ux[fluid], uy[fluid])]
begin += n
row = dict(step=begin, tstar=begin * rr.SPEED / p["diameter"],
**sample(impulse, n, p["diameter"],
h.pressure_probes(rho, fluid, probe_p)["dp"]),
rho_min=float(min(v[0].min() for v in vals)),
rho_max=float(max(v[0].max() for v in vals)),
max_mach=float(max(np.sqrt(3 * (v[1]**2 + v[2]**2)).max() for v in vals)))
rr.ensure_finite(row)
rows.append(row)
if not healthy(row):
break
summary = summarize(rows) if healthy(rows[-1]) else dict(status="UNHEALTHY", steady=False,
verdict=None)
return dict(case=case, collision=a.collision, lattice=lp, rows=rows, **summary)
def plan(collision="SRT"):
return dict(re=RE, collision=collision, tstar_max=TSTAR_MAX, sample_tstar=SAMPLE_TSTAR,
window_tstar=WINDOW_TSTAR, bands=BANDS, range_limits=STEADY_RANGE, slope_limits=STEADY_SLOPE,
deviations=DEVIATIONS, pressure_scale=U_PHYS**2,
refined_root=lattice_parameters(40), refined_fine_tau=0.5 + 3 * 2 * rr.SPEED * 40 / RE,
uniform=lattice_parameters(80),
label="compared against the official reference bands (non-canonical neq_extrap outlet, zou_he_local inlet); not a full benchmark certification")
def provenance(source):
"""Current checkout identity: HEAD, dirty flag, src tree and this runner's hash."""
import hashlib
import subprocess
from source_provenance import checkout_provenance
tree = subprocess.check_output(["git", "-C", str(source), "rev-parse", "HEAD:src"],
text=True).strip()
record = dict(source_mode="current_checkout", **checkout_provenance(source), src_tree=tree,
runner_sha256=hashlib.sha256(Path(__file__).read_bytes()).hexdigest())
if record["checkout_dirty"]: # HEAD would not describe the code that runs
raise RuntimeError("DFG Re20 runs require a clean checkout")
return record
def reject_output_inside(output, source):
"""Receipts inside the checkout would make it dirty and change the recorded source."""
output = Path(output).resolve()
if output == source or source in output.parents:
raise SystemExit(f"--output must be outside the source checkout {source}")
def main():
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--case", choices=["refined", "uniform", "both"], default="refined")
parser.add_argument("--collision", choices=["SRT", "MRT"], default="SRT")
parser.add_argument("--source", type=Path, default=rr.DEFAULT_SOURCE)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--plan", action="store_true", help="Write the frozen plan; no CUDA")
parser.add_argument("--max-wall-seconds", type=float)
a = parser.parse_args()
a.source = a.source.resolve()
reject_output_inside(a.output, a.source)
record = dict(plan(a.collision), schema=RESULT_SCHEMA, provenance=provenance(a.source))
a.output.mkdir(parents=True, exist_ok=False)
(a.output / "plan.json").write_text(json.dumps(record, indent=2, allow_nan=False))
if a.plan:
return
if not a.max_wall_seconds or a.max_wall_seconds <= 0 or not os.environ.get("CUDA_VISIBLE_DEVICES"):
raise SystemExit("GPU run needs --max-wall-seconds > 0 and explicit CUDA_VISIBLE_DEVICES")
h, _ = rr.helpers(a.source)
rr.config = config # host_geometry builds its root configuration through this name
a.deadline = time.monotonic() + a.max_wall_seconds
cases = ["refined", "uniform"] if a.case == "both" else [a.case]
results = [run_case(a, h, case) for case in cases]
after = provenance(a.source)
if after != record["provenance"]:
raise RuntimeError("Source checkout changed during the run")
(a.output / "result.json").write_text(json.dumps(dict(schema=RESULT_SCHEMA, provenance=after,
cases=results),
indent=2, allow_nan=False))
if __name__ == "__main__":
main()
+90 -12
View File
@@ -3,6 +3,7 @@
import argparse
import hashlib
import subprocess
import json
import os
from pathlib import Path
@@ -74,7 +75,25 @@ def cv(g, fluid, origin, rectangle, nu, h=0.5):
Cauchy stress from centred velocity gradients, NOT postcollision non-equilibrium.
Rectangle faces lie on cell edges; fluid mask excludes the staircase cylinder.
"""
rho, mom, vel = moments(g)
return _cv_quantities(*moments(g), fluid, origin, rectangle, nu, h)
def cv_from_macros(rho, ux, uy, fluid, origin, rectangle, nu, h=0.5):
"""Same CV calculation from public macros, in one common length/time frame.
``origin`` is the lower cell edge, ``h`` the cell width, and ``nu`` uses
that frame. Returns [mass, Px, Py] inventory and outward total flux.
Crop fields/mask together and translate origin before calling if desired.
No boundary-exchange observation enters this independent estimator.
"""
rho = np.asarray(rho, np.float64)
if rho.ndim != 2 or any(np.shape(v) != rho.shape for v in (ux, uy, fluid)):
raise ValueError("CV macros and fluid mask must have matching 2D shapes")
vel = np.asarray([ux, uy], np.float64)
return _cv_quantities(rho, rho[None] * vel, vel, fluid, origin, rectangle, nu, h)
def _cv_quantities(rho, mom, vel, fluid, origin, rectangle, nu, h):
x0, y0, x1, y1 = np.rint((np.array(rectangle) - np.tile(origin, 2)) / h).astype(int)
np.testing.assert_allclose(
np.array([x0, y0, x1, y1]) * h + np.tile(origin, 2), rectangle, atol=1e-12
@@ -202,6 +221,60 @@ def verify(repo):
return manifest
def verify_historical(repo):
"""The pinned manifest describes the PIN tree (read with git, not the working tree)."""
from source_provenance import verify_manifest
manifest = json.loads((HERE / "data/jet_circle_cv/source_manifest.json").read_text())
if manifest["revision"] != PIN:
raise RuntimeError("Wrong pinned revision")
verify_manifest(repo, PIN, manifest["files"])
return manifest
def current_checkout(repo):
"""Provenance for a run on the current checkout; never labelled as the pinned source."""
files = subprocess.check_output(["git", "-C", str(repo), "ls-files", "src"],
text=True).splitlines()
return dict(source_mode="current_checkout",
source_capture="current checkout, not the frozen pinned qualification",
source_revision=checkout_provenance(repo)["checkout_revision"],
src_tree=subprocess.check_output(["git", "-C", str(repo), "rev-parse",
"HEAD:src"], text=True).strip(),
src_sha256={f: hashlib.sha256((repo / f).read_bytes()).hexdigest()
for f in files})
def source_identity(repo, current):
"""Recorded source of a run; recomputed after the run by finish_receipt."""
if current:
provenance = dict(**checkout_provenance(repo), **current_checkout(repo))
if provenance["checkout_dirty"]:
raise RuntimeError("Current-checkout runs require a clean checkout")
return provenance
source = verify(repo)
return dict(source_experiment=source["experiment"], source_mode="pinned",
**checkout_provenance(repo))
def finish_receipt(path, payload, provenance, recheck):
"""Mark the receipt COMPLETE only if the source is unchanged; otherwise INVALID.
Receipts written during the run carry run_status="INCOMPLETE", so an interrupted
run never leaves a complete-looking receipt.
"""
from source_provenance import final_source_check
fields = final_source_check(provenance, recheck)
unchanged = fields["source_recheck"] == "UNCHANGED"
payload = dict(payload, **fields,
run_status="COMPLETE" if unchanged else "INVALID_SOURCE_CHANGED")
temporary = Path(str(path) + ".tmp")
temporary.write_text(json.dumps(payload, indent=2) + "\n")
temporary.replace(path)
if not unchanged:
raise RuntimeError("Source changed during the run; receipt marked invalid")
return payload
def run_case(c, ctx, output, provenance, continuation=False):
from CelerisLab.body.geometry.circle import CircleGeometry
from CelerisLab.body.objects import SimObject
@@ -512,16 +585,16 @@ def run_case(c, ctx, output, provenance, continuation=False):
recorder.flush()
payload = dict(
case=c,
source_revision=PIN,
**provenance,
**{"source_revision": PIN, **provenance},
status="PILOT_DIAGNOSTIC_ONLY",
run_status="INCOMPLETE", # final source recheck in main() completes it
windows=rows,
physics_validation_pass=False,
precision="FP32 evolution, FP64 host diagnostics",
)
(output / f"re{c['re_d']}_s{c['scale']}.json").write_text(
json.dumps(payload, indent=2) + "\n"
)
receipt = output / f"re{c['re_d']}_s{c['scale']}.json"
Path(str(receipt) + ".tmp").write_text(json.dumps(payload, indent=2) + "\n")
Path(str(receipt) + ".tmp").replace(receipt)
print(
json.dumps(
dict(
@@ -553,6 +626,9 @@ def main():
help="Fixed measurement campaign: Re100, scale1/2, 3200 reference steps",
)
p.add_argument("--plan", action="store_true")
p.add_argument("--current-checkout", action="store_true",
help="Run on the current clean checkout and record its provenance "
"instead of requiring the frozen pinned source")
a = p.parse_args()
if a.steps <= 0 or a.steps > 3200:
p.error("steps must be 1..3200")
@@ -577,7 +653,8 @@ def main():
)
plan = dict(
case=c,
required_revision=PIN,
required_revision=None if a.current_checkout else PIN,
source_mode="current_checkout" if a.current_checkout else "pinned",
required_free_device_mib=1024,
estimated_peak_device_mib=128 * a.scale**2,
estimated_output_mib=64 if a.continuation else 2,
@@ -593,10 +670,9 @@ def main():
):
p.error("Root must select one verified idle GPU via CUDA_VISIBLE_DEVICES")
repo = a.repo.resolve()
source = verify(repo)
provenance = dict(
source_experiment=source["experiment"], **checkout_provenance(repo)
)
from source_provenance import reject_output_inside
reject_output_inside(a.output, repo)
provenance = source_identity(repo, a.current_checkout)
sys.path.insert(0, str(repo / "src"))
import CelerisLab
@@ -612,7 +688,9 @@ def main():
free, total = cuda.mem_get_info()
if free < max(1024, 2 * plan["estimated_peak_device_mib"]) * 1024**2:
raise RuntimeError("Insufficient free device memory")
run_case(c, ctx, a.output, provenance, continuation=a.continuation)
payload = run_case(c, ctx, a.output, provenance, continuation=a.continuation)
finish_receipt(a.output / f"re{c['re_d']}_s{c['scale']}.json", payload, provenance,
lambda: source_identity(repo, a.current_checkout))
print(
json.dumps(
dict(status="COMPLETE_DIAGNOSTIC_ONLY", physics_validation_pass=False)
+276 -38
View File
@@ -17,8 +17,109 @@ HERE = Path(__file__).resolve().parent
DEFAULT_SOURCE = HERE.parents[1]
BOUNDS = (40, 24, 320, 140)
BASE_BLOCK = (2, 881) # Exact 166 x 881 domain; no geometric padding.
FINE_BLOCK = (8, 8) # 240 x 568 including four fine halo cells.
FINE_BLOCK = (8, 8) # Wide 240 x 568 or compact 168 x 168, including fine halos.
SPEED = 0.01
JET_PEAK = 0.001
JET_WINDOWS = [(t, t + 10) for t in range(250, 300, 10)]
CV_CONTOURS = np.array([(48., 44., 160., 120.), (44., 36., 180., 128.)])
PLACEMENTS = {"wide": BOUNDS, "compact": (48, 48, 128, 128)}
COMPACT_CV_CONTOURS = np.array([(52., 52., 112., 112.), (50., 50., 120., 120.)])
def selected_cases(selection="both", uniform_diameter=80):
return [(case, diameter) for case, diameter in (("uniform", uniform_diameter), ("refined", 40))
if selection in ("both", case)]
def uniform_origin(diameter):
h = 40 / diameter
return (.5 - .5*h, 1 - h)
def cv_contours(refined, diameter, placement="wide"):
contours = CV_CONTOURS if placement == "wide" else COMPACT_CV_CONTOURS
return contours if refined else contours + [20/diameter, 0., 20/diameter, 0.]
def cv_selected(step, diameter, mode):
return mode == "pilot" or round(290 * diameter / SPEED) <= step <= round(300 * diameter / SPEED)
def cv_snapshot(rho, ux, uy, fluid, origin, refined, begin, n, diameter, impulse, placement="wide"):
"""Cropped public macros only; all inventories, fluxes and impulses use root units."""
from run_jet_circle_cv import cv_from_macros
rectangles = cv_contours(refined, diameter, placement)
h_cell = .5 if refined else 40 / diameter
origin = np.asarray(origin, dtype=float)
x0, y0, x1, y1 = np.rint((rectangles[-1] - np.tile(origin, 2)) / h_cell).astype(int)
selection = np.s_[y0-2:y1+2, x0-2:x1+2]
cropped_origin = origin + h_cell * np.array([x0-2, y0-2])
fields = [v[selection] for v in (rho, ux, uy, fluid)]
values = [cv_from_macros(*fields, cropped_origin, rect, .004, h=h_cell)
for rect in rectangles]
h_native = 1.0 if refined else 40 / diameter
return dict(step=begin+n, tstar=(begin+n)*SPEED/diameter,
time_root=(begin+n)*h_native, interval_start_root=begin*h_native,
interval_dt_root=n*h_native,
interval_impulse_root=(np.asarray(impulse)*h_native**2).tolist(),
inventory=[inventory.tolist() for inventory, _ in values],
outward_flux=[flux.tolist() for _, flux in values])
def public_jets(center, radius, scenario):
"""Same physical apertures; caller supplies its public geometry frame."""
if scenario == "none":
return ()
from CelerisLab.body.jets import Jet, Sector, Radial
peak = 0.0 if scenario == "zero" else JET_PEAK
return tuple(
Jet(Sector(center, angle, math.pi / 12, radius * 1.005),
Radial(center, inward=False), "parabolic", amplitude)
for angle, amplitude in ((math.pi / 2, peak), (3 * math.pi / 2, -peak))
)
def jet_interval(handle, scenario, begin, n, diameter):
"""Apply a reversal only at a completed interval boundary, before advancing."""
if scenario != "none":
boundaries = [round(t * diameter / SPEED) for t in (250, 260, 270, 280, 290, 300, 305)]
n = min([n] + [step - begin for step in boundaries if step > begin])
if scenario != "reverse":
return n, None
switch = round(300 * diameter / SPEED)
if begin < switch:
return min(n, switch - begin), None
if begin == switch:
handle.set_jet_amplitude(0, -JET_PEAK)
handle.set_jet_amplitude(1, JET_PEAK)
return n, dict(step=begin, tstar=300.0, amplitudes=[-JET_PEAK, JET_PEAK])
return n, None
def jet_diagnostics(rows, scenario):
"""Raw interval-weighted diagnostics only; no driven-flow qualification gate."""
windows = JET_WINDOWS + ([(300, 305)] if scenario == "reverse" else [])
result = []
for start, end in windows:
selected = [r for r in rows if start <= r["tstar_force"] < end]
if not selected:
continue
if any(2 * r["tstar_force"] - r["tstar"] < start - 1e-9
or r["tstar"] > end + 1e-9 for r in selected):
raise ValueError("Jet diagnostic interval crosses a window boundary")
weights = np.array([2 * (r["tstar"] - r["tstar_force"]) for r in selected])
record = dict(start_tstar=start, end_tstar=end, samples=len(selected),
covered_tstar=float(weights.sum()),
complete=bool(np.isclose(weights.sum(), end - start, rtol=0, atol=1e-9)))
for key in ("cd", "cl"):
values = np.array([r[key] for r in selected])
record[key] = dict(mean=float(np.average(values, weights=weights)),
min=float(values.min()), max=float(values.max()),
rms=float(np.sqrt(np.average(values**2, weights=weights))))
result.append(record)
return result
def helpers(source):
@@ -39,26 +140,34 @@ def helpers(source):
return h, periodic
def config(h, diameter, collision="SRT"):
def config(h, diameter, collision="SRT", jet_scenario="none"):
cfg, body, p = h.make_config(diameter, SPEED, collision, "neq_extrap", "zou_he_local")
cfg["method"]["outlet"]["srt_neq_damp"] = 0.5
if jet_scenario != "none":
if collision != "MRT":
raise ValueError("Public Jet scenarios require MRT")
cfg["method"]["ddf_shifting"] = False
return cfg, body, p
def plan(h, collision="SRT", observation_mode="last_base_step"):
_, _, c = config(h, 40, collision)
_, _, f = config(h, 80, collision)
x0, y0, x1, y1 = BOUNDS
def plan(h, collision="SRT", observation_mode="last_base_step", jet_scenario="none", uniform_diameter=80, placement="wide"):
_, _, c = config(h, 40, collision, jet_scenario)
_, _, f = config(h, uniform_diameter, collision, jet_scenario)
bounds = PLACEMENTS[placement]
x0, y0, x1, y1 = bounds
fine_active = 4 * (x1 - x0) * (y1 - y0)
coarse_uncovered = c["nx"] * c["ny"] - fine_active // 4
return dict(
status="prepared_not_run",
collision=collision,
jet_scenario=jet_scenario,
ddf_shifting=jet_scenario == "none",
observation_mode=observation_mode,
root_gpu_gates_required=True,
root=c,
uniform=f,
bounds_root_edge=list(BOUNDS),
bounds_root_edge=list(bounds),
fine_origin_root_edge=[x0 - 2, y0 - 2],
physical_patch_m=[
(x0 - 0.5) * c["dx"],
(y0 - 1) * c["dx"],
@@ -78,19 +187,31 @@ def plan(h, collision="SRT", observation_mode="last_base_step"):
physical_viscosity_m2_s=0.001,
finest_dx_m=f["dx"],
finest_dt_s=f["dt"],
refined_finest_dx_m=c["dx"]/2,
refined_finest_dt_s=c["dt"]/2,
approximate_leaf_cell_ratio=(coarse_uncovered + fine_active)
/ (f["nx"] * f["ny"]),
approximate_collision_work_ratio=(coarse_uncovered + 2 * fine_active)
/ (2 * f["nx"] * f["ny"]),
/ ((uniform_diameter/40) * f["nx"] * f["ny"]),
cost_caveat="Geometry-only estimates include ghosts/solids; exclude block halos, transfers, observation and launch overhead. Not measured speedup.",
sampling_offset="For origin (38,22), fine physical x=(j+75.5)*dx80 versus uniform i*dx80: half-fine-cell x offset. Fine physical y=(k+42.5)*dx80 equals uniform (l-.5)*dx80 at l=k+43: y samples align. Body, wall and source planes coincide.",
sampling_offset="Uniform nodes map to root edges X=.5+i*(40/D), Y=1+(j-.5)*(40/D). Refined centers are fine_origin_root_edge+(index+.5)*.5. Body, wall and source planes coincide; compare observables, not pointwise arrays.",
outlet=(
"SRT pressure NEQ, all nine fneq damped by .5, no backflow clamp; noncanonical DFG outlet"
if collision == "SRT" else
"MRT pressure NEQ, unknowns q2/6/8 extrapolated; no backflow clamp; noncanonical DFG outlet"
),
modes={"pilot_tstar": 5.0, "full_tstar": 300.0},
modes={"pilot_tstar": 5.0, "full_tstar": 305.0 if jet_scenario == "reverse" else 300.0},
observation_dense_tstar=0.02,
**(dict(jets=dict(
uniform_origin_node=f["center"], refined_origin_root_edge=[80.5, 81.0],
angles_rad=[math.pi / 2, 3 * math.pi / 2], half_width_rad=math.pi / 12,
radial_extent_over_radius=[0.0, 1.005],
field="Radial(outward)", profile="parabolic", condition="velocity",
initial_amplitudes=[0.0, 0.0] if jet_scenario == "zero" else [JET_PEAK, -JET_PEAK],
reversal_tstar=300.0 if jet_scenario == "reverse" else None,
diagnostic_windows_tstar=JET_WINDOWS + ([(300, 305)] if jet_scenario == "reverse" else []),
transfer="constant", driven_qualification="not assessed",
)) if jet_scenario != "none" else {}),
)
@@ -126,7 +247,7 @@ def initial_state(shape, origin, spacing, height=164.0, speed=SPEED, root=False)
return w * (1 + 3 * cu + 4.5 * cu * cu - 1.5 * velocity * velocity)
def host_geometry(source, h, collision="SRT"):
def host_geometry(source, h, collision="SRT", jet_scenario="none", placement="wide"):
"""Production donor/transfer and complete cut-link checks, without CUDA."""
from CelerisLab.config import load_lbm_config
from CelerisLab.lbm.refinement import FixedPatch2D, prepare_fine_boundary
@@ -139,12 +260,12 @@ def host_geometry(source, h, collision="SRT"):
from CelerisLab.body.geometry.circle import CircleGeometry
from CelerisLab.body.objects import SimObject
cfgdict, _, _ = config(h, 40, collision)
cfgdict, _, _ = config(h, 40, collision, jet_scenario)
with TemporaryDirectory() as tmp:
p = Path(tmp) / "cfg.json"
p.write_text(json.dumps(cfgdict))
cfg = load_lbm_config(str(p))
pair = FixedPatch2D.from_config(cfg, BOUNDS)
pair = FixedPatch2D.from_config(cfg, PLACEMENTS[placement])
layout = BlockLayout2D.build(cfg.ny, cfg.nx, *BASE_BLOCK)
validate_channel_config(cfg, [BASE_BLOCK, FINE_BLOCK])
targets = blocked_base_tables(pair, layout, np.argsort(pair.coalesce_coarse))[1]
@@ -153,18 +274,41 @@ def host_geometry(source, h, collision="SRT"):
body = SimObject(
0, geometry=CircleGeometry(80.5, 81.0, 20.0), center=(80.5, 81.0), radius=20.0
)
host = prepare_fine_boundary(cfg, pair, (body,), _shifted_blocked=True)
host = prepare_fine_boundary(
cfg, pair, (body,), jets=public_jets((80.5, 81.0), 20.0, jet_scenario),
_shifted_blocked=jet_scenario == "none",
)
return cfg, pair, body, host
def verify_source(source, collision="SRT", observation_mode="last_base_step"):
def verify_source(source, collision="SRT", observation_mode="last_base_step", jet_scenario="none", observe_cv=False):
if collision not in ("SRT", "MRT"):
raise ValueError("Matched collision must be SRT or MRT")
if observation_mode not in ("last_base_step", "window"):
raise ValueError("Unknown refined observation mode")
if observe_cv and jet_scenario == "none":
raise ValueError("--observe-cv requires an explicit public Jet scenario")
name = ("source_manifest_window.json" if observation_mode == "window" else
"source_manifest.json" if collision == "SRT" else "source_manifest_mrt.json")
expected = json.loads((HERE / "data/refined_dfg" / name).read_text())
if jet_scenario != "none":
if collision != "MRT" or observation_mode != "window":
raise ValueError("Public Jet scenarios require MRT and --exchange-window")
import subprocess
sys.path.insert(0, str(HERE))
from source_provenance import checkout_provenance
paths = set(expected["sha256"])
if observe_cv:
paths.add("tests/validation/run_jet_circle_cv.py")
paths.update(subprocess.check_output(
["git", "-C", str(source), "ls-files", "src"], text=True).splitlines())
return dict(
source_capture="current checkout, not historical DFG qualification",
**checkout_provenance(source), observation_mode=observation_mode,
sha256={name: hashlib.sha256((source / name).read_bytes()).hexdigest()
for name in sorted(paths)},
)
for name, digest in expected["sha256"].items():
if hashlib.sha256((source / name).read_bytes()).hexdigest() != digest:
raise RuntimeError("Source mismatch: " + name)
@@ -219,29 +363,33 @@ def run_case(a, h, periodic, case):
from contextlib import ExitStack
refined = case == "refined"
cfgdict, bodydict, p = config(h, 40 if refined else 80, a.collision)
cap = 5.0 if a.mode == "pilot" else 300.0
cfgdict, bodydict, p = config(h, 40 if refined else a.uniform_diameter, a.collision, a.jet_scenario)
cap = 5.0 if a.mode == "pilot" else (305.0 if a.jet_scenario == "reverse" else 300.0)
max_steps = math.ceil(cap * p["diameter"] / SPEED)
rows = []
healthy = True
step_seconds = 0.0
cv_seconds, cv_samples = 0.0, 0
amplitude_changes = []
started = time.monotonic()
check_deadline(a.deadline)
dest = a.output / case
dest.mkdir()
with ExitStack() as stack:
cv_out = stack.enter_context((dest / "cv_history.jsonl").open("w")) if a.observe_cv else None
if refined:
cfg, pair, body, host = host_geometry(a.source, h, a.collision)
cfg, pair, body, host = host_geometry(a.source, h, a.collision, a.jet_scenario, a.placement)
context = stack.enter_context(CudaContext(0))
backend = stack.enter_context(
FixedPatchStepper(
cfg,
BOUNDS,
PLACEMENTS[a.placement],
context=context,
coarse_boundary="channel",
base_block=BASE_BLOCK,
fine_block=FINE_BLOCK,
fine_objects=(body,),
jets=public_jets((80.5, 81.0), 20.0, a.jet_scenario),
observe_raw_exchange=True,
**({"observe_window_exchange": True} if a.observation_mode == "window" else {}),
)
@@ -274,6 +422,9 @@ def run_case(a, h, periodic, case):
sim = stack.enter_context(
Simulation(str(tmp / "lbm.json"), str(tmp / "body.json"), device_id=0)
)
for index, jet in enumerate(public_jets(p["center"], p["radius"], a.jet_scenario)):
if sim.add_jet(jet) != index:
raise RuntimeError("Unexpected public Jet registration index")
sim.initialize()
probe_p = p
fluid = (sim.field.flag.reshape(p["ny"], p["nx"]) & FLUID) != 0
@@ -290,6 +441,10 @@ def run_case(a, h, periodic, case):
else 0.5
)
n = min(max(1, round(width * p["diameter"] / SPEED)), max_steps - begin)
n, change = jet_interval(backend if refined else sim, a.jet_scenario,
begin, n, p["diameter"])
if change is not None:
amplitude_changes.append(change)
impulse = np.zeros(2)
tick = time.monotonic()
if refined:
@@ -359,6 +514,19 @@ def run_case(a, h, periodic, case):
/ SPEED,
**h.pressure_probes(rho, fluid, probe_p),
)
if a.jet_scenario != "none":
row.update(step_start=begin, tstar_start=begin * SPEED / p["diameter"],
interval_steps=n, impulse_x=float(impulse[0]),
impulse_y=float(impulse[1]))
if a.observe_cv and cv_selected(begin+n, p["diameter"], a.mode):
tick = time.monotonic()
record = cv_snapshot(rho, ux, uy, fluid,
pair.fine_origin if refined else uniform_origin(p["diameter"]),
refined, begin, n, p["diameter"], impulse, a.placement)
cv_out.write(json.dumps(record, allow_nan=False) + "\n")
cv_out.flush()
cv_samples += 1
cv_seconds += time.monotonic() - tick
check_deadline(a.deadline)
ensure_finite(row)
rows.append(row)
@@ -402,14 +570,24 @@ def run_case(a, h, periodic, case):
steps=begin,
startup_seconds=startup,
step_seconds=step_seconds,
observation_and_overhead_seconds=time.monotonic() - started - startup - step_seconds,
wall_seconds=time.monotonic() - started,
force="raw stationary-body link momentum reaction, per-unit-depth; average over entire observation interval; normalized in each base lattice",
mass_units="rho_reference * m^2 per unit depth",
flux_units="rho_reference * m^2/s per unit depth",
pressure="postcollision end-sample; physical pressure difference / (rho_reference * 1 m/s squared)",
suite_validated=False,
**periodic.assess(rows),
**(periodic.assess(rows) if a.jet_scenario in ("none", "zero") else
dict(periodic=False, checks={}, cycles=[])),
)
if a.jet_scenario != "none":
result.update(jet_diagnostic_windows=jet_diagnostics(rows, a.jet_scenario),
amplitude_changes=amplitude_changes,
pressure_classification=("DFG reference verdict retained, including pressure"
if a.jet_scenario == "zero" else "diagnostic only; no official DFG reference gate"))
if a.observe_cv:
result.update(cv_observation_seconds=cv_seconds, cv_samples=cv_samples,
cv_qualification="not assessed; scalar observations only")
if not healthy:
result["periodic"] = False
reference = json.loads(
@@ -425,7 +603,8 @@ def run_case(a, h, periodic, case):
else (
"pilot_complete"
if a.mode == "pilot"
else ("periodic" if result["periodic"] else "incomplete_periodicity")
else ("driven_diagnostics_complete" if a.jet_scenario in ("fixed", "reverse") else
"periodic" if result["periodic"] else "incomplete_periodicity")
)
)
(dest / "summary.json").write_text(json.dumps(result, indent=2, allow_nan=False))
@@ -464,27 +643,39 @@ def check_deadline(deadline):
)
def verify_pilot(path, manifest, collision="SRT", observation_mode="last_base_step"):
def verify_pilot(path, manifest, collision="SRT", observation_mode="last_base_step", jet_scenario="none", observe_cv=False,
case_selection="both", uniform_diameter=80, placement="wide"):
if path is None:
raise ValueError(
"Full comparison requires --pilot-results from a completed matched pilot"
)
previous = read_finite_json(path / "plan.json")
options = dict(case_selection=(case_selection, "both"), uniform_diameter=(uniform_diameter, 80), placement=(placement, "wide"))
if any(previous.get(k, default) != expected for k, (expected, default) in options.items()):
raise ValueError("Pilot case, resolution or placement differs")
if previous.get("observe_cv", False) != observe_cv:
raise ValueError("Pilot CV observation mode differs")
if previous.get("jet_scenario", "none") != jet_scenario:
raise ValueError("Pilot Jet scenario differs")
if previous.get("observation_mode") != observation_mode:
raise ValueError("Pilot observation mode differs")
if previous.get("collision") != collision:
raise ValueError("Pilot collision differs")
if previous["source_manifest"] != manifest or previous["bounds_root_edge"] != list(
BOUNDS
PLACEMENTS[placement]
):
raise ValueError("Pilot source or frozen geometry differs")
if previous.get("runner_sha256") != runner_hash():
raise ValueError("Pilot runner differs")
calibrated_seconds = 0.0
for case, diameter in (("uniform", 80), ("refined", 40)):
for case, diameter in selected_cases(case_selection, uniform_diameter):
summary = read_finite_json(path / case / "summary.json")
if (
summary.get("observation_mode") != observation_mode
any(summary.get(k, default) != expected for k, (expected, default) in options.items())
or summary.get("observe_cv", False) != observe_cv
or (observe_cv and summary.get("cv_samples") != 250)
or summary.get("jet_scenario", "none") != jet_scenario
or summary.get("observation_mode") != observation_mode
or summary.get("collision") != collision
or summary.get("case") != case
or summary.get("mode") != "pilot"
@@ -537,7 +728,10 @@ def verify_pilot(path, manifest, collision="SRT", observation_mode="last_base_st
raise ValueError("Unhealthy pilot observations")
ready = read_finite_json(path / case / "ready.json")
if (
ready.get("observation_mode") != observation_mode
any(ready.get(k, default) != expected for k, (expected, default) in options.items())
or ready.get("observe_cv", False) != observe_cv
or ready.get("jet_scenario", "none") != jet_scenario
or ready.get("observation_mode") != observation_mode
or ready.get("collision") != collision
or ready.get("case") != case
or ready.get("mode") != "pilot"
@@ -555,9 +749,9 @@ def verify_pilot(path, manifest, collision="SRT", observation_mode="last_base_st
or not 0 < stepping <= wall
):
raise ValueError("Invalid pilot calibration timings")
# Full physical duration is 60 times pilot; dense pilot sampling makes
# Full duration is 60 (or 61 with reversal) times pilot; dense sampling makes
# this conservative for the sparser early full observations. Not a guarantee.
calibrated_seconds += startup + 60 * (wall - startup)
calibrated_seconds += startup + (61 if jet_scenario == "reverse" else 60) * (wall - startup)
return 3 * calibrated_seconds
@@ -566,6 +760,13 @@ def main():
parser.add_argument("--exchange-window", action="store_true",
help="Opt in to refined window-ledger observations on the pinned candidate")
parser.add_argument("--collision", choices=["SRT", "MRT"], default="SRT")
parser.add_argument("--case", choices=["both", "uniform", "refined"], default="both")
parser.add_argument("--uniform-diameter", type=int, choices=[80, 160], default=80)
parser.add_argument("--placement", choices=list(PLACEMENTS), default="wide")
parser.add_argument("--jet-scenario", choices=["none", "zero", "fixed", "reverse"],
default="none", help="Opt in to public unshifted MRT Jets; reverse switches at t*=300 and ends at 305")
parser.add_argument("--observe-cv", action="store_true",
help="Record independent cropped-macro CV samples during pilot or full t*=290..300")
parser.add_argument("--source", type=Path, default=DEFAULT_SOURCE)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--mode", choices=["plan", "pilot", "full"], default="plan")
@@ -577,15 +778,26 @@ def main():
help="Required explicit campaign wall-time ceiling for GPU execution",
)
a = parser.parse_args()
if a.uniform_diameter == 160 and a.case != "uniform":
raise ValueError("D160 requires --case uniform; no matched-resolution claim")
if a.placement == "compact" and a.case != "refined":
raise ValueError("Compact placement requires --case refined")
if (a.uniform_diameter != 80 or a.placement != "wide") and a.jet_scenario == "none":
raise ValueError("Resolution/placement variants require an explicit public Jet scenario")
a.source = a.source.resolve()
a.observation_mode = "window" if a.exchange_window else "last_base_step"
manifest = verify_source(a.source, a.collision, a.observation_mode)
manifest = verify_source(a.source, a.collision, a.observation_mode, a.jet_scenario, a.observe_cv)
h, periodic = helpers(a.source)
from source_provenance import checkout_provenance
a.provenance = checkout_provenance(a.source)
a.provenance.update(jet_scenario=a.jet_scenario, ddf_shifting=a.jet_scenario == "none")
a.provenance.update(case_selection=a.case, uniform_diameter=a.uniform_diameter, placement=a.placement)
if a.observe_cv:
a.provenance["observe_cv"] = True
calibration = (
verify_pilot(a.pilot_results, manifest, a.collision, a.observation_mode)
verify_pilot(a.pilot_results, manifest, a.collision, a.observation_mode, a.jet_scenario, a.observe_cv,
a.case, a.uniform_diameter, a.placement)
if a.mode == "full" else None
)
if a.mode != "plan" and (
@@ -602,17 +814,31 @@ def main():
if a.output.exists() and any(a.output.iterdir()):
raise ValueError("Output must be empty/new")
a.output.mkdir(parents=True, exist_ok=True)
specification = plan(h, a.collision, a.observation_mode)
specification = plan(h, a.collision, a.observation_mode, a.jet_scenario, a.uniform_diameter, a.placement)
specification.update(
source_manifest=manifest,
**a.provenance,
gates=periodic.GATES,
reference_gates=periodic.REFERENCE_GATES,
gates=periodic.GATES if a.jet_scenario in ("none", "zero") else None,
reference_gates=periodic.REFERENCE_GATES if a.jet_scenario in ("none", "zero") else None,
runner_sha256=runner_hash(),
wall_time_budget_seconds=budget,
full_pilot_calibration_seconds=calibration,
)
host_geometry(a.source, h, a.collision)
if a.observe_cv:
specification["cv_observation"] = dict(
full_tstar=[290, 300], pilot_tstar=[.02, 5], spacing_tstar=.02,
refined_contours_root=cv_contours(True, 40, a.placement).tolist(),
uniform_contours_root=cv_contours(False, a.uniform_diameter, a.placement).tolist(),
uniform_grid_origin_root=uniform_origin(a.uniform_diameter),
refined_grid_origin_root=specification["fine_origin_root_edge"],
cell_width_root={"uniform": 40/a.uniform_diameter, "refined": .5},
viscosity_root=.004, components=["mass", "Px", "Py"],
units="root lattice per unit depth; outward flux per root time",
interval="Impulse belongs to the interval ending at each snapshot; first retained endpoint (pilot .02, full 290) supplies initial inventory only: exclude its preceding impulse",
qualification="not assessed; trapezoidal integration and time-coarsening are offline",
)
if a.case != "uniform":
host_geometry(a.source, h, a.collision, a.jet_scenario, a.placement)
(a.output / "plan.json").write_text(json.dumps(specification, indent=2))
if a.mode == "plan":
return 0
@@ -621,8 +847,9 @@ def main():
"GPU execution blocked: root GPU gates approval and explicit device selection required"
)
results = []
cases = selected_cases(a.case, a.uniform_diameter)
a.deadline = time.monotonic() + budget
for case in ("uniform", "refined"):
for case, _ in cases:
try:
result = run_case(a, h, periodic, case)
except TimeoutError as exc:
@@ -650,7 +877,12 @@ def main():
status="pilot_only" if a.mode == "pilot" else "incomplete",
matched_case_pass=False,
)
if len(results) == 2 and all(r["healthy"] and r["periodic"] for r in results):
if a.jet_scenario in ("fixed", "reverse"):
comparison.update(status=("pilot_only" if a.mode == "pilot" else "driven_diagnostics_complete")
if len(results) == len(cases) and all(r["healthy"] for r in results) else "invalid_state",
qualification="not assessed; execution completion is not physical acceptance",
jet_diagnostic_windows={r["case"]: r["jet_diagnostic_windows"] for r in results})
if a.case == "both" and len(results) == 2 and all(r["healthy"] and r["periodic"] for r in results):
comparison.update(
status="periodic_comparison",
refined_vs_uniform=periodic.compare(
@@ -660,14 +892,20 @@ def main():
comparison["matched_case_pass"] = comparison["refined_vs_uniform"][
"reference_agreement"
]
if a.case != "both":
comparison.update(status="single_case_" + results[0]["status"],
own_reference_comparison=results[0].get("reference_comparison"))
(a.output / "comparison.json").write_text(
json.dumps(comparison, indent=2, allow_nan=False)
)
return (
0
if len(results) == 2
if len(results) == len(cases)
and all(r["healthy"] for r in results)
and (a.mode == "pilot" or comparison["matched_case_pass"])
and (a.mode == "pilot" or comparison["matched_case_pass"]
or a.jet_scenario in ("fixed", "reverse")
or (a.case != "both" and results[0]["periodic"]
and results[0].get("reference_comparison", {}).get("reference_agreement", False)))
else 2
)
+96
View File
@@ -13,3 +13,99 @@ def checkout_provenance(repo):
text=True,
)
return dict(checkout_revision=revision, checkout_dirty=bool(dirty))
def final_source_check(recorded, recheck):
"""Re-derive the source identity after a run and compare it with the recorded one.
Returns receipt fields: source_recheck "UNCHANGED"; "CHANGED" with source_after; or
"RECHECK_FAILED" (source_after None) with the error. Callers mark a receipt complete
only for "UNCHANGED"; anything else invalidates the run.
"""
try:
after = recheck()
except Exception as error: # a failed recheck cannot confirm the source: invalid too
return dict(source_recheck="RECHECK_FAILED", source_after=None,
source_recheck_error=f"{type(error).__name__}: {error}")
if after == recorded:
return dict(source_recheck="UNCHANGED")
return dict(source_recheck="CHANGED", source_after=after, source_recheck_error=None)
def reject_output_inside(output, repo):
"""Refuse run outputs inside the checkout before any work: they would dirty it."""
from pathlib import Path
output, repo = Path(output).resolve(), Path(repo).resolve()
if output == repo or repo in output.parents:
raise SystemExit(f"--output must be outside the source checkout {repo}")
class HistoryUnavailable(RuntimeError):
"""The clone cannot show the recorded commit (no git, shallow clone, installed copy)."""
def historical_sha256(repo, commit, paths):
"""SHA-256 of each path as stored in `commit`, read with git, never from the working tree.
Raises HistoryUnavailable when git or the commit object is missing, so callers can
report an unchecked integrity record explicitly instead of passing or failing silently.
"""
import hashlib
git = ["git", "-C", str(repo)]
try:
subprocess.run([*git, "cat-file", "-e", f"{commit}^{{commit}}"], check=True,
capture_output=True)
except (OSError, subprocess.CalledProcessError) as error:
raise HistoryUnavailable(f"commit {commit} is not available in {repo}") from error
digests = {}
for path in paths:
listed = subprocess.run([*git, "ls-tree", "--name-only", commit, "--", path],
capture_output=True, text=True)
if listed.returncode:
raise HistoryUnavailable(f"tree of {commit} is not available in {repo}")
if listed.stdout.strip() != path:
raise ValueError(f"{path} is not part of {commit}")
shown = subprocess.run([*git, "show", f"{commit}:{path}"], capture_output=True)
if shown.returncode: # e.g. a blobless partial clone without network access
raise HistoryUnavailable(f"content of {path} at {commit} is not available in {repo}")
digests[path] = hashlib.sha256(shown.stdout).hexdigest()
return digests
def verify_manifest(repo, commit, digests):
"""Raise RuntimeError naming the first path whose recorded digest differs at `commit`."""
actual = historical_sha256(repo, commit, digests)
for path, digest in digests.items():
if actual[path] != digest:
raise RuntimeError(f"Historical manifest mismatch at {commit}: {path}")
return actual
def historical_tree(repo, commit, prefix="src"):
"""Tracked file names under `prefix` at `commit` (HistoryUnavailable if missing)."""
historical_sha256(repo, commit, [])
return subprocess.check_output(["git", "-C", str(repo), "ls-tree", "-r", "--name-only",
commit, "--", prefix], text=True).splitlines()
def historical_files(repo, commit, paths, dest):
"""Write each path as stored in `commit` under `dest` (HistoryUnavailable if missing)."""
from pathlib import Path
historical_sha256(repo, commit, paths) # HistoryUnavailable / ValueError before writing
for path in paths:
shown = subprocess.run(["git", "-C", str(repo), "show", f"{commit}:{path}"],
capture_output=True)
if shown.returncode:
raise HistoryUnavailable(f"cannot read {commit}:{path} (blob not in this clone)")
target = Path(dest) / path
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(shown.stdout)
def history_unavailable(testcase, error):
"""Skip with the reason, or fail when CELERIS_REQUIRE_HISTORY=1 (release gating)."""
import os
message = f"historical integrity NOT checked: {error}"
if os.environ.get("CELERIS_REQUIRE_HISTORY") == "1":
raise AssertionError(message)
testcase.skipTest(message)
@@ -73,7 +73,31 @@ def valid_builds(record, n, depth):
return True
def assess(records, expected_revision=None):
# run_case.py (sha256) of the runners before the final source recheck (git: 9805383,
# 2cf9853 = the a135252 qualification runner, db3d098); they wrote schema 1 receipts.
LEGACY_RUN_CASE_SHA256 = {
"9805383": "fbceab8c1aaaf357195a4fb9e6a699a195326e480e274692a277ce3df54ee6d9",
"2cf9853": "0789d2e1ffa581078bd11e6251b660b2c268363171fda0c6b26ac41205f1de3a",
"db3d098": "618c14ba94584aa9c0b714e0e2064a746d2b882973a13424f09dd7e54c68f7f1",
}
def rechecked(r, current):
"""Schema 2 receipts need source_recheck UNCHANGED; a missing recheck is accepted only
for frozen-mode schema 1 receipts written by a verified legacy runner."""
if "invalid" in r:
return False
if r.get("schema") == 2:
return r.get("source_recheck") == "UNCHANGED"
harness = r.get("source", {}).get("harness_sha256") or {}
return (not current and r.get("schema") == 1 and "source_recheck" not in r
and harness.get("run_case.py") in LEGACY_RUN_CASE_SHA256.values())
def assess(records, expected_revision=None, current=False):
"""current=True assesses current-checkout receipts (run_case --current-checkout) with
the same gates; it never accepts them as the frozen a135252 qualification, and the
frozen mode never accepts current-checkout receipts."""
indexed, duplicates = {}, []
for record in records:
key = tuple(record.get(k) for k in ("re_L", "n", "depth", "explosion"))
@@ -87,11 +111,19 @@ def assess(records, expected_revision=None):
sources = [r.get("source", {}) for r in records]
checks["same_source_and_harness"] = bool(sources) and len({
json.dumps(s, sort_keys=True) for s in sources}) == 1
checks["clean_a135252_numerical_source"] = bool(sources) and all(
s.get("dirty") is False and s.get("source_tree_sha256") == SOURCE_SHA256
and bool(s.get("revision")) and bool(s.get("harness_sha256"))
and s.get("build_capture_sha256") == BUILD_CAPTURE_SHA256
and (expected_revision is None or s.get("revision") == expected_revision) for s in sources)
if current:
checks["clean_current_checkout_source"] = bool(sources) and expected_revision is not None and all(
s.get("dirty") is False and s.get("source_mode") == "current_checkout"
and bool(s.get("source_tree_sha256")) and bool(s.get("harness_sha256"))
and s.get("build_capture_sha256") == BUILD_CAPTURE_SHA256
and s.get("revision") == expected_revision for s in sources)
else:
checks["clean_a135252_numerical_source"] = bool(sources) and all(
s.get("dirty") is False and s.get("source_tree_sha256") == SOURCE_SHA256
and "source_mode" not in s
and bool(s.get("revision")) and bool(s.get("harness_sha256"))
and s.get("build_capture_sha256") == BUILD_CAPTURE_SHA256
and (expected_revision is None or s.get("revision") == expected_revision) for s in sources)
runtimes = [r.get("runtime_identity") for r in records]
checks["same_recorded_runtime"] = bool(runtimes) and all(
isinstance(v, dict) and all(v.get(k) for k in
@@ -125,8 +157,15 @@ def assess(records, expected_revision=None):
and r.get("mach") == U * math.sqrt(3)
)
checks[label + " actual_builds"] = valid_builds(r, n, depth)
checks[label + " reference"] = (r.get("numerical_revision") == NUMERICAL_REVISION
and r.get("reference_harness") == REFERENCE_HARNESS)
if current:
checks[label + " reference"] = (r.get("numerical_revision") is None
and r.get("source_mode") == "current_checkout"
and r.get("reference_harness") == REFERENCE_HARNESS)
else:
checks[label + " reference"] = (r.get("numerical_revision") == NUMERICAL_REVISION
and "source_mode" not in r
and r.get("reference_harness") == REFERENCE_HARNESS)
checks[label + " source_recheck"] = rechecked(r, current)
checks[label + " numerics"] = r.get("numerics") == NUMERICS and r.get("initialization") == INITIALIZATION
runtime = r.get("runtime_source", {})
package = Path(runtime.get("checkout", "")) / "src/CelerisLab"
@@ -185,7 +224,10 @@ def assess(records, expected_revision=None):
valid_case_and_controls=valid, baseline_passed=baseline_passed)
return dict(passed=all(checks.values()), missing=missing, duplicates=duplicates, checks=checks,
comparisons=comparisons, scheme_results=scheme_results, early_pressure=early,
numerical_revision=NUMERICAL_REVISION, source_tree_sha256=SOURCE_SHA256,
source_mode="current_checkout" if current else "frozen",
numerical_revision=None if current else NUMERICAL_REVISION,
source_tree_sha256=(sources[0].get("source_tree_sha256") if current and sources
else None if current else SOURCE_SHA256),
reference_harness=REFERENCE_HARNESS, expected_revision=expected_revision,
floors=FLOORS, baseline_factor=BASELINE_FACTOR,
main_statistic="maximum of four post-initial samples at t*=.12,.24,.36,.48",
@@ -199,10 +241,13 @@ if __name__ == "__main__":
p.add_argument("files", type=Path, nargs="+")
p.add_argument("--output", type=Path, required=True)
p.add_argument("--expected-revision", required=True)
p.add_argument("--current-checkout", action="store_true",
help="assess current-checkout receipts; never the frozen qualification")
args = p.parse_args()
if args.output.exists():
p.error("Output already exists; choose a new path to retain prior evidence")
result = assess([json.loads(path.read_text()) for path in args.files], args.expected_revision)
result = assess([json.loads(path.read_text()) for path in args.files], args.expected_revision,
current=args.current_checkout)
args.output.write_text(json.dumps(result, indent=2, allow_nan=False) + "\n")
print(json.dumps(result, indent=2, allow_nan=False))
raise SystemExit(0 if result["passed"] else 1)
+40 -5
View File
@@ -64,8 +64,16 @@ def provenance(repo):
)
def verify_source(repo, revision):
"""Exact clean harness commit, fixed production tree, and reviewed capture helper."""
CURRENT_CAPTURE = "current checkout, not the frozen a135252 qualification"
def verify_source(repo, revision, current=False):
"""Exact clean harness commit, fixed production tree, and reviewed capture helper.
current=True records the clean checkout itself instead of requiring the frozen
numerical tree; such receipts carry source_mode="current_checkout" and are only
assessed with assess(..., current=True), never as the frozen qualification.
"""
import oracle
from tests.validation.multigrid import check_projected_pairs as gate
@@ -77,6 +85,17 @@ def verify_source(repo, revision):
if (Path(gate.__file__).resolve() != helper
or hashlib.sha256(helper.read_bytes()).hexdigest() != BUILD_CAPTURE_SHA256):
raise RuntimeError("Wrong build-capture helper")
if current:
result = provenance(repo)
if result["revision"] != revision or result["dirty"]:
raise ValueError("requires the exact expected clean checkout")
result.update(source_mode="current_checkout", source_capture=CURRENT_CAPTURE,
build_capture_sha256=BUILD_CAPTURE_SHA256)
return result
if subprocess.run(["git", "-C", str(repo), "cat-file", "-e", f"{NUMERICAL_REVISION}^{{commit}}"],
capture_output=True).returncode:
raise RuntimeError(f"frozen numerical revision {NUMERICAL_REVISION} is not available "
"in this clone; the frozen qualification cannot be checked")
if gate.preflight(repo, revision, "candidate") != NUMERICAL_REVISION:
raise RuntimeError("Wrong production revision")
result = provenance(repo)
@@ -149,6 +168,8 @@ def main():
p.add_argument("--depth", type=int, choices=[1, 2, 3], required=True)
p.add_argument("--output", type=Path, required=True)
p.add_argument("--explosion", choices=["constant", "projected_linear"], default="constant")
p.add_argument("--current-checkout", action="store_true",
help="record the clean current checkout instead of the frozen a135252 tree")
p.add_argument("--early-pressure", action="store_true",
help="also save 65 scalar pressure samples at t*=k*.001875, k=0..64")
args = p.parse_args()
@@ -161,7 +182,9 @@ def main():
p.error("--run-gpu and exactly one explicit CUDA_VISIBLE_DEVICES are required")
repo = args.repo.resolve()
sys.path[:0] = [str(repo / "src"), str(repo)]
source = verify_source(repo, args.expected_revision)
from tests.validation.source_provenance import reject_output_inside
reject_output_inside(args.output, repo)
source = verify_source(repo, args.expected_revision, current=args.current_checkout)
runtime = runtime_source(repo)
if args.dry_run:
print(json.dumps(dict(source=source, runtime_source=runtime, dry_run=True)))
@@ -180,9 +203,10 @@ def main():
masks = [k == ACTIVE for k in h.kinds]
main_steps, early_steps = sample_steps(args.n, args.early_pressure)
report = dict(
schema=1,
schema=2, # 2: final source recheck (source_recheck) is recorded
source=source,
numerical_revision=NUMERICAL_REVISION,
numerical_revision=None if args.current_checkout else NUMERICAL_REVISION,
**({"source_mode": "current_checkout"} if args.current_checkout else {}),
reference_harness=REFERENCE_HARNESS,
actual_build_sidecars=[],
runtime_identity=None,
@@ -297,6 +321,17 @@ def main():
),
flush=True,
)
from tests.validation.source_provenance import final_source_check
recheck = final_source_check(
dict(source=source, runtime_source=runtime),
lambda: dict(source=verify_source(repo, args.expected_revision,
current=args.current_checkout),
runtime_source=runtime_source(repo)))
report.update(recheck)
if recheck["source_recheck"] != "UNCHANGED":
report.update(invalid="SOURCE_CHANGED_DURING_RUN") # completed stays False
save()
raise RuntimeError("Source changed during the run; receipt marked invalid")
report.update(
completed=True,
seconds=time.monotonic() - started,
@@ -12,13 +12,13 @@ import sys
import tempfile
from types import SimpleNamespace
import unittest
from unittest.mock import patch
from unittest.mock import call, patch
import numpy as np
import run_case
from assess_projected_pairs import (BUILD_SOURCES, CASES, NUMERICS, NUMERICAL_REVISION,
SOURCE_SHA256, assess)
LEGACY_RUN_CASE_SHA256, SOURCE_SHA256, assess)
from oracle import LIMITS, TIMES, U, rectangles
REPO = Path(__file__).resolve().parents[3]
@@ -31,7 +31,7 @@ RUNTIME = dict(device=dict(name="host mock", pci_bus_id="mock", capability=[7, 0
def source():
return dict(revision=MOCK_REVISION, dirty=False, source_tree_sha256=SOURCE_SHA256,
harness_sha256={"run_case.py": "a" * 64},
harness_sha256={"run_case.py": LEGACY_RUN_CASE_SHA256["2cf9853"]},
build_capture_sha256=run_case.BUILD_CAPTURE_SHA256)
@@ -53,7 +53,7 @@ def records(early=False):
h = build_hierarchy(cfg, refinements)
main_steps, early_steps = run_case.sample_steps(n, early)
out.append(dict(
re_L=re, n=n, depth=depth, explosion=scheme, U=U, mach=float(U*np.sqrt(3)),
schema=1, re_L=re, n=n, depth=depth, explosion=scheme, U=U, mach=float(U*np.sqrt(3)),
viscosity=cfg.viscosity, tau_by_level=[0.5+3*c.viscosity for c in h.configs],
refinements=refinements, numerics=NUMERICS.copy(), initialization=run_case.INITIALIZATION,
source=source(), numerical_revision=NUMERICAL_REVISION,
@@ -291,6 +291,105 @@ class ProjectedPairsTests(unittest.TestCase):
else:
self.assertEqual(run_case.verify_source(REPO, expected), source())
def test_frozen_mode_reports_missing_history_not_a_diff(self):
from tests.validation.multigrid import check_projected_pairs as gate
missing = SimpleNamespace(returncode=128)
with patch.object(run_case.subprocess, "run", return_value=missing), \
patch.object(gate, "preflight") as preflight:
with self.assertRaisesRegex(RuntimeError, "not available in this clone"):
run_case.verify_source(REPO, MOCK_REVISION)
preflight.assert_not_called()
def test_current_checkout_source_is_recorded_not_frozen(self):
from tests.validation.multigrid import check_projected_pairs as gate
current = dict(source(), source_tree_sha256="c" * 64)
with patch.object(run_case, "provenance", return_value=dict(current)), \
patch.object(gate, "preflight") as preflight:
result = run_case.verify_source(REPO, MOCK_REVISION, current=True)
preflight.assert_not_called()
self.assertEqual(result["source_mode"], "current_checkout")
self.assertEqual(result["source_capture"], run_case.CURRENT_CAPTURE)
for bad in (dict(current, dirty=True), dict(current, revision=OLD_REVISION)):
with self.subTest(bad=bad), patch.object(run_case, "provenance", return_value=bad):
with self.assertRaisesRegex(ValueError, "exact expected clean checkout"):
run_case.verify_source(REPO, MOCK_REVISION, current=True)
def test_current_and_frozen_receipts_never_cross_assess(self):
frozen = records()
current = deepcopy(frozen)
for r in current:
r["source"] = dict(r["source"], source_tree_sha256="c" * 64,
source_mode="current_checkout",
source_capture=run_case.CURRENT_CAPTURE)
r["numerical_revision"], r["source_mode"] = None, "current_checkout"
r["schema"], r["source_recheck"] = 2, "UNCHANGED"
ok = assess(current, MOCK_REVISION, current=True)
self.assertTrue(ok["checks"]["clean_current_checkout_source"])
self.assertTrue(all(v for k, v in ok["checks"].items() if k.endswith(" reference")))
self.assertEqual((ok["source_mode"], ok["numerical_revision"]), ("current_checkout", None))
self.assertEqual(ok["source_tree_sha256"], "c" * 64)
self.assertFalse(assess(current, MOCK_REVISION)["checks"]["clean_a135252_numerical_source"])
self.assertFalse(assess(current, MOCK_REVISION)["passed"])
self.assertFalse(assess(frozen, MOCK_REVISION, current=True)["checks"]["clean_current_checkout_source"])
self.assertFalse(assess(current, None, current=True)["checks"]["clean_current_checkout_source"])
self.assertTrue(assess(frozen, MOCK_REVISION)["checks"]["clean_a135252_numerical_source"])
def test_assessor_requires_unchanged_final_source(self):
frozen = records() # schema 1 receipts from the verified legacy runner 2cf9853
label = lambda r: f"{r['n']}/{r['depth']}/{r['explosion']} source_recheck"
self.assertTrue(all(assess(frozen, MOCK_REVISION)["checks"][label(r)] for r in frozen))
new = deepcopy(frozen)
for r in new:
r.update(schema=2, source_recheck="UNCHANGED")
self.assertTrue(all(assess(new, MOCK_REVISION)["checks"][label(r)] for r in new))
unknown_runner = dict(frozen[0]["source"], harness_sha256={"run_case.py": "a" * 64})
for case, mutate in (
("schema 1, unverified runner", dict(source=unknown_runner)),
("schema 2 without recheck", dict(schema=2)),
("schema 2 changed", dict(schema=2, source_recheck="CHANGED")),
("schema 2 recheck failed", dict(schema=2, source_recheck="RECHECK_FAILED")),
("schema 1 with a recheck field", dict(source_recheck="UNCHANGED")),
("unknown schema", dict(schema=3, source_recheck="UNCHANGED")),
("no schema", dict(schema=None)),
("invalid", dict(schema=2, source_recheck="UNCHANGED",
invalid="SOURCE_CHANGED_DURING_RUN")),
):
with self.subTest(case):
bad = deepcopy(frozen)
bad[0].update(mutate)
out = assess(bad, MOCK_REVISION)
self.assertFalse(out["checks"][label(bad[0])])
self.assertFalse(out["passed"])
current = deepcopy(frozen)
for r in current:
r["source"] = dict(r["source"], source_mode="current_checkout")
r["numerical_revision"], r["source_mode"] = None, "current_checkout"
# Legacy acceptance is frozen-mode only.
self.assertFalse(assess(current, MOCK_REVISION, current=True)["checks"][label(current[0])])
def test_legacy_runner_hashes_are_the_recorded_commits(self):
sys.path.insert(0, str(REPO))
from tests.validation.source_provenance import (HistoryUnavailable, historical_sha256,
history_unavailable)
try:
for commit, digest in LEGACY_RUN_CASE_SHA256.items():
path = "tests/validation/taylor_green/run_case.py"
self.assertEqual(historical_sha256(REPO, commit, [path])[path], digest)
except HistoryUnavailable as error:
history_unavailable(self, error)
current = hashlib.sha256((REPO / "tests/validation/taylor_green/run_case.py").read_bytes())
self.assertNotIn(current.hexdigest(), LEGACY_RUN_CASE_SHA256.values())
def test_output_inside_checkout_is_refused_before_source_check(self):
argv = ["run_case.py", "--repo", str(REPO), "--re", "100", "--n", "128", "--depth", "1",
"--output", str(REPO / "tg-out.json"), "--expected-revision", MOCK_REVISION,
"--dry-run"]
with patch.object(sys, "argv", argv), \
patch.object(run_case, "verify_source", side_effect=AssertionError("work started")):
with self.assertRaisesRegex(SystemExit, "outside the source checkout"):
run_case.main()
self.assertFalse((REPO / "tg-out.json").exists())
def test_explicit_opt_in_selector_and_dry_run_without_cuda(self):
with tempfile.TemporaryDirectory() as tmp:
output = Path(tmp) / "case.json"
@@ -319,7 +418,7 @@ class ProjectedPairsTests(unittest.TestCase):
patch.object(run_case, "runtime_identity", side_effect=AssertionError("CUDA reached")), \
redirect_stdout(stdout):
self.assertEqual(run_case.main(), 0)
verify.assert_called_once_with(REPO, MOCK_REVISION)
verify.assert_called_once_with(REPO, MOCK_REVISION, current=False)
self.assertTrue(json.loads(stdout.getvalue())["dry_run"])
self.assertFalse(output.exists())
@@ -351,7 +450,8 @@ class ProjectedPairsTests(unittest.TestCase):
# Replace the GPU context and steppers before import. No CUDA module or flow is used.
for early, depth, fail in ((False, 1, None), (False, 2, None), (True, 2, None),
(True, 2, "resource"), (True, 2, "nonfinite"),
(True, 2, "probe_nonfinite")):
(True, 2, "probe_nonfinite"),
(False, 2, "changed"), (False, 1, "recheck_error")):
calls, selectors = [], []
class FakeRun:
@@ -406,13 +506,31 @@ class ProjectedPairsTests(unittest.TestCase):
patch.object(run_case, "runtime_identity", return_value=deepcopy(RUNTIME)), \
patch.object(run_case, "build_capture", return_value=nullcontext()), \
patch.object(run_case, "metrics", side_effect=metric), redirect_stdout(io.StringIO()):
if fail:
if fail in ("changed", "recheck_error"):
verify.side_effect = [source(), dict(source(), source_tree_sha256="d" * 64)
if fail == "changed" else RuntimeError("Wrong numerical source fingerprint")]
with self.assertRaisesRegex(RuntimeError, "Source changed during the run"):
run_case.main()
elif fail:
with self.assertRaises(MemoryError if fail == "resource" else FloatingPointError):
run_case.main()
else:
self.assertEqual(run_case.main(), 0)
receipt = json.loads(output.read_text())
verify.assert_called_once_with(REPO, MOCK_REVISION)
rechecked = fail in (None, "changed", "recheck_error")
self.assertEqual(verify.call_args_list,
[call(REPO, MOCK_REVISION, current=False)] * (2 if rechecked else 1))
if fail in ("changed", "recheck_error"):
# Full trajectory, but invalid: never a complete-looking receipt.
self.assertEqual(receipt["source_recheck"],
"CHANGED" if fail == "changed" else "RECHECK_FAILED")
self.assertEqual(receipt["invalid"], "SOURCE_CHANGED_DURING_RUN")
self.assertFalse(receipt["completed"])
self.assertNotIn("screening_pass", receipt)
self.assertEqual(receipt["source_after"] is None, fail == "recheck_error")
continue
self.assertEqual(receipt.get("source_recheck"), None if fail else "UNCHANGED")
self.assertEqual(receipt["schema"], 2)
self.assertEqual(selectors, ["projected_linear" if early else "constant"] if depth == 2 else [])
self.assertEqual(receipt["source"], source())
self.assertEqual(receipt["runtime_identity"], RUNTIME)