9.7 KiB
DFG 2D-2 Re100 GPU campaign (validation-only)
Current experimental source provenance
This development branch now includes the reviewed AMR-channel source changes originating in 5338864748ac67fe2544487c059ba149ebf916da and 47b05bc8abfc957fe876b0c63bdb386e3c6cbaa5. The current full-source manifest pins the latter reviewed source snapshot, including the new channel files. This is an explicit new experiment; neither source review nor host tests transfer historical d79 GPU/flow acceptance to it. The channel GPU gate has not yet run at integration time.
New run JSON records the source-origin pin plus checkout_revision (actual Git HEAD), checkout_dirty, and the experimental-source label. The origin pin is not claimed to be the run's checkout commit. Numerical definitions and thresholds are unchanged. Runs require a Git checkout for this runtime identity; source hash checks remain active.
Historical d79 Re100/Jet execution stays reproducible at 69e9b78de9483a5ff21e2d0c74311d240bfaf88c; the d79 LES diagnostic checkpoint is 20ae6770924df15f31305aad56e44a54a3dd4d88. Their manifests and existing result files are not rewritten. Use those commits to reproduce or interpret old d79 evidence. The current AMR-channel experiment requires its own GPU and applicable flow evidence.
Scope and locked definitions
Explicitly pinned AMR-channel experimental production snapshot (see provenance below). Uniform D2Q9/MRT, shifted FP32, double buffer, no LES, stationary cylinder and no-slip walls. This is not an AMR, jet, or full canonical-outlet certification.
Physical geometry: [0,2.2] × [0,0.41], center (0.2,0.2), radius .05, rho=1, nu=.001, inlet mean=1 and peak=1.5. D is lattice diameter; dx=.1/D, dt=U_latticedx, nu_lattice=U_latticeD/100. U_lattice=.01 gives peak inlet Mach .02598. Existing local Zou–He inlet is selected in validation config only. Existing pressure-NEQ outlet is unclamped: it does not impose the canonical vector do-nothing condition nud_n(u)-pn=0. The existing production channel-flow initial condition is used, not zero velocity. Periodic eligibility is delayed to t*=250 (25 physical seconds), with fixed end t*=300 (30 seconds).
Official source: https://wwwold.mathematik.tu-dortmund.de/~featflow/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark2_re100.html
The 20141124 official newer reference archive has higher-order line-integration force data and supersedes the older example table for this comparison. The compact tests/validation/data/dfg_re100/reference_cycle.json preserves the last complete lift-minimum cycle from lv6/dt4. Reference Cd=[3.164266,3.227392], Cl=[-1.021286,.986571], pressure difference=[2.420371,2.507013], St=.301841. These are derived sampled extrema, not exact analytical values. Official mean=(min+max)/2 and amplitude=max-min; separate time averages are also recorded.
Exact commands and startup ownership
Use the existing CUDA/PyCUDA environment; no package installation or production edits required. From the repository root, use a new task-owned output path:
export PYTHONPATH="$PWD/src${PYTHONPATH:+:$PYTHONPATH}"
python -m pytest -q tests/unit/test_re100.py
CUDA_VISIBLE_DEVICES=0 python tests/validation/run_dfg_re100.py --diameter 20 --speed .01 --mode pilot --output output/validation/pilot_d20_u001 > pilot.log 2>&1
Pilot is t*=5 (10,000 steps), with dense .02-t* sampling to calibrate diagnostic cost. It cannot pass periodicity or external benchmark validation. Root launch owner reviews health, memory, timing and stdout before full runs; the full pair is not launched automatically.
CUDA_VISIBLE_DEVICES=0 python tests/validation/run_dfg_re100.py --diameter 40 --speed .01 --mode full --output output/validation/full_d40_u001 > d40.log 2>&1
CUDA_VISIBLE_DEVICES=1 python tests/validation/run_dfg_re100.py --diameter 80 --speed .01 --mode full --output output/validation/full_d80_u001 > d80.log 2>&1
These are separate invocations, not a parallel shell script. A single root launch owner serializes startup/compilation across all Horsea work. Do not launch the next process until the previous log contains an explicit READY {...} line. READY comes after the first physical sample, including Simulation initialize, sim.run, read_force and get_macroscopic, so lazy runtime/diagnostic paths are exercised. That sample remains in the physical trajectory/history. No hidden warmup/reset, no later new diagnostic kernel. Do not treat a process merely existing as READY. Keep separate output directories and retain exit codes/logs. GPU assignment is chosen by root after checking existing workloads; shown indices are examples of physical device selection.
Measurement, frozen criteria and outcomes
Force uses advancing link-momentum exchange, accumulated in batches <=100 steps, combined in FP64; Cd/Cl=2F/(U_lattice² D). Force timestamp is the batch midpoint. Pressure uses fluid-only h/2h extrapolation to cylinder front/back points and physical conversion 1/(3 U_lattice²), at each sample endpoint. For phase comparisons endpoint pressure is interpolated to the same cycle phase. Both raw times are saved.
Before t*=240 sample width=.5; afterwards .02 (about 166 samples per lift cycle). Pilot uses dense sampling throughout. Extract complete lift-minimum-to-minimum cycles using quadratic local minimum locations, accepting periods 2–5 t*. Latest five complete cycles must be consecutive with the last ending no more than 5 t* before the final observation, and must start at or after t*=250. Eligibility requires lift amplitude >=.2, period range/mean <=.005, Cd/Cl/pressure min and max ranges <=.005 absolute, cycle-mean mass range/mean <=1e-4, and each cycle's mean mass-flux imbalance <=.005. Instantaneous flux equality is not required in this unsteady compressible LBM diagnostic.
Reference gates: Cd extrema relative errors <=2%, Cl extrema absolute errors <=.03, pressure extrema relative errors <=2%, St relative error <=2%. These are predeclared engineering tolerances, not official uncertainty intervals. Phase-aligned waveform RMS errors, extrema, midpoint means, time means, amplitudes, outlet residual, sampled density bounds and Mach are all reported. Waveform RMS is diagnostic (no retroactive gate). No threshold changes after GPU results. At cap, failed periodicity stays incomplete. Invalid/nonfinite state, rho outside [.5,1.5], or sampled max Mach>.2 causes a safety stop; this is a gross-health guard, not a Mach-independence claim.
A case_reference_pass requires both periodicity and reference gates. Every raw case keeps suite_validated=false. The two-grid comparison must report each case and errors; matched-Mach D40/D80 is evidence of resolution sensitivity only, not an asymptotic convergence order. If either fails or finer errors increase materially, report the failure instead of adjusting thresholds. No full canonical/AMR/Mach-independent validation claim is allowed.
Resource and calibration plan
D20: 441×84=37,044 nodes, pilot 10,000 steps. D40: 881×166=146,246 nodes, 1,200,000 full steps. D80: 1761×330=581,130 nodes, 2,400,000 full steps. Main double-buffer DDF arrays alone cost 72 bytes/node: 2.7/10.5/41.8 MB decimal. Flags, masks, macro host arrays, body buffers and CUDA context/JIT memory add overhead; root verifies actual free GPU/host memory and disk before launch. Conservative per-process reservation: 1 GiB VRAM and 2 GiB host RAM, excluding shared/compiler overhead; measurements take precedence.
Historical Re20 D80 U=.01 step throughput was ~6082 MLUPS (45.9 seconds/480k steps). At equal throughput D80 Re100 full advancing steps would take ~229 seconds, excluding dense full-field downloads/reduction and compile overhead. D40 estimate must be calibrated, not assumed proportional. Pilot records startup_seconds, step_seconds, measurement_seconds, mlups. For each full grid, calibrate a same-grid pilot if D20 cannot predict cost adequately. Full has about 3480 records (480 coarse+3000 dense); use measured seconds/step × full steps plus seconds/record ×3480 plus startup. D20-to-D80 extrapolation is only a rough planning upper/lower guide, not an ETA commitment.
Only geometry.npz, one final_field.npz, history.csv, summary.json and stdout/stderr are persisted. No per-step or repeated full-field snapshots. D80 final float64 rho/ux/uy plus mask are <15 MB uncompressed; history/summary several MB, compact versioned reference/source guards <50 KiB. Budget <100 MB per case and <500 MB whole campaign including logs and optional externally retained archives, safely below 2 GiB; root still checks baseline Horsea usage. Retain sparse evidence only.
CPU validation and remaining execution
Ten tests cover geometry/units/config, pressure scaling, periodic and nonperiodic synthetic histories, immutable production hashes, and self-reference comparison. CUDA execution is not available in the preparation environment and has not been claimed. Run pilot first on Horsea; do not confuse successful CPU tests with GPU readiness.
Reference provenance and reproduction
Only the compact derived cycle is versioned. Its source URL, official ZIP SHA-256, column selection, and cycle-selection description are embedded in reference_cycle.json. Raw archives and GPU results are not repository inputs. To reproduce it, obtain the exact archive identified there, verify its SHA-256, and extract logs/results_cc2d_bench2_referencenew/bdforces_q2_lv6_dt4 and pointvalues_q2_lv6_dt4. Use physical time column 2 multiplied by 10, drag/lift columns 4/5, and pressure columns 7 minus 12 (one-based columns). Build rows with mass=flux_in=flux_out=1, then call periodic.extract(rows, min_time=0) and select the last complete cycle. The source guard is tests/validation/data/dfg_re100/source_manifest.json; it verifies the current reviewed production snapshot rather than equating its origin with the validation branch HEAD.