Compare commits

..
Author SHA1 Message Date
ClaudeandClaude Opus 5.5 888c80381c kernels (K10): refinement.cu becomes an include-only entry; patch kernels under step/
refinement.cu keeps its header, includes, the curved_double.cu entry
block, the #error guard and the optional CELERIS_REFINEMENT_AUDIT
counters, then includes step/patch_step.cu, which holds patch_collide,
PatchExplode, PatchFineStep and PatchCoarseStep moved verbatim (checked
against the previous refinement.cu) in the same order.
test_refinement_mrt reads refinement.cu + step/patch_step.cu; its
assertions are unchanged. The audit-build tests include refinement.cu
from the build root, so step/patch_step.cu still resolves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 02:03:28 +00:00
ClaudeandClaude Opus 5.5 bf5088f84c kernels (K9): blocks.cu becomes an include-only entry; kernels under step/
blocks.cu keeps its header, includes and #error guards and then includes
the kernels, as kernel_v2.cu does:
- step/block_transfer.cu: BlockHaloCopy, BlockExplode,
  BlockExplodeProjected, BlockCoalesce
- step/block_boundary.cu: BlockCurvedBoundary, BlockAccumulateExchange
- step/block_step.cu: BlockStep (rejects BLOCK_ROOT_BC == 2)
- step/block_channel.cu: unchanged (mode 2 only)
Kernel text moved verbatim (checked against the previous blocks.cu) and in
the same definition order, so the translation unit is the same sequence of
kernels. The two source-reading tests follow the move:
test_projected_explosion reads step/block_transfer.cu, test_refinement_mrt
reads blocks.cu + step/block_step.cu; their assertions are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 01:46:49 +00:00
ClaudeandClaude Opus 5.5 af406b44bf Merge K4 helper withdrawal into K5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 01:32:42 +00:00
ClaudeandClaude Opus 5.5 78c0f1811b kernels (K4): withdraw the address helpers; keep only fill_invalid
Horsea's gates showed block_ddf/block_slot change real PTX in
BlockHaloCopy, BlockExplode and BlockExplodeProjected (6/10 modules;
BlockExplodeProjected 38 -> 40 registers) although the results stayed
exact. The stack's bar is byte-identical PTX for naming-only steps, so
those three kernels return to their K3 expressions, the helpers leave
core/layout.cuh, and test_projected_explosion goes back to its K3 form.
fill_invalid stays (its kernels were byte-identical). K4 vs K3 is now
only the NaN-fill helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 01:32:40 +00:00
ClaudeandClaude Opus 5.5 0ddc6c3d62 Merge K4 test harness fix into K5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 00:35:09 +00:00
ClaudeandClaude Opus 5.5 43e226c59a test_projected_explosion: give the host harness the real block address helpers
K4 made BlockExplodeProjected address storage through block_ddf and
block_slot (core/layout.cuh). This host test compiles the kernel text
extracted from blocks.cu, so it now also extracts those two helper
definitions from core/layout.cuh (device qualifiers -> inline) instead of
failing to compile. Assertions and tolerances unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 00:35:07 +00:00
ClaudeandClaude Opus 5.5 93e692a40f kernels (K5): reuse the existing bounce_back_swap instead of a new copy
operators/helpers.cuh already had bounce_back_swap (the uniform steps use
it). It moves verbatim, #pragma unroll included, into its own
operators/bounce_back_swap.cuh, which helpers.cuh includes; the channel
root includes that header and calls it. bounce_back_swap_d2q9 from the
previous commit is removed. bounce_back.cuh keeps only the half-way rows
helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 00:32:07 +00:00
ClaudeandClaude Opus 5.5 33837697da kernels (K5): channel root reuses boundary and streaming helpers
- boundary/bounce_back.cuh: apply_wall_bb_d2q9_rows(y, top_row, ...) is
  the existing half-way y-wall correction with an explicit top row;
  apply_wall_bb_d2q9_y_pull keeps its signature and calls it with NY - 2.
  bounce_back_swap_d2q9(f) names the full opposite-pair swap.
- streaming/pull_double_buffer.cuh: load_node(fi, n, f) loads one node's
  stored populations.
- step/block_channel.cu uses all three instead of its inline copies: the
  wall swap, the outlet donor gather, and rows 1 / ny-2 (now one call with
  the physical top row; same y == 1 / y == top_row order and loads).
Same statements, branch order, indices and arithmetic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 00:31:26 +00:00
ClaudeandClaude Opus 5.5 b47af12c9d kernels (K4): blocked storage address helpers and one invalid-cell fill
- core/layout.cuh: block_ddf(f, block_cell[, cells]) returns the start of a
  block's populations and block_slot(block_cell, cells, q) the offset of one
  population; BlockHaloCopy, BlockExplode and BlockExplodeProjected use them
  instead of repeating (c / cells) * NQ * cells [+ q * cells + c % cells].
  BlockExplode keeps its loop-hoisted parent base; types stay uint64_t /
  unsigned long; the expressions are the same ones, now named.
- core/cell_kind.cuh: fill_invalid(f) writes the NaN marker; BlockStep,
  BlockChannelStep and PatchFineStep call it instead of three copies of the
  loop.
No change to operation order, types, launches or arithmetic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 00:29:03 +00:00
ClaudeandClaude Opus 5.5 a3df5855a2 kernels (K3): cell kinds in core/, capability guard first, self-contained block collision
- core/cell_kind.cuh holds KIND_ACTIVE/INNER/OUTER (was a constexpr in
  blocks.cu that step/block_channel.cu and the channel_root.cpp shim
  relied on or redefined); blocks.cu, block_channel.cu and the shim now
  include it.
- blocks.cu checks the lattice/model #error guard before the
  BLOCK_ROOT_BC guard and mode includes, so an unsupported build reports
  the real reason first.
- operators/block_collision.cuh includes what it uses (params, macro,
  SRT/MRT collision) instead of relying on blocks.cu's include order.
Preprocessor-only: same kernels, same statements, same arithmetic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 00:26:41 +00:00
Frank14f 11e96a188b Test historical build fingerprints with their pinned renderer 2026-10-21 18:50:15 +08:00
Frank14f 3a374faace Specialize blocked root boundary mode at compile time 2026-10-21 18:35:27 +08:00
25 changed files with 467 additions and 309 deletions
+12 -4
View File
@@ -54,14 +54,18 @@ def _write(path: str, content: str):
def generate_config(cfg: LBMConfig, n_objects: int = 0,
*, build_root: Optional[str] = None) -> str:
*, build_root: Optional[str] = None, block_root_bc: int = 0) -> str:
"""Snapshot kernels and render headers in a fresh private directory.
Returns its path. Pass it explicitly to compile_kernel for interleaved
builds. Legacy generate_config(); compile_kernel() remains context-local.
The caller owns cleanup_build_dir() once the loaded module is no longer
needed for recompilation/debugging. No package files are changed.
block_root_bc specializes a blocked root module; fine/dense builds use 0.
"""
if (isinstance(block_root_bc, bool) or not isinstance(block_root_bc, int)
or block_root_bc not in (0, 1, 2)):
raise ValueError("block_root_bc must be an integer mode 0, 1 or 2")
macros = cfg.to_macros()
if isinstance(n_objects, bool) or not isinstance(n_objects, int) or n_objects < 0:
raise ValueError("n_objects must be a nonnegative integer")
@@ -71,14 +75,15 @@ def generate_config(cfg: LBMConfig, n_objects: int = 0,
try:
shutil.copytree(_KERNEL_DIR, build_dir, dirs_exist_ok=True,
ignore=shutil.ignore_patterns("*.ptx", "*.cubin", "*.tmp"))
_render_config(macros, n_objects, os.path.join(build_dir, "config"))
_render_config(macros, n_objects, os.path.join(build_dir, "config"),
block_root_bc=block_root_bc)
digest = hashlib.sha256()
for path in sorted(Path(build_dir).rglob("*")):
if path.is_file():
digest.update(str(path.relative_to(build_dir)).encode())
digest.update(path.read_bytes())
metadata = {"source_sha256": digest.hexdigest(), "config": asdict(cfg),
"n_objects": n_objects}
"n_objects": n_objects, "block_root_bc": block_root_bc}
_write(os.path.join(build_dir, ".celeris-build"), json.dumps(metadata, sort_keys=True))
except BaseException:
shutil.rmtree(build_dir)
@@ -101,7 +106,7 @@ def cleanup_build_dir(build_dir: Optional[str]) -> None:
_last_ptx_path.set(None)
def _render_config(m: dict, n_objects: int, cdir: str):
def _render_config(m: dict, n_objects: int, cdir: str, *, block_root_bc: int = 0):
_write(os.path.join(cdir, "config_grid.h"), f"""{_HEADER}\
// Layer: Global / Grid
#ifndef CELERIS_CONFIG_GRID_H
@@ -165,6 +170,9 @@ def _render_config(m: dict, n_objects: int, cdir: str):
#define STORE_PRECISION {m['STORE_PRECISION']}
#define USE_DDF_SHIFTING {m['USE_DDF_SHIFTING']}
// Internal blocked root mode: 0=periodic/fine, 1=slot/reservoir, 2=channel.
#define BLOCK_ROOT_BC {block_root_bc}
#define USE_LES {m['USE_LES']}
#define LES_CS {m['LES_CS']}
#define LES_CLOSED_FORM {m['LES_CLOSED_FORM']}
+22 -173
View File
@@ -1,5 +1,6 @@
// CelerisLab/lbm/kernels/blocks.cu
// One periodic level of fixed-size blocks (lbm/blocks.py layout). Separate
// One fixed-size blocked level (lbm/blocks.py layout), specialized at compile
// time for periodic/fine, slot/reservoir or channel stepping. Separate
// compilation entry: NX, NY are the block window including halo, so index_f
// and linear_index address one block; blockIdx.z selects the block, whose
// populations start at b*NQ*TOTAL_CELLS (block-contiguous, Q-major inside).
@@ -12,6 +13,7 @@
#include "core/params.cuh"
#include "core/flags.cuh"
#include "core/obs.cuh"
#include "core/cell_kind.cuh"
#include "operators/macro.cuh"
#include "operators/collision_srt.cuh"
#if COLLISION_MODEL == 2
@@ -19,182 +21,29 @@
#endif
#include "streaming/pull_double_buffer.cuh"
#include "boundary/curved_boundary.cuh"
#include "boundary/planar_velocity.cuh"
#if DIM != 2 || NQ != 9 || (COLLISION_MODEL != 0 && COLLISION_MODEL != 2) || STORE_PRECISION != 0 || USE_LES || STREAMING_MODEL != 0
#error "Blocks require FP32 D2Q9 SRT or MRT double buffers without LES"
#endif
// Fixed-patch cell kinds, matching refinement.py ACTIVE/INNER_GHOST/OUTER_GHOST.
constexpr unsigned char KIND_ACTIVE = 0u, KIND_INNER = 1u, KIND_OUTER = 2u;
// Zero-initialized for ordinary blocks/fine levels. Set only for parent walls;
// NX/NY remain local storage strides, never physical wall coordinates.
struct BlockWall { unsigned int nx, ny, slot_width; float amplitude; };
extern "C" { __constant__ BlockWall d_block_wall; }
// Halo refresh: block cell ids b*TOTAL_CELLS+c, sources are always core cells.
extern "C" __global__ void BlockHaloCopy(
fpxx* f, const uint64_t* dst, const uint64_t* src, unsigned int count)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const uint64_t d = dst[row], s = src[row];
fpxx* to = f + (d / TOTAL_CELLS) * NQ * TOTAL_CELLS;
const fpxx* from = f + (s / TOTAL_CELLS) * NQ * TOTAL_CELLS;
for (unsigned int q = 0; q < NQ; ++q)
to[index_f(d % TOTAL_CELLS, q)] = from[index_f(s % TOTAL_CELLS, q)];
}
// Fixed-patch explosion into blocked fine storage: target = owner block cell
// id of a patch-ghost cell; donors are parent block cell ids. Dense parent
// storage is the one-window case with coarse_cells = full domain size.
extern "C" __global__ void BlockExplode(
const fpxx* coarse, fpxx* fine, const uint64_t* target, const uint64_t* donor,
unsigned int count, unsigned long coarse_cells)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const uint64_t t = target[row];
fpxx* to = fine + (t / TOTAL_CELLS) * NQ * TOTAL_CELLS;
const uint64_t d = donor[row];
const fpxx* from = coarse + (d / coarse_cells) * NQ * coarse_cells;
for (unsigned int q = 0; q < NQ; ++q)
to[index_f(t % TOTAL_CELLS, q)] = from[q * coarse_cells + d % coarse_cells];
}
// Experimental Schornbaum--Ruede (arXiv:1508.07982, Algorithm 2)
// and Chen et al. (doi:10.1016/j.physa.2005.09.036) population interpolation.
// Independent implementation of the mathematical stencil; no upstream source copied.
// Stencil cells are owner cores; a disabled axis repeats its center twice.
extern "C" __global__ void BlockExplodeProjected(
const fpxx* coarse, fpxx* fine, const uint64_t* target, const uint64_t* donor,
unsigned int count, unsigned long coarse_cells,
const uint64_t* neighbors, const float* offsets)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const uint64_t t = target[row], d = donor[row];
fpxx* to = fine + (t / TOTAL_CELLS) * NQ * TOTAL_CELLS;
for (unsigned int q = 0; q < NQ; ++q) {
float v[4];
for (unsigned int a = 0; a < 4; ++a) {
const uint64_t n = neighbors[4ul * row + a];
v[a] = coarse[(n / coarse_cells) * NQ * coarse_cells
+ q * coarse_cells + n % coarse_cells];
}
float gx = .5f * (v[1] - v[0]), gy = .5f * (v[3] - v[2]);
const float ex = d_cx[q], ey = d_cy[q];
if (q != 0) {
const float along = (ex * gx + ey * gy) / (ex * ex + ey * ey);
gx -= ex * along;
gy -= ey * along;
}
to[index_f(t % TOTAL_CELLS, q)] =
coarse[(d / coarse_cells) * NQ * coarse_cells + q * coarse_cells + d % coarse_cells]
+ offsets[2ul * row] * gx + offsets[2ul * row + 1] * gy;
}
}
// Targets are receiver-local SOURCE slots for the immediately following pull.
// Keep child summation order identical to PatchCoarseStep; no collision here.
extern "C" __global__ void BlockCoalesce(
const fpxx* fine, fpxx* coarse, const uint64_t* target, const uint64_t* children,
unsigned int count)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const uint64_t* s = children + 4ul * row;
coarse[target[row]] = 0.25f * (fine[s[0]] + fine[s[1]] + fine[s[2]] + fine[s[3]]);
}
// Static cut links, receiver-local staging: cell is the receiver's local core
// cell, so the existing BFL writes the solid-source window image that this
// block's next pull reads (no halo copy between this kernel and BlockStep).
// surface_velocity (nullable, 2 floats per link): prescribed fluid velocity on
// fixed geometry, same convention as CurvedBoundaryKernel. Optional raw rows
// (jx,jy,torque,dm) are unique physical links, never body-observation slots.
extern "C" __global__ void BlockCurvedBoundary(
fpxx* f, const uint32_t* block, const uint32_t* cell, const unsigned char* dir,
const float* q, const unsigned char* fallback, unsigned int count,
const float* surface_velocity, const float* rx, const float* ry, float* raw)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
float Uw = 0.0f, Vw = 0.0f;
if (surface_velocity != nullptr) {
const float u = surface_velocity[2 * row], v = surface_velocity[2 * row + 1];
if (u != 0.0f || v != 0.0f) { Uw = u; Vw = v; }
}
apply_bouzidi_link(dir[row], q[row], cell[row], f + (unsigned long)block[row] * NQ * TOTAL_CELLS,
Uw, Vw, rx ? rx[row] : 0.0f, ry ? ry[row] : 0.0f,
fallback[row], nullptr, -1, raw ? raw + 4ul * row : nullptr,
raw ? raw + 4ul * row + 3 : nullptr);
}
// Same-stream BFL -> this row-owned accumulation -> pull step. No atomics,
// no cross-row reduction and no history allocation proportional to substeps.
extern "C" __global__ void BlockAccumulateExchange(
const float* raw, double* integral, unsigned int count, double h, double dt)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const unsigned long k = 4ul * row;
const double impulse = h * dt;
integral[k] += impulse * (double)raw[k];
integral[k + 1] += impulse * (double)raw[k + 1];
integral[k + 2] += h * impulse * (double)raw[k + 2];
integral[k + 3] += h * h * (double)raw[k + 3];
}
// Pull/collide/store core cells only; halo cells of out are left untouched.
// flags (block cell ids, nullable): solid cores store NaN and never collide.
// kind (block cell ids, nullable = all ACTIVE): OUTER skips the pull and
// stores NaN, INNER pulls without collision, ACTIVE pulls and collides.
extern "C" __global__ void BlockStep(const fpxx* in, fpxx* out, unsigned int halo,
const uint16_t* flags, const unsigned char* kind) {
unsigned int x, y; unsigned long k;
index_from_thread(x, y, k);
if (x < halo || y < halo || x >= NX - halo || y >= NY - halo) return;
const unsigned long base = (unsigned long)blockIdx.z * NQ * TOTAL_CELLS;
const unsigned long cell = (unsigned long)blockIdx.z * TOTAL_CELLS + k;
const unsigned char cell_kind = kind != nullptr ? kind[cell] : KIND_ACTIVE;
float f[NQ];
if (cell_kind == KIND_OUTER || (flags != nullptr && is_solid(flags[cell]))) {
for (int q = 0; q < NQ; ++q) f[q] = NAN;
} else {
unsigned long j[NQ];
compute_neighbors(x, y, j); // Core neighbors stay inside the window: no wrap.
stream_pull_load(k, f, in + base, j);
if (cell_kind == KIND_ACTIVE) {
if (d_block_wall.nx != 0) {
unsigned int gx, gy;
block_global_xy(x, y, halo, d_block_wall.nx, gx, gy);
if (gy == 1)
reconstruct_bottom_velocity(f, 0.0f, bottom_wall_normal_velocity(
gx, d_block_wall.nx, d_block_wall.slot_width, d_block_wall.amplitude));
else if (gy == d_block_wall.ny - 2) reconstruct_top_reservoir(f, RHO);
}
#if COLLISION_MODEL == 2
collide_mrt_unforced(f); // SRT path below unchanged.
#else
float rho, ux, uy, feq[NQ];
compute_rho_u(f, rho, ux, uy);
#if USE_DDF_SHIFTING
// Preserve the density perturbation before adding the unit reference.
const float drho = f[0]+f[1]+f[2]+f[3]+f[4]+f[5]+f[6]+f[7]+f[8];
const float jx = f[1]-f[2]+f[5]-f[6]+f[7]-f[8];
const float jy = f[3]-f[4]+f[5]-f[6]-f[7]+f[8];
compute_feq_from_momentum(drho, jx, jy, ux, uy, feq);
#else
compute_feq(rho, ux, uy, feq);
#ifndef BLOCK_ROOT_BC
#error "Blocks require a generated BLOCK_ROOT_BC mode"
#elif BLOCK_ROOT_BC < 0 || BLOCK_ROOT_BC > 2
#error "BLOCK_ROOT_BC must be 0, 1 or 2"
#endif
collide_srt_no_force(f, feq, d_params.omega);
#endif
}
}
stream_pull_store(k, f, out + base);
}
// Root channel closure; fine and periodic levels keep BlockStep.
#if BLOCK_ROOT_BC == 1
#include "boundary/block_wall.cuh"
#endif
#if BLOCK_ROOT_BC != 2
#include "operators/block_collision.cuh"
#endif
// Kernels, in definition order: transfers, boundary staging, then the one full
// step selected for this module (BlockStep, or BlockChannelStep for a channel root).
#include "step/block_transfer.cu"
#include "step/block_boundary.cu"
#if BLOCK_ROOT_BC != 2
#include "step/block_step.cu"
#else
#include "step/block_channel.cu"
#endif
@@ -0,0 +1,20 @@
// Physical parent walls; NX/NY remain local storage strides.
#ifndef CELERIS_BOUNDARY_BLOCK_WALL_CUH
#define CELERIS_BOUNDARY_BLOCK_WALL_CUH
#include "planar_velocity.cuh"
struct BlockWall { unsigned int nx, ny, slot_width; float amplitude; };
extern "C" { __constant__ BlockWall d_block_wall; }
__device__ __forceinline__ void block_wall(float* f, unsigned int x, unsigned int y,
unsigned int halo) {
unsigned int gx, gy;
block_global_xy(x, y, halo, d_block_wall.nx, gx, gy);
if (gy == 1)
reconstruct_bottom_velocity(f, 0.0f, bottom_wall_normal_velocity(
gx, d_block_wall.nx, d_block_wall.slot_width, d_block_wall.amplitude));
else if (gy == d_block_wall.ny - 2) reconstruct_top_reservoir(f, RHO);
}
#endif
@@ -15,10 +15,13 @@
// D2Q9 y-wall half-way bounce-back for pull double-buffer streaming.
// ---------------------------------------------------------------------------
#if LATTICE_MODEL == LATTICE_D2Q9
__device__ inline void apply_wall_bb_d2q9_y_pull(unsigned int y,
float* __restrict__ f,
const fpxx* __restrict__ fi_in,
unsigned long k)
// Rows 1 and top_row (= physical ny - 2) next to walls at rows 0 and ny - 1.
// y is the physical row; k addresses the node in fi_in (a block window may
// have NX/NY strides different from the physical extent).
__device__ inline void apply_wall_bb_d2q9_rows(unsigned int y, unsigned int top_row,
float* __restrict__ f,
const fpxx* __restrict__ fi_in,
unsigned long k)
{
if (y == 1) {
// Directions sourced from y=0 wall in pull step: +y, +x+y, -x+y.
@@ -26,14 +29,22 @@ __device__ inline void apply_wall_bb_d2q9_y_pull(unsigned int y,
f[5] = load_ddf(fi_in, index_f(k, 6u));
f[8] = load_ddf(fi_in, index_f(k, 7u));
}
else if (y == (unsigned int)(NY - 2)) {
// Directions sourced from y=NY-1 wall in pull step: -y, -x-y, +x-y.
else if (y == top_row) {
// Directions sourced from the y = ny - 1 wall in pull step: -y, -x-y, +x-y.
f[4] = load_ddf(fi_in, index_f(k, 3u));
f[6] = load_ddf(fi_in, index_f(k, 5u));
f[7] = load_ddf(fi_in, index_f(k, 8u));
}
}
__device__ inline void apply_wall_bb_d2q9_y_pull(unsigned int y,
float* __restrict__ f,
const fpxx* __restrict__ fi_in,
unsigned long k)
{
apply_wall_bb_d2q9_rows(y, (unsigned int)(NY - 2), f, fi_in, k);
}
// ---------------------------------------------------------------------------
// D2Q9 y-wall free-slip (specular reflection) for pull double-buffer streaming.
//
@@ -8,6 +8,9 @@
#define STORE_PRECISION 0
#define USE_DDF_SHIFTING 0
// Internal blocked root mode: 0=periodic/fine, 1=slot/reservoir, 2=channel.
#define BLOCK_ROOT_BC 0
#define USE_LES 0
#define LES_CS 0.160000f
#define LES_CLOSED_FORM 1
@@ -0,0 +1,16 @@
// CelerisLab – core/cell_kind.cuh
// Fixed-patch cell kinds stored per block cell, matching lbm/refinement.py
// ACTIVE/INNER_GHOST/OUTER_GHOST: OUTER is invalid (stores NaN, no pull),
// INNER streams without collision, ACTIVE pulls and collides.
#ifndef CELERIS_CORE_CELL_KIND_CUH
#define CELERIS_CORE_CELL_KIND_CUH
constexpr unsigned char KIND_ACTIVE = 0u, KIND_INNER = 1u, KIND_OUTER = 2u;
// NaN populations mark a cell without valid state (OUTER ghost or solid core).
__device__ __forceinline__ void fill_invalid(float* f)
{
for (int q = 0; q < NQ; ++q) f[q] = NAN;
}
#endif
@@ -0,0 +1,31 @@
// Collision used by periodic/fine and slot/reservoir BlockStep.
#ifndef CELERIS_OPERATORS_BLOCK_COLLISION_CUH
#define CELERIS_OPERATORS_BLOCK_COLLISION_CUH
#include "../core/params.cuh"
#include "macro.cuh"
#include "collision_srt.cuh"
#if COLLISION_MODEL == 2
#include "collision_mrt.cuh"
#endif
__device__ __forceinline__ void block_collide(float* f) {
#if COLLISION_MODEL == 2
collide_mrt_unforced(f); // SRT path below unchanged.
#else
float rho, ux, uy, feq[NQ];
compute_rho_u(f, rho, ux, uy);
#if USE_DDF_SHIFTING
// Preserve the density perturbation before adding the unit reference.
const float drho = f[0]+f[1]+f[2]+f[3]+f[4]+f[5]+f[6]+f[7]+f[8];
const float jx = f[1]-f[2]+f[5]-f[6]+f[7]-f[8];
const float jy = f[3]-f[4]+f[5]-f[6]-f[7]+f[8];
compute_feq_from_momentum(drho, jx, jy, ux, uy, feq);
#else
compute_feq(rho, ux, uy, feq);
#endif
collide_srt_no_force(f, feq, d_params.omega);
#endif
}
#endif
@@ -0,0 +1,14 @@
// CelerisLab – operators/bounce_back_swap.cuh
// Full-way bounce-back at a wall node: swap every opposite pair (paired
// order q, q+1). Needs only NQ; shared by the uniform and blocked steps.
#ifndef CELERIS_OPERATORS_BOUNCE_BACK_SWAP_CUH
#define CELERIS_OPERATORS_BOUNCE_BACK_SWAP_CUH
__device__ __forceinline__ void bounce_back_swap(float* f) {
#pragma unroll
for (int i = 1; i < NQ; i += 2) {
float t = f[i]; f[i] = f[i+1]; f[i+1] = t;
}
}
#endif
@@ -5,12 +5,7 @@
#ifndef CELERIS_OPERATORS_HELPERS_CUH
#define CELERIS_OPERATORS_HELPERS_CUH
__device__ __forceinline__ void bounce_back_swap(float* f) {
#pragma unroll
for (int i = 1; i < NQ; i += 2) {
float t = f[i]; f[i] = f[i+1]; f[i+1] = t;
}
}
#include "bounce_back_swap.cuh"
__device__ __forceinline__ void collide_dispatch(
float* f, float rho_n,
+3 -90
View File
@@ -9,6 +9,7 @@
#include "core/params.cuh"
#include "core/flags.cuh"
#include "core/obs.cuh"
#include "core/cell_kind.cuh"
#include "operators/macro.cuh"
#include "operators/collision_srt.cuh"
#if COLLISION_MODEL == 2
@@ -35,93 +36,5 @@ __device__ unsigned int incoming_counts[NQ * TOTAL_CELLS];
}
#endif
__device__ __forceinline__ void patch_collide(float* f, unsigned long k) {
#if COLLISION_MODEL == 2
collide_mrt_unforced(f);
#else
float rho, ux, uy, feq[NQ];
compute_rho_u(f, rho, ux, uy);
compute_feq(rho, ux, uy, feq);
collide_srt_no_force(f, feq, d_params.omega);
#endif
#ifdef CELERIS_REFINEMENT_AUDIT
++collision_counts[k];
#endif
}
extern "C" __global__ void PatchExplode(
const float* coarse, float* fine, const uint32_t* targets,
const uint32_t* donors, unsigned int count, unsigned long coarse_cells)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
for (unsigned int q = 0; q < NQ; ++q)
fine[index_f(targets[row], q)] = coarse[q * coarse_cells + donors[row]];
#ifdef CELERIS_REFINEMENT_AUDIT
++explosion_counts[targets[row]];
#endif
}
extern "C" __global__ void PatchFineStep(const float* in, float* out, const uint16_t* flags) {
unsigned int x, y; unsigned long k;
index_from_thread(x, y, k);
if (x >= NX || y >= NY) return;
float f[NQ];
// Outer halo is invalid after EACH hop, including when reusing the buffer
// that held explosion data. Inner halo only streams; it never collides.
if (x < 2 || y < 2 || x >= NX - 2 || y >= NY - 2
|| (flags != nullptr && is_solid(flags[k]))) {
for (int q = 0; q < NQ; ++q) f[q] = NAN;
} else {
unsigned long j[NQ];
compute_neighbors(x, y, j); // These interior targets never wrap.
stream_pull_load(k, f, in, j);
if (x >= 4 && y >= 4 && x < NX - 4 && y < NY - 4)
patch_collide(f, k);
}
stream_pull_store(k, f, out);
}
extern "C" __global__ void PatchCoarseStep(
const float* in, float* out, const float* fine,
const uint64_t* targets, const uint64_t* children, unsigned int count,
unsigned int x0, unsigned int y0, unsigned int x1, unsigned int y1,
unsigned int wall_aware)
{
unsigned int x, y; unsigned long k;
index_from_thread(x, y, k);
if (x >= NX || y >= NY) return;
// Both buffers were poisoned at initialization; covered cells stay invalid.
if (x >= x0 && x < x1 && y >= y0 && y < y1) return;
if (wall_aware && (y == 0 || y == NY - 1)) return;
unsigned long j[NQ];
compute_neighbors(x, y, j);
float f[NQ];
stream_pull_load(k, f, in, j);
if (x >= x0 - 1 && x <= x1 && y >= y0 - 1 && y <= y1) {
for (unsigned int q = 1; q < NQ; ++q) {
uint64_t slot = index_f(k, q);
unsigned int lo = 0, hi = count;
while (lo < hi) {
unsigned int mid = lo + (hi - lo) / 2;
if (targets[mid] < slot) lo = mid + 1;
else hi = mid;
}
if (lo < count && targets[lo] == slot) {
const uint64_t* s = children + 4ul * lo;
f[q] = 0.25f * (fine[s[0]] + fine[s[1]] + fine[s[2]] + fine[s[3]]);
#ifdef CELERIS_REFINEMENT_AUDIT
++incoming_counts[slot];
#endif
}
}
}
// Includes the eight grazing-corner links with UNCOVERED coarse donors.
// All replacement arrivals must be in place before computing rho/u.
if (wall_aware) {
if (y == 1) reconstruct_bottom_velocity(f, 0.0f, bottom_wall_normal_velocity(x));
else if (y == NY - 2) reconstruct_top_reservoir(f, RHO);
}
patch_collide(f, k);
stream_pull_store(k, f, out);
}
// Kernels: patch collision helper, PatchExplode, PatchFineStep, PatchCoarseStep.
#include "step/patch_step.cu"
@@ -0,0 +1,42 @@
// CelerisLab/lbm/kernels/step/block_boundary.cu
// Receiver-local curved-boundary staging and its raw-exchange accumulation.
// Part of the blocks.cu compilation entry; relies on its includes and guards.
// Static cut links, receiver-local staging: cell is the receiver's local core
// cell, so the existing BFL writes the solid-source window image that this
// block's next pull reads (no halo copy between this kernel and BlockStep).
// surface_velocity (nullable, 2 floats per link): prescribed fluid velocity on
// fixed geometry, same convention as CurvedBoundaryKernel. Optional raw rows
// (jx,jy,torque,dm) are unique physical links, never body-observation slots.
extern "C" __global__ void BlockCurvedBoundary(
fpxx* f, const uint32_t* block, const uint32_t* cell, const unsigned char* dir,
const float* q, const unsigned char* fallback, unsigned int count,
const float* surface_velocity, const float* rx, const float* ry, float* raw)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
float Uw = 0.0f, Vw = 0.0f;
if (surface_velocity != nullptr) {
const float u = surface_velocity[2 * row], v = surface_velocity[2 * row + 1];
if (u != 0.0f || v != 0.0f) { Uw = u; Vw = v; }
}
apply_bouzidi_link(dir[row], q[row], cell[row], f + (unsigned long)block[row] * NQ * TOTAL_CELLS,
Uw, Vw, rx ? rx[row] : 0.0f, ry ? ry[row] : 0.0f,
fallback[row], nullptr, -1, raw ? raw + 4ul * row : nullptr,
raw ? raw + 4ul * row + 3 : nullptr);
}
// Same-stream BFL -> this row-owned accumulation -> pull step. No atomics,
// no cross-row reduction and no history allocation proportional to substeps.
extern "C" __global__ void BlockAccumulateExchange(
const float* raw, double* integral, unsigned int count, double h, double dt)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const unsigned long k = 4ul * row;
const double impulse = h * dt;
integral[k] += impulse * (double)raw[k];
integral[k + 1] += impulse * (double)raw[k + 1];
integral[k + 2] += h * impulse * (double)raw[k + 2];
integral[k + 3] += h * h * (double)raw[k + 3];
}
@@ -1,5 +1,12 @@
#if BLOCK_ROOT_BC != 2
#error "BlockChannelStep requires BLOCK_ROOT_BC == 2"
#endif
// Root-only blocked counterpart of OneStep's D2Q9/SRT channel closure.
// NX/NY are LOCAL strides; all physical boundary decisions use global coordinates.
#include "../core/cell_kind.cuh"
#include "../operators/bounce_back_swap.cuh"
#include "../boundary/bounce_back.cuh"
#include "../boundary/inlet/common.cuh"
#include "../boundary/inlet/zou_he_local.cuh"
#include "../boundary/outlet/pressure_neq.cuh"
@@ -21,7 +28,7 @@ extern "C" __global__ void BlockChannelStep(
const unsigned long cell = (unsigned long)blockIdx.z * TOTAL_CELLS + k;
float f[NQ];
if (kind != nullptr && kind[cell] == KIND_OUTER) {
for (int q = 0; q < NQ; ++q) f[q] = NAN;
fill_invalid(f);
} else {
unsigned long j[NQ];
compute_neighbors(x, y, j);
@@ -29,27 +36,17 @@ extern "C" __global__ void BlockChannelStep(
const bool wall = gy == 0 || gy == d_block_channel.ny - 1;
const bool outlet = gx == d_block_channel.nx - 1;
if (wall) {
// Wall owns all four corners. Ghosts pull and swap, never collide.
for (int q = 1; q < NQ; q += 2) {
const float t = f[q]; f[q] = f[q + 1]; f[q + 1] = t;
}
bounce_back_swap(f); // Wall owns all four corners; ghosts never collide.
} else if (gx == 0) {
apply_zou_he_left_velocity_inlet_d2q9(
f, inlet_target_u((float)gy, d_block_channel.ny, d_block_channel.velocity), 0.0f);
} else if (outlet) {
// Immutable input donor, receiver-local (also valid across a block seam).
float donor[NQ];
for (int q = 0; q < NQ; ++q)
donor[q] = load_ddf(in + base, index_f(k - 1, q));
load_node(in + base, k - 1, donor);
apply_pressure_outlet_d2q9(f, donor);
} else if (gy == 1) {
f[3] = load_ddf(in + base, index_f(k, 4));
f[5] = load_ddf(in + base, index_f(k, 6));
f[8] = load_ddf(in + base, index_f(k, 7));
} else if (gy == d_block_channel.ny - 2) {
f[4] = load_ddf(in + base, index_f(k, 3));
f[6] = load_ddf(in + base, index_f(k, 5));
f[7] = load_ddf(in + base, index_f(k, 8));
} else {
apply_wall_bb_d2q9_rows(gy, d_block_channel.ny - 2, f, in + base, k);
}
if (!wall && !outlet) {
#if COLLISION_MODEL == 2
@@ -0,0 +1,35 @@
// CelerisLab/lbm/kernels/step/block_step.cu
// Generic blocked pull/collide/store for periodic/fine and slot/reservoir levels.
// Part of the blocks.cu compilation entry; relies on its includes and guards.
#if BLOCK_ROOT_BC == 2
#error "BlockStep is not built for the channel root (BLOCK_ROOT_BC == 2)"
#endif
// Pull/collide/store core cells only; halo cells of out are left untouched.
// flags (block cell ids, nullable): solid cores store NaN and never collide.
// kind (block cell ids, nullable = all ACTIVE): OUTER skips the pull and
// stores NaN, INNER pulls without collision, ACTIVE pulls and collides.
extern "C" __global__ void BlockStep(const fpxx* in, fpxx* out, unsigned int halo,
const uint16_t* flags, const unsigned char* kind) {
unsigned int x, y; unsigned long k;
index_from_thread(x, y, k);
if (x < halo || y < halo || x >= NX - halo || y >= NY - halo) return;
const unsigned long base = (unsigned long)blockIdx.z * NQ * TOTAL_CELLS;
const unsigned long cell = (unsigned long)blockIdx.z * TOTAL_CELLS + k;
const unsigned char cell_kind = kind != nullptr ? kind[cell] : KIND_ACTIVE;
float f[NQ];
if (cell_kind == KIND_OUTER || (flags != nullptr && is_solid(flags[cell]))) {
fill_invalid(f);
} else {
unsigned long j[NQ];
compute_neighbors(x, y, j); // Core neighbors stay inside the window: no wrap.
stream_pull_load(k, f, in + base, j);
if (cell_kind == KIND_ACTIVE) {
#if BLOCK_ROOT_BC == 1
block_wall(f, x, y, halo);
#endif
block_collide(f);
}
}
stream_pull_store(k, f, out + base);
}
@@ -0,0 +1,78 @@
// CelerisLab/lbm/kernels/step/block_transfer.cu
// Same-level halo refresh and 2:1 explosion/coalescence between blocked levels.
// Part of the blocks.cu compilation entry; relies on its includes and guards.
// Halo refresh: block cell ids b*TOTAL_CELLS+c, sources are always core cells.
extern "C" __global__ void BlockHaloCopy(
fpxx* f, const uint64_t* dst, const uint64_t* src, unsigned int count)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const uint64_t d = dst[row], s = src[row];
fpxx* to = f + (d / TOTAL_CELLS) * NQ * TOTAL_CELLS;
const fpxx* from = f + (s / TOTAL_CELLS) * NQ * TOTAL_CELLS;
for (unsigned int q = 0; q < NQ; ++q)
to[index_f(d % TOTAL_CELLS, q)] = from[index_f(s % TOTAL_CELLS, q)];
}
// Fixed-patch explosion into blocked fine storage: target = owner block cell
// id of a patch-ghost cell; donors are parent block cell ids. Dense parent
// storage is the one-window case with coarse_cells = full domain size.
extern "C" __global__ void BlockExplode(
const fpxx* coarse, fpxx* fine, const uint64_t* target, const uint64_t* donor,
unsigned int count, unsigned long coarse_cells)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const uint64_t t = target[row];
fpxx* to = fine + (t / TOTAL_CELLS) * NQ * TOTAL_CELLS;
const uint64_t d = donor[row];
const fpxx* from = coarse + (d / coarse_cells) * NQ * coarse_cells;
for (unsigned int q = 0; q < NQ; ++q)
to[index_f(t % TOTAL_CELLS, q)] = from[q * coarse_cells + d % coarse_cells];
}
// Experimental Schornbaum--Ruede (arXiv:1508.07982, Algorithm 2)
// and Chen et al. (doi:10.1016/j.physa.2005.09.036) population interpolation.
// Independent implementation of the mathematical stencil; no upstream source copied.
// Stencil cells are owner cores; a disabled axis repeats its center twice.
extern "C" __global__ void BlockExplodeProjected(
const fpxx* coarse, fpxx* fine, const uint64_t* target, const uint64_t* donor,
unsigned int count, unsigned long coarse_cells,
const uint64_t* neighbors, const float* offsets)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const uint64_t t = target[row], d = donor[row];
fpxx* to = fine + (t / TOTAL_CELLS) * NQ * TOTAL_CELLS;
for (unsigned int q = 0; q < NQ; ++q) {
float v[4];
for (unsigned int a = 0; a < 4; ++a) {
const uint64_t n = neighbors[4ul * row + a];
v[a] = coarse[(n / coarse_cells) * NQ * coarse_cells
+ q * coarse_cells + n % coarse_cells];
}
float gx = .5f * (v[1] - v[0]), gy = .5f * (v[3] - v[2]);
const float ex = d_cx[q], ey = d_cy[q];
if (q != 0) {
const float along = (ex * gx + ey * gy) / (ex * ex + ey * ey);
gx -= ex * along;
gy -= ey * along;
}
to[index_f(t % TOTAL_CELLS, q)] =
coarse[(d / coarse_cells) * NQ * coarse_cells + q * coarse_cells + d % coarse_cells]
+ offsets[2ul * row] * gx + offsets[2ul * row + 1] * gy;
}
}
// Targets are receiver-local SOURCE slots for the immediately following pull.
// Keep child summation order identical to PatchCoarseStep; no collision here.
extern "C" __global__ void BlockCoalesce(
const fpxx* fine, fpxx* coarse, const uint64_t* target, const uint64_t* children,
unsigned int count)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
const uint64_t* s = children + 4ul * row;
coarse[target[row]] = 0.25f * (fine[s[0]] + fine[s[1]] + fine[s[2]] + fine[s[3]]);
}
@@ -0,0 +1,95 @@
// CelerisLab/lbm/kernels/step/patch_step.cu
// Dense fixed-patch collision, explosion and the fine/coarse steps.
// Part of the refinement.cu compilation entry; relies on its includes, guards
// and the optional CELERIS_REFINEMENT_AUDIT counters declared there.
__device__ __forceinline__ void patch_collide(float* f, unsigned long k) {
#if COLLISION_MODEL == 2
collide_mrt_unforced(f);
#else
float rho, ux, uy, feq[NQ];
compute_rho_u(f, rho, ux, uy);
compute_feq(rho, ux, uy, feq);
collide_srt_no_force(f, feq, d_params.omega);
#endif
#ifdef CELERIS_REFINEMENT_AUDIT
++collision_counts[k];
#endif
}
extern "C" __global__ void PatchExplode(
const float* coarse, float* fine, const uint32_t* targets,
const uint32_t* donors, unsigned int count, unsigned long coarse_cells)
{
unsigned int row = blockIdx.x * blockDim.x + threadIdx.x;
if (row >= count) return;
for (unsigned int q = 0; q < NQ; ++q)
fine[index_f(targets[row], q)] = coarse[q * coarse_cells + donors[row]];
#ifdef CELERIS_REFINEMENT_AUDIT
++explosion_counts[targets[row]];
#endif
}
extern "C" __global__ void PatchFineStep(const float* in, float* out, const uint16_t* flags) {
unsigned int x, y; unsigned long k;
index_from_thread(x, y, k);
if (x >= NX || y >= NY) return;
float f[NQ];
// Outer halo is invalid after EACH hop, including when reusing the buffer
// that held explosion data. Inner halo only streams; it never collides.
if (x < 2 || y < 2 || x >= NX - 2 || y >= NY - 2
|| (flags != nullptr && is_solid(flags[k]))) {
fill_invalid(f);
} else {
unsigned long j[NQ];
compute_neighbors(x, y, j); // These interior targets never wrap.
stream_pull_load(k, f, in, j);
if (x >= 4 && y >= 4 && x < NX - 4 && y < NY - 4)
patch_collide(f, k);
}
stream_pull_store(k, f, out);
}
extern "C" __global__ void PatchCoarseStep(
const float* in, float* out, const float* fine,
const uint64_t* targets, const uint64_t* children, unsigned int count,
unsigned int x0, unsigned int y0, unsigned int x1, unsigned int y1,
unsigned int wall_aware)
{
unsigned int x, y; unsigned long k;
index_from_thread(x, y, k);
if (x >= NX || y >= NY) return;
// Both buffers were poisoned at initialization; covered cells stay invalid.
if (x >= x0 && x < x1 && y >= y0 && y < y1) return;
if (wall_aware && (y == 0 || y == NY - 1)) return;
unsigned long j[NQ];
compute_neighbors(x, y, j);
float f[NQ];
stream_pull_load(k, f, in, j);
if (x >= x0 - 1 && x <= x1 && y >= y0 - 1 && y <= y1) {
for (unsigned int q = 1; q < NQ; ++q) {
uint64_t slot = index_f(k, q);
unsigned int lo = 0, hi = count;
while (lo < hi) {
unsigned int mid = lo + (hi - lo) / 2;
if (targets[mid] < slot) lo = mid + 1;
else hi = mid;
}
if (lo < count && targets[lo] == slot) {
const uint64_t* s = children + 4ul * lo;
f[q] = 0.25f * (fine[s[0]] + fine[s[1]] + fine[s[2]] + fine[s[3]]);
#ifdef CELERIS_REFINEMENT_AUDIT
++incoming_counts[slot];
#endif
}
}
}
// Includes the eight grazing-corner links with UNCOVERED coarse donors.
// All replacement arrivals must be in place before computing rho/u.
if (wall_aware) {
if (y == 1) reconstruct_bottom_velocity(f, 0.0f, bottom_wall_normal_velocity(x));
else if (y == NY - 2) reconstruct_top_reservoir(f, RHO);
}
patch_collide(f, k);
stream_pull_store(k, f, out);
}
@@ -21,6 +21,14 @@
// i.e. the population traveling in direction i at node n
// arrived from the node at n - c_i = neighbor in direction opp(i)
// ---------------------------------------------------------------------------
// load_node: all NQ populations of node n as stored (no streaming).
__device__ __forceinline__ void load_node(const fpxx* __restrict__ fi, unsigned long n,
float* __restrict__ f)
{
for (int q = 0; q < NQ; ++q)
f[q] = load_ddf(fi, index_f(n, q));
}
__device__ __forceinline__ void stream_pull_load(unsigned long n,
float* __restrict__ f,
const fpxx* __restrict__ fi,
+4 -1
View File
@@ -362,7 +362,10 @@ class FixedPatchStepper:
raise RuntimeError("level omega/tau mismatch")
host = self._boundary_hosts[index]
n_objects = len(host.objects) if name == "PatchFineStep" and host is not None else 0
build = compiler.generate_config(config, n_objects=n_objects)
block_root_bc = 0
if index == 0 and layout is not None:
block_root_bc = 2 if self._channel else 1 if self._wall_aware else 0
build = compiler.generate_config(config, n_objects=n_objects, block_root_bc=block_root_bc)
try:
ptx = compiler.compile_kernel(self.ctx.sm_arch, source=source, build_dir=build)
module = compiler.load_module(ptx)
+2 -1
View File
@@ -16,6 +16,7 @@
#define NZ 1
#define NT 1
#define COLLISION_MODEL 0
#define BLOCK_ROOT_BC 2
#define USE_LES 0
#define RHO 1.f
#define U0 .025f
@@ -40,7 +41,7 @@ inline float __fdiv_rn(float a,float b){return a/b;}
#include "boundary/inlet_outlet.cuh"
#include "step/one_step_double.cu"
#else
constexpr unsigned char KIND_OUTER=2;
#include "core/cell_kind.cuh"
#include "step/block_channel.cu"
#endif
int main(){
+26
View File
@@ -83,6 +83,32 @@ assert abs(sum(compute_feq_d2q9(1.,0.,0.)) - 1.) < 1e-6
class BuildContracts(unittest.TestCase):
def test_block_root_modes_are_rendered_and_recorded(self):
with tempfile.TemporaryDirectory() as root:
for mode in (0, 1, 2):
with self.subTest(mode=mode):
build = compiler.generate_config(LBMConfig(), build_root=root, block_root_bc=mode)
try:
method = Path(build, "config/config_method.h").read_text()
self.assertIn(f"#define BLOCK_ROOT_BC {mode}\n", method)
metadata = json.loads(Path(build, ".celeris-build").read_text())
self.assertEqual(metadata["block_root_bc"], mode)
self.assertNotIn("block_root_bc", metadata["config"])
finally:
compiler.cleanup_build_dir(build)
build = compiler.generate_config(LBMConfig(), build_root=root)
try:
self.assertEqual(json.loads(Path(build, ".celeris-build").read_text())["block_root_bc"], 0)
finally:
compiler.cleanup_build_dir(build)
def test_invalid_block_root_mode_fails_before_snapshot(self):
for mode in (-1, 3, True, False, 1.0, "1", None):
with self.subTest(mode=mode), patch.object(compiler.tempfile, "mkdtemp") as allocate:
with self.assertRaisesRegex(ValueError, "block_root_bc"):
compiler.generate_config(LBMConfig(), block_root_bc=mode)
allocate.assert_not_called()
@staticmethod
def fake_compile(command, check):
source = Path(command[-3])
+1
View File
@@ -97,5 +97,6 @@ def test_evolved_ghost_snapshot_and_physical_download():
with pytest.raises(ValueError, match="finite"):
backend.initialize(*arrays)
assert backend._levels[0].kernel == host.compiler.load_module.return_value.get_function("BlockChannelStep")
assert [call.kwargs["block_root_bc"] for call in host.compiler.generate_config.call_args_list] == [2, 0]
finally:
backend.close()
+1 -1
View File
@@ -162,7 +162,7 @@ class ProjectedKernelTests(unittest.TestCase):
targets = child.owner.ravel()[p.ghost_fine]
donors = parent.owner.ravel()[p.ghost_coarse]
source = (Path(__file__).resolve().parents[2] /
"src/CelerisLab/lbm/kernels/blocks.cu").read_text()
"src/CelerisLab/lbm/kernels/step/block_transfer.cu").read_text()
kernel = source.split('extern "C" __global__ void BlockExplodeProjected(',1)[1]
kernel = 'void BlockExplodeProjected(' + kernel.split('// Targets are receiver-local SOURCE slots',1)[0]
def array(name, values, kind):
+8 -4
View File
@@ -22,13 +22,16 @@ class RefinementMRTHost(unittest.TestCase):
def test_srt_statements_and_guards_preserved(self):
"""MRT is an additive compile-time branch; SRT statements and other guards remain."""
blocks = (KERNELS / "blocks.cu").read_text()
blocks = (KERNELS / "blocks.cu").read_text() + (KERNELS / "step/block_step.cu").read_text()
channel = (KERNELS / "step/block_channel.cu").read_text()
refinement = (KERNELS / "refinement.cu").read_text()
block_collision = (KERNELS / "operators/block_collision.cuh").read_text()
self.assertIn('#include "operators/block_collision.cuh"', blocks)
self.assertEqual(blocks.count("block_collide(f);"), 1)
refinement = (KERNELS / "refinement.cu").read_text() + (KERNELS / "step/patch_step.cu").read_text()
for text in ("compute_feq_from_momentum(drho, jx, jy, ux, uy, feq);",
"compute_feq(rho, ux, uy, feq);",
"collide_srt_no_force(f, feq, d_params.omega);"):
self.assertIn(text, blocks)
self.assertIn(text, block_collision)
self.assertIn("collide_srt(f, feq, forcing, omega);", channel)
self.assertIn("collide_srt_no_force(f, feq, d_params.omega);", refinement)
for source in (blocks, refinement):
@@ -40,10 +43,11 @@ class RefinementMRTHost(unittest.TestCase):
# One shared unforced MRT body; each blocked/patch path calls it exactly once.
mrt = (KERNELS / "operators/collision_mrt.cuh").read_text()
self.assertEqual(mrt.count("void collide_mrt_unforced("), 1)
for source in (blocks, channel, refinement):
for source in (blocks + "\n" + block_collision, channel, refinement):
self.assertEqual(source.count("collide_mrt_unforced(f);"), 1)
self.assertNotIn("collide_mrt_no_force(", source)
self.assertNotIn("compute_rho_u(f, rho, ux, uy);\n collide_mrt", source)
self.assertNotRegex(source, r"compute_rho_u\(f, rho, ux, uy\);\s*collide_mrt")
if __name__ == "__main__": unittest.main()
@@ -27,7 +27,7 @@ def state(ny, nx):
def compile_module(cfg, context, source):
cfg.validate()
build = compiler.generate_config(cfg)
build = compiler.generate_config(cfg, block_root_bc=2 if source == "blocks.cu" else 0)
try:
module = compiler.load_module(compiler.compile_kernel(context.sm_arch, source=source, build_dir=build))
finally:
@@ -223,6 +223,8 @@ class BlockedBaseCPU(unittest.TestCase):
self.assertIs(collide[4], coarse.kind)
walls = [a for ptr, a in m["uploads"] if isinstance(ptr, str) and ptr == "d_block_wall"]
self.assertEqual(len(walls), 1)
self.assertEqual([call.kwargs["block_root_bc"] for call in
compiler.generate_config.call_args_list], [1, 0])
width, height, slot, speed = struct.unpack("=IIIf", walls[0])
self.assertEqual((width, height, slot), (NX, NY, 8))
self.assertAlmostEqual(speed, .001)
@@ -249,35 +249,41 @@ class ProjectedPairsTests(unittest.TestCase):
self.assertFalse(assess(incomplete)["comparisons"][3]["passed"])
def test_pinned_build_sources_match_host_generated_inputs(self):
"""The pinned fingerprints are what today's compiler generates from the qualified
NUMERICAL_REVISION kernels; the current kernels may legitimately differ."""
"""Check the pinned fingerprints using NUMERICAL_REVISION's renderer and kernels.
The imported LBMConfig is unchanged from that revision; this is a host
build-input check, not a reconstruction of the historical CUDA toolchain."""
import importlib.util
import tarfile
from CelerisLab.cuda import compiler_v2
kernels = "src/CelerisLab/lbm/kernels"
renderer = "src/CelerisLab/cuda/compiler_v2.py"
if subprocess.run(["git", "-C", str(REPO), "cat-file", "-e",
f"{NUMERICAL_REVISION}^{{commit}}"], capture_output=True).returncode:
self.fail(f"{NUMERICAL_REVISION} is not in this clone; the pinned build "
"sources cannot be checked")
archive = subprocess.run(["git", "-C", str(REPO), "archive", "--format=tar",
NUMERICAL_REVISION, kernels],
NUMERICAL_REVISION, kernels, renderer],
capture_output=True, check=True).stdout
with tempfile.TemporaryDirectory() as snapshot:
with tarfile.open(fileobj=io.BytesIO(archive)) as tar:
tar.extractall(snapshot)
with patch.object(compiler_v2, "_KERNEL_DIR", str(Path(snapshot) / kernels)):
spec = importlib.util.spec_from_file_location(
"CelerisLab.cuda._compiler_a135252", Path(snapshot) / renderer)
compiler = importlib.util.module_from_spec(spec)
spec.loader.exec_module(compiler)
with patch.object(compiler, "_KERNEL_DIR", str(Path(snapshot) / kernels)):
for size in (128, 256):
with self.subTest(size=size):
cfg = replace(run_case.config(size, 100), nx=10, ny=10)
cfg.validate()
root = compiler_v2.generate_config(cfg)
root = compiler.generate_config(cfg)
try:
actual = json.loads((Path(root) / ".celeris-build").read_text())
expected = build_metadata(size)
self.assertEqual(actual, {key: expected[key] for key in
("source_sha256", "config", "n_objects")})
finally:
compiler_v2.cleanup_build_dir(root)
compiler.cleanup_build_dir(root)
def test_source_preflight_requires_clean_exact_revision_and_tree(self):
import oracle