Aleph-w 3.0
A C++ Library for Data Structures and Algorithms
Loading...
Searching...
No Matches
Aleph::CA::ca_checkpoint_detail Namespace Reference

Classes

struct  Header_Snapshot
 Header bundle pre-computed from an engine. More...
 
class  Tmp_File_Guard
 RAII guard that removes a temporary file unless committed. More...
 

Functions

std::vector< std::uint8_t > deflate_bytes (const std::uint8_t *src, std::size_t len, int level)
 Compress a raw byte buffer with DEFLATE (miniz backend).
 
std::vector< std::uint8_t > inflate_bytes (const std::uint8_t *src, std::size_t len, std::size_t expected_uncompressed)
 Decompress a DEFLATE byte buffer back to its raw form.
 
std::vector< std::uint8_t > inflate_bytes_up_to (const std::uint8_t *src, std::size_t len, std::size_t max_uncompressed)
 Decompress a DEFLATE byte buffer with an upper bound.
 
constexpr std::uint64_t fnv1a_64 (std::string_view s) noexcept
 FNV-1a 64-bit hash.
 
template<typename T >
std::uint64_t type_hash () noexcept
 Stable type hash of T.
 
template<typename T >
void write_le (std::ostream &out, const T &value)
 
template<typename T >
T read_le (std::istream &in)
 
std::filesystem::path make_tmp_path (const std::filesystem::path &target)
 Generate a temporary sibling path for atomic writes.
 
void sync_file (const std::filesystem::path &path)
 Force a file's contents to be durable on the underlying storage device.
 
void sync_directory (const std::filesystem::path &dir) noexcept
 Force a directory's metadata to be durable.
 
template<typename Engine >
requires Checkpointable_Engine<Engine>
std::vector< std::uint8_t > snapshot_frame_bytes (const Engine &engine)
 Flatten an engine frame into a row-major byte buffer.
 
template<typename Engine >
requires Checkpointable_Engine<Engine>
void restore_frame_from_bytes (Engine &engine, const std::vector< std::uint8_t > &bytes)
 Inverse of snapshot_frame_bytes: spray the buffer back into the engine's write lattice in row-major order.
 
template<typename Engine >
requires Checkpointable_Engine<Engine>
Header_Snapshot capture_header (const Engine &engine)
 Capture all header fields a checkpoint file needs.
 
void write_header_bytes (std::ostream &out, const Header_Snapshot &hs, const std::uint32_t flags, const std::uint32_t compression_level, const std::uint64_t payload_size, const std::uint64_t delta_base_step)
 Serialise the v2 header bytes for a checkpoint file.
 
void atomic_write_file (const std::filesystem::path &path, const Header_Snapshot &hs, const std::uint32_t flags, const std::uint32_t compression_level, const std::vector< std::uint8_t > &payload, const std::uint64_t delta_base_step, const Checkpoint_Options &options)
 Write header + payload atomically to path.
 
std::vector< std::uint8_t > build_payload (const std::vector< std::uint8_t > &raw, const Checkpoint_Options &options, std::uint32_t &flags_out, std::uint32_t &level_out)
 Build the (possibly compressed) frame payload from a flat byte buffer, honouring options.compress.
 
std::vector< std::uint8_t > read_raw_payload (const std::filesystem::path &path, const Checkpoint_Header &h)
 Read the raw, uncompressed payload from a checkpoint file given a validated header.
 

Variables

constexpr std::array< char, 8 > magic_bytes = {{'A', 'L', 'E', 'P', 'H', 'C', 'A', '1'}}
 Magic prefix identifying an Aleph CA checkpoint stream.
 
constexpr std::uint32_t format_version = 2
 Current format version.
 
constexpr std::uint32_t format_version_min_read = 1
 Oldest format version this file knows how to read.
 
constexpr std::streamoff header_bytes_v1 = 8 + 4 + 8 + 4 + 4 + 4 * 8 + 8 + 1 + 7 + 8 + 8
 Size in bytes of the v1 header (Phase 15 layout).
 
constexpr std::streamoff header_bytes_v2 = header_bytes_v1 + 24
 Size in bytes of the v2 header (Phase 17 layout: v1 + 4 trailer fields of 4 + 4 + 8 + 8 = 24 bytes).
 

Function Documentation

◆ atomic_write_file()

void Aleph::CA::ca_checkpoint_detail::atomic_write_file ( const std::filesystem::path &  path,
const Header_Snapshot &  hs,
const std::uint32_t  flags,
const std::uint32_t  compression_level,
const std::vector< std::uint8_t > &  payload,
const std::uint64_t  delta_base_step,
const Checkpoint_Options &  options 
)
inline

Write header + payload atomically to path.

Uses the temp-file + fsync + rename pattern. On any failure the temporary file is removed by Tmp_File_Guard; the final path retains its previous contents (or stays absent if it did not exist before the call).

Definition at line 814 of file ca-checkpoint.H.

References ah_runtime_error_if, atomic_write_file(), Aleph::CA::ca_checkpoint_detail::Tmp_File_Guard::commit(), make_tmp_path(), options, out, sync_directory(), sync_file(), and write_header_bytes().

Referenced by atomic_write_file().

◆ build_payload()

std::vector< std::uint8_t > Aleph::CA::ca_checkpoint_detail::build_payload ( const std::vector< std::uint8_t > &  raw,
const Checkpoint_Options &  options,
std::uint32_t &  flags_out,
std::uint32_t &  level_out 
)
inline

Build the (possibly compressed) frame payload from a flat byte buffer, honouring options.compress.

Definition at line 862 of file ca-checkpoint.H.

References build_payload(), deflate_bytes(), and options.

Referenced by build_payload().

◆ capture_header()

template<typename Engine >
requires Checkpointable_Engine<Engine>
Header_Snapshot Aleph::CA::ca_checkpoint_detail::capture_header ( const Engine &  engine)

Capture all header fields a checkpoint file needs.

Pulls the values from the engine in a single synchronous pass so the async writer can be fed an opaque task without holding a reference to the engine across worker activations.

Template Parameters
Engineengine type satisfying Checkpointable_Engine.
Parameters
[in]engineengine to snapshot.
Returns
populated Header_Snapshot.

Definition at line 755 of file ca-checkpoint.H.

References capture_header(), Aleph::CA::ca_checkpoint_detail::Header_Snapshot::cell_count, engine, Aleph::CA::ca_checkpoint_detail::Header_Snapshot::engine_type_hash, Aleph::CA::ca_checkpoint_detail::Header_Snapshot::extents, Aleph::CA::ca_checkpoint_detail::Header_Snapshot::has_rng, Aleph::CA::ca_checkpoint_detail::Header_Snapshot::master_seed, Aleph::CA::ca_checkpoint_detail::Header_Snapshot::rank, Aleph::CA::ca_checkpoint_detail::Header_Snapshot::state_type_size, and Aleph::CA::ca_checkpoint_detail::Header_Snapshot::step_count.

Referenced by capture_header().

◆ deflate_bytes()

std::vector< std::uint8_t > Aleph::CA::ca_checkpoint_detail::deflate_bytes ( const std::uint8_t *  src,
std::size_t  len,
int  level 
)

Compress a raw byte buffer with DEFLATE (miniz backend).

Parameters
[in]srcpointer to the uncompressed bytes.
[in]lennumber of bytes available at src.
[in]levelDEFLATE level in [1, 9]; clamped to that range.
Returns
owning byte buffer holding the compressed stream.
Exceptions
std::runtime_errorwhen miniz reports an error.

Definition at line 46 of file ca-checkpoint-compress.C.

References ah_runtime_error_if, Aleph::blossom_maximum_cardinality_matching(), and out.

Referenced by build_payload().

◆ fnv1a_64()

constexpr std::uint64_t Aleph::CA::ca_checkpoint_detail::fnv1a_64 ( std::string_view  s)
inlineconstexprnoexcept

FNV-1a 64-bit hash.

Used to derive a stable hash from typeid(Engine).name() so that load_checkpoint_into can reject snapshots produced by a different engine type within the same build.

Definition at line 163 of file ca-checkpoint.H.

References h.

Referenced by type_hash().

◆ inflate_bytes()

std::vector< std::uint8_t > Aleph::CA::ca_checkpoint_detail::inflate_bytes ( const std::uint8_t *  src,
std::size_t  len,
std::size_t  expected_uncompressed 
)

Decompress a DEFLATE byte buffer back to its raw form.

Parameters
[in]srcpointer to the compressed bytes.
[in]lennumber of bytes available at src.
[in]expected_uncompressedexpected size of the decompressed payload; the function pre-allocates the output buffer to this size and rejects the input if miniz returns a different length.
Returns
owning byte buffer with the decompressed stream.
Exceptions
std::runtime_erroron miniz failure or size mismatch.

Definition at line 66 of file ca-checkpoint-compress.C.

References ah_runtime_error_if, Aleph::blossom_maximum_cardinality_matching(), and out.

Referenced by read_raw_payload().

◆ inflate_bytes_up_to()

std::vector< std::uint8_t > Aleph::CA::ca_checkpoint_detail::inflate_bytes_up_to ( const std::uint8_t *  src,
std::size_t  len,
std::size_t  max_uncompressed 
)

Decompress a DEFLATE byte buffer with an upper bound.

Unlike inflate_bytes, this overload only requires that the actual uncompressed size is <= max_uncompressed. The returned buffer is resized to the actual decompressed length. Used by the delta reader, where the exact uncompressed size is not stored in the header.

Parameters
[in]srcpointer to the compressed bytes.
[in]lennumber of bytes available at src.
[in]max_uncompressedupper bound on the decompressed size.
Returns
owning byte buffer trimmed to the actual decompressed length.
Exceptions
std::runtime_erroron miniz failure or when the output would exceed max_uncompressed.

Definition at line 91 of file ca-checkpoint-compress.C.

References ah_runtime_error_if, Aleph::blossom_maximum_cardinality_matching(), and out.

◆ make_tmp_path()

std::filesystem::path Aleph::CA::ca_checkpoint_detail::make_tmp_path ( const std::filesystem::path &  target)
inline

Generate a temporary sibling path for atomic writes.

The temporary name lives next to target so the final std::filesystem::rename is an atomic, same-filesystem operation.

Parameters
[in]targetfinal path that the caller wants to publish.
Returns
path of the form <target>.tmp.<pid>.<rand>.

Definition at line 223 of file ca-checkpoint.H.

References Aleph::blossom_maximum_cardinality_matching(), Aleph::count(), r, rng, and Aleph::suffix().

Referenced by atomic_write_file().

◆ read_le()

template<typename T >
T Aleph::CA::ca_checkpoint_detail::read_le ( std::istream &  in)
inline

Definition at line 201 of file ca-checkpoint.H.

References Aleph::blossom_maximum_cardinality_matching(), and value.

◆ read_raw_payload()

std::vector< std::uint8_t > Aleph::CA::ca_checkpoint_detail::read_raw_payload ( const std::filesystem::path &  path,
const Checkpoint_Header &  h 
)
inline

Read the raw, uncompressed payload from a checkpoint file given a validated header.

Definition at line 883 of file ca-checkpoint.H.

References ah_runtime_error_if, h, header_bytes_v1, header_bytes_v2, inflate_bytes(), and read_raw_payload().

Referenced by read_raw_payload().

◆ restore_frame_from_bytes()

template<typename Engine >
requires Checkpointable_Engine<Engine>
void Aleph::CA::ca_checkpoint_detail::restore_frame_from_bytes ( Engine &  engine,
const std::vector< std::uint8_t > &  bytes 
)

Inverse of snapshot_frame_bytes: spray the buffer back into the engine's write lattice in row-major order.

Definition at line 672 of file ca-checkpoint.H.

References ah_runtime_error_if, engine, k, Aleph::next(), restore_frame_from_bytes(), and Aleph::CA::Lattice< Storage, Boundary >::size().

Referenced by restore_frame_from_bytes().

◆ snapshot_frame_bytes()

template<typename Engine >
requires Checkpointable_Engine<Engine>
std::vector< std::uint8_t > Aleph::CA::ca_checkpoint_detail::snapshot_frame_bytes ( const Engine &  engine)

Flatten an engine frame into a row-major byte buffer.

Shared by save_checkpoint, Async_Checkpoint_Writer::submit and save_delta_checkpoint. The output is laid out with the host's little-endian representation of state_t, matching the on-disk layout.

Template Parameters
Engineengine type satisfying Checkpointable_Engine.
Parameters
[in]engineengine to snapshot.
Returns
owning byte buffer with cell_count * sizeof(state_t) bytes.

Definition at line 621 of file ca-checkpoint.H.

References Aleph::CA::Lattice< Storage, Boundary >::at(), Aleph::blossom_maximum_cardinality_matching(), engine, Aleph::CA::Lattice< Storage, Boundary >::rank, and Aleph::CA::Lattice< Storage, Boundary >::size().

Referenced by TEST().

◆ sync_directory()

void Aleph::CA::ca_checkpoint_detail::sync_directory ( const std::filesystem::path &  dir)
inlinenoexcept

Force a directory's metadata to be durable.

Required on POSIX to ensure a rename(tmp, final) survives a kernel crash. Windows reports the rename as durable via FlushFileBuffers on the renamed file, so this call is a no-op there.

Parameters
[in]dirdirectory whose metadata to sync.

Definition at line 291 of file ca-checkpoint.H.

References Aleph::blossom_maximum_cardinality_matching().

Referenced by atomic_write_file().

◆ sync_file()

void Aleph::CA::ca_checkpoint_detail::sync_file ( const std::filesystem::path &  path)
inline

Force a file's contents to be durable on the underlying storage device.

Wraps fsync (POSIX) and _commit (Windows) on a freshly opened file descriptor of path. Callers must have already closed any buffered C++ stream attached to the same file.

Parameters
[in]pathfile to fsync.
Exceptions
std::runtime_erroron open/sync failure.

Definition at line 251 of file ca-checkpoint.H.

References ah_runtime_error_if, Aleph::blossom_maximum_cardinality_matching(), and h.

Referenced by atomic_write_file().

◆ type_hash()

template<typename T >
std::uint64_t Aleph::CA::ca_checkpoint_detail::type_hash ( )
inlinenoexcept

Stable type hash of T.

Mixes the typeid name and sizeof so pointer-equal typeid strings still differ across template instantiations of the same name (rare but possible in mixed dynamic-library setups).

Definition at line 179 of file ca-checkpoint.H.

References Aleph::blossom_maximum_cardinality_matching(), fnv1a_64(), and Aleph::CA::splitmix64().

◆ write_header_bytes()

◆ write_le()

template<typename T >
void Aleph::CA::ca_checkpoint_detail::write_le ( std::ostream &  out,
const T &  value 
)
inline

Definition at line 188 of file ca-checkpoint.H.

References Aleph::blossom_maximum_cardinality_matching(), out, and value.

Variable Documentation

◆ format_version

constexpr std::uint32_t Aleph::CA::ca_checkpoint_detail::format_version = 2
inlineconstexpr

Current format version.

Bumped to 2 in Phase 17 to introduce the flags / compression_level / payload_size / delta_base_step trailer fields.

Definition at line 145 of file ca-checkpoint.H.

Referenced by write_header_bytes().

◆ format_version_min_read

constexpr std::uint32_t Aleph::CA::ca_checkpoint_detail::format_version_min_read = 1
inlineconstexpr

Oldest format version this file knows how to read.

Loaders accept any version in [format_version_min_read, format_version].

Definition at line 149 of file ca-checkpoint.H.

◆ header_bytes_v1

constexpr std::streamoff Aleph::CA::ca_checkpoint_detail::header_bytes_v1 = 8 + 4 + 8 + 4 + 4 + 4 * 8 + 8 + 1 + 7 + 8 + 8
inlineconstexpr

Size in bytes of the v1 header (Phase 15 layout).

Definition at line 152 of file ca-checkpoint.H.

Referenced by read_raw_payload().

◆ header_bytes_v2

constexpr std::streamoff Aleph::CA::ca_checkpoint_detail::header_bytes_v2 = header_bytes_v1 + 24
inlineconstexpr

Size in bytes of the v2 header (Phase 17 layout: v1 + 4 trailer fields of 4 + 4 + 8 + 8 = 24 bytes).

Definition at line 157 of file ca-checkpoint.H.

Referenced by read_raw_payload().

◆ magic_bytes

constexpr std::array<char, 8> Aleph::CA::ca_checkpoint_detail::magic_bytes = {{'A', 'L', 'E', 'P', 'H', 'C', 'A', '1'}}
inlineconstexpr

Magic prefix identifying an Aleph CA checkpoint stream.

The trailing digit is kept fixed at ‘'1’across format-version bumps; consumers discriminate viaformat_version` instead.

Definition at line 139 of file ca-checkpoint.H.

Referenced by write_header_bytes().