Binary checkpoint format for Aleph::CA engines (Phase 15 + Phase 17 crash-safe / compress / async writer). More...

#include <array>
#include <atomic>
#include <cerrno>
#include <chrono>
#include <concepts>
#include <condition_variable>
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <deque>
#include <filesystem>
#include <fstream>
#include <functional>
#include <iosfwd>
#include <mutex>
#include <optional>
#include <random>
#include <string>
#include <string_view>
#include <system_error>
#include <thread>
#include <type_traits>
#include <typeinfo>
#include <utility>
#include <vector>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <ah-errors.H>
#include <ca-checkpoint-compress.H>
#include <ca-rng.H>
#include <ca-traits.H>
#include <tpl_ca_concepts.H>

Include dependency graph for ca-checkpoint.H:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	Aleph::CA::ca_checkpoint_detail::Tmp_File_Guard
	RAII guard that removes a temporary file unless committed. More...

struct	Aleph::CA::Checkpoint_Header
	Public on-disk header surfaced by `inspect_checkpoint`. More...

struct	Aleph::CA::Checkpoint_Options
	User-tunable knobs for `save_checkpoint`. More...

struct	Aleph::CA::Resume_Token
	Resume handle returned by `load_checkpoint_into`. More...

struct	Aleph::CA::ca_checkpoint_detail::Header_Snapshot
	Header bundle pre-computed from an engine. More...

class	Aleph::CA::Async_Checkpoint_Writer< Engine >
	Background writer pumping snapshots onto a dedicated thread. More...

struct	Aleph::CA::Async_Checkpoint_Writer< Engine >::Task

class	Aleph::CA::Periodic_Checkpoint_Observer< Engine >
	Observer that auto-saves the engine state every `every` steps to a templated path. More...

Namespaces
namespace	Aleph
	Main namespace for Aleph-w library functions.

namespace	Aleph::CA

namespace	Aleph::CA::ca_checkpoint_detail

namespace	Aleph::CA::ca_checkpoint_flags
	Bitset values used in `Checkpoint_Header::flags` (v2+).

Concepts
concept	Aleph::CA::Has_Master_Seed
	Concept: rule type that exposes a master RNG seed.

concept	Aleph::CA::Checkpointable_Lattice
	Concept: lattice exposing the standard CA layout interface.

concept	Aleph::CA::Checkpointable_Engine
	Concept: engine that exposes the minimum surface needed by checkpoint save/load.

Functions
constexpr std::uint64_t	Aleph::CA::ca_checkpoint_detail::fnv1a_64 (std::string_view s) noexcept
	FNV-1a 64-bit hash.

template<typename T >
std::uint64_t	Aleph::CA::ca_checkpoint_detail::type_hash () noexcept
	Stable type hash of `T`.

template<typename T >
void	Aleph::CA::ca_checkpoint_detail::write_le (std::ostream &out, const T &value)

template<typename T >
T	Aleph::CA::ca_checkpoint_detail::read_le (std::istream &in)

std::filesystem::path	Aleph::CA::ca_checkpoint_detail::make_tmp_path (const std::filesystem::path &target)
	Generate a temporary sibling path for atomic writes.

void	Aleph::CA::ca_checkpoint_detail::sync_file (const std::filesystem::path &path)
	Force a file's contents to be durable on the underlying storage device.

void	Aleph::CA::ca_checkpoint_detail::sync_directory (const std::filesystem::path &dir) noexcept
	Force a directory's metadata to be durable.

Checkpoint_Header	Aleph::CA::inspect_checkpoint (const std::filesystem::path &path)
	Read just the header from a checkpoint file.

template<typename Engine > requires Checkpointable_Engine<Engine>
std::vector< std::uint8_t >	Aleph::CA::ca_checkpoint_detail::snapshot_frame_bytes (const Engine &engine)
	Flatten an engine frame into a row-major byte buffer.

template<typename Engine > requires Checkpointable_Engine<Engine>
void	Aleph::CA::ca_checkpoint_detail::restore_frame_from_bytes (Engine &engine, const std::vector< std::uint8_t > &bytes)
	Inverse of `snapshot_frame_bytes`: spray the buffer back into the engine's write lattice in row-major order.

template<typename Engine > requires Checkpointable_Engine<Engine>
Header_Snapshot	Aleph::CA::ca_checkpoint_detail::capture_header (const Engine &engine)
	Capture all header fields a checkpoint file needs.

void	Aleph::CA::ca_checkpoint_detail::write_header_bytes (std::ostream &out, const Header_Snapshot &hs, const std::uint32_t flags, const std::uint32_t compression_level, const std::uint64_t payload_size, const std::uint64_t delta_base_step)
	Serialise the v2 header bytes for a checkpoint file.

void	Aleph::CA::ca_checkpoint_detail::atomic_write_file (const std::filesystem::path &path, const Header_Snapshot &hs, const std::uint32_t flags, const std::uint32_t compression_level, const std::vector< std::uint8_t > &payload, const std::uint64_t delta_base_step, const Checkpoint_Options &options)
	Write `header + payload` atomically to `path`.

std::vector< std::uint8_t >	Aleph::CA::ca_checkpoint_detail::build_payload (const std::vector< std::uint8_t > &raw, const Checkpoint_Options &options, std::uint32_t &flags_out, std::uint32_t &level_out)
	Build the (possibly compressed) frame payload from a flat byte buffer, honouring `options.compress`.

std::vector< std::uint8_t >	Aleph::CA::ca_checkpoint_detail::read_raw_payload (const std::filesystem::path &path, const Checkpoint_Header &h)
	Read the raw, uncompressed payload from a checkpoint file given a validated header.

template<typename Engine > requires Checkpointable_Engine<Engine>
void	Aleph::CA::save_checkpoint (Engine &engine, const std::filesystem::path &path, const Checkpoint_Options &options={})
	Write a complete engine snapshot to disk atomically.

template<typename Engine > requires Checkpointable_Engine<Engine>
Resume_Token	Aleph::CA::load_checkpoint_into (Engine &engine, const std::filesystem::path &path)
	Restore an engine's state in-place from a checkpoint file.

template<typename Engine > requires Checkpointable_Engine<Engine>
void	Aleph::CA::save_delta_checkpoint (Engine &engine, const std::vector< std::uint8_t > &baseline, const std::uint64_t base_step, const std::filesystem::path &path, const Checkpoint_Options &options={})
	Write a delta checkpoint relative to a previous payload.

template<typename Engine > requires Checkpointable_Engine<Engine>
Resume_Token	Aleph::CA::apply_delta_checkpoint (Engine &engine, const std::filesystem::path &path)
	Apply a delta checkpoint to the engine's current state.

Variables
constexpr std::array< char, 8 >	Aleph::CA::ca_checkpoint_detail::magic_bytes = {{'A', 'L', 'E', 'P', 'H', 'C', 'A', '1'}}
	Magic prefix identifying an Aleph CA checkpoint stream.

constexpr std::uint32_t	Aleph::CA::ca_checkpoint_detail::format_version = 2
	Current format version.

constexpr std::uint32_t	Aleph::CA::ca_checkpoint_detail::format_version_min_read = 1
	Oldest format version this file knows how to read.

constexpr std::streamoff	Aleph::CA::ca_checkpoint_detail::header_bytes_v1 = 8 + 4 + 8 + 4 + 4 + 4 * 8 + 8 + 1 + 7 + 8 + 8
	Size in bytes of the v1 header (Phase 15 layout).

constexpr std::streamoff	Aleph::CA::ca_checkpoint_detail::header_bytes_v2 = header_bytes_v1 + 24
	Size in bytes of the v2 header (Phase 17 layout: v1 + 4 trailer fields of 4 + 4 + 8 + 8 = 24 bytes).

constexpr std::uint32_t	Aleph::CA::ca_checkpoint_flags::compressed = 0x1
	Frame payload was written through DEFLATE (miniz).

constexpr std::uint32_t	Aleph::CA::ca_checkpoint_flags::delta = 0x2
	File contains only the cells that changed relative to a baseline.

Detailed Description

Binary checkpoint format for Aleph::CA engines (Phase 15 + Phase 17 crash-safe / compress / async writer).

A checkpoint is a self-describing binary snapshot of an engine's state on disk. It captures everything required to resume a long- running simulation bit-by-bit:

A magic + version + engine type-hash header that lets load reject incompatible files early.
The lattice extents and the current step counter.
The full frame data, optionally DEFLATE-compressed.
The optional master RNG seed, when the rule exposes one through the Has_Master_Seed concept (the Per_Thread_RNG pattern from ca-rng.H).

Because every contextual stochastic rule in the module derives its per-cell engine deterministically from (master_seed, step, coord), restoring (master_seed, step, frame) is sufficient to reproduce the future trajectory bit-by-bit, regardless of thread scheduling or build mode.

Phase 17 additions:

Atomic-rename writes: snapshots are first written to a .tmp.<pid>.<rand> sibling, fsync'd, then renamed onto the final path. A kill -9 between flush and rename leaves any previous file intact.
Optional compression: Checkpoint_Options{ .compress = true } activates a DEFLATE pass on the frame payload (bumps the file to format_version = 2).
**Async_Checkpoint_Writer**: pumps checkpoint writes onto a dedicated worker thread so step() is never blocked by disk I/O.
Delta snapshots *(best-effort)*: when a baseline payload is available, save_delta_checkpoint persists only the cells that changed.

Backward compatibility: load_checkpoint_into and inspect_checkpoint accept both format_version = 1 (Phase 15) and format_version = 2 (Phase 17). New writes always emit v2.

Two top-level entry points:

save_checkpoint(engine, path[, options]) writes a complete snapshot.
load_checkpoint_into(engine, path) rebuilds the engine state in-place; a Resume_Token records the path and the validated header for later auditing.

Author: Leandro Rabindranath Leon

Definition in file ca-checkpoint.H.

Classes

Namespaces

Concepts

Functions

Variables

Detailed Description