Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

lazily-spec

Language-agnostic wire protocol specification for the lazily reactive signals family.

This site is the rendered companion to the lazily-spec repository. It defines the canonical message schemas shared across every lazily implementation:

  • lazily-rs (Rust)
  • lazily-py (Python)
  • lazily-zig (Zig)
  • @lazily/signaling (TypeScript / Cloudflare Worker)

Cell Model

Upstream of every transport, the Cell Model fixes how a cell’s value converges. A cell is either single-writer (local/direct, no merge) or multi-write, and a multi-write cell carries a pluggable merge: <mechanism> attribute. CRDT is the first multi-write merge mechanism (merge: crdt), not the only one — lww, ot, lease, and custom are reserved alongside it. All transports below carry cells classified by this model.

Protocol Layers

LayerSpecSchema
IPC (Snapshot + Delta)Wire Protocol § IPCsnapshot.json, delta.json
Cross-language FFIWire Protocol § FFIffi.json
Signaling (WebSocket)Wire Protocol § Signalingsignaling.json
Distributed (CRDT)Wire Protocol § Distributeddistributed.json
Capability negotiationWire Protocol § Capability Negotiationinline

Every layer in this matrix is required of every binding. The Distributed CRDT row and the required keyed cell collections layer are unconditional. The C-ABI FFI row is required by default with a narrow platform carve-out (a binding whose runtime cannot host a native in-process C ABI — e.g. browser/Worker JS — declares ffi = none and interops over the wire instead). The thread-safe and async reactive contexts are required where the platform supports them (a platform that structurally lacks threading or suspendable async declares thread_safe = none / async = none); the shared-memory payload path is required where the platform supports it, with an I/O-channel fallback (Inline payloads over IPC/WebSocket/WebRTC) when it does not. See the Binding Conformance Matrix for the full MUST/MAY breakdown and the carve-out terms.

Wire Format

All messages use JSON with serde-compatible tagging ("type" discriminant). Future binary codecs (bincode, postcard, protobuf) encode the same schemas — the JSON representation is normative.

Schema Format

Schemas are provided as JSON Schema (Draft 2020-12). Each implementation must validate against these schemas. See JSON Schemas.

Scope & non-goals

This repo extracts the cross-language, wire-protocol, and behavioral sections from lazily-rs/SPEC.md into a standalone reference. Every lazily-rs feature area is accounted for here exactly once: either normatively specified (with a link below) or explicitly marked Rust-specific. Rust-specific internals remain in the Rust crate and are intentionally out of scope.

Covered (normative, cross-language)

lazily-rs areaSpec
SourceCell / FormulaCell / Effect (reactive core) + driven FormulaCell (formula().drive())Reactive Graph, Cell Model, Wire Protocol § Eager Signals
ReactiveMap (SourceMap / ComputedMap, keyed collections)Cell Model § Keyed cell collections
SourceTree (ordered keyed tree)Cell Model § Ordered keyed tree
reconcile (LIS keyed reconciliation)Cell Model § Keyed reconciliation
SemTree (memoized semantic tree)Cell Model § Memoized semantic tree
stable_id (manufactured text identity)Cell Model § Manufactured identity
TextCrdt (free-text CRDT + re-parse)Cell Model § Free-text CRDT
SeqCrdt (move-aware sequence order)Cell Model § Move-aware sequence order
Tombstone GCCell Model § Tombstone garbage collection
StateMachine (flat FSM)State Machine
StateChart (Harel/SCXML)State Charts
ThreadSafeContext (thread-safe reactive graph)Reactive Graph § Context layers, Wire Protocol § Concurrency layers are required
AsyncContext (async reactive graph)Async Reactive Context, Wire Protocol § Concurrency layers are required
IPC Snapshot/Delta + ShmBlobArenaWire Protocol § IPC, Wire Protocol § Shared-memory payload path is required, Conformance Fixtures
FFI boundaryWire Protocol § FFI, ffi.json
Signaling (WebSocket)Wire Protocol § Signaling, signaling.json
Distributed CRDT plane (CrdtSync/WireStamp)Wire Protocol § Distributed, distributed.json
Permission boundary (RemoteOp/PeerPermissions)Wire Protocol § Permission Boundary
Capability negotiationWire Protocol § Capability Negotiation
Transport abstraction (IpcSink/IpcSource/DataChannel)Wire Protocol § Cross-language channels

Out of scope (Rust-specific implementation)

These lazily-rs features are implementation choices, not cross-language contracts. Other bindings pick their own; they MUST meet the normative contracts above but need not mirror Rust’s approach.

lazily-rs areaWhy out of scope
Context / ThreadSafeContext lock strategy (ReadStrategy, inline seqlock, typed cache fast-path)Internal scheduling/locking; each binding picks its own concurrency strategy. The existence of the thread-safe and async context surfaces is required where the platform supports it (Wire Protocol § Concurrency layers are required); only the lock internals are out of scope
SlotId internal representationVolatile internal handle; the wire-stable identity is NodeId / NodeKey
instrumentation (lock-site tracking)Rust diagnostics
str0m_backend / str0m_netConcrete Rust WebRTC backend (str0m crate); only the transport abstraction is cross-language
Performance benchmarksRust-specific measurement
lazily-serde type-erasure internalsRust serialization approach; the wire shape, not the codec implementation, is normative

Versioning

Protocol versioning follows the IPC capability negotiation: each session exchanges { protocol_id, protocol_major_version, codec } before any graph state flows. A major version bump is a breaking change; minor additions are additive.

Reactive Graph

The reactive graph is the dependency-tracking core of every lazily binding: a set of nodes whose values are derived from each other, where a change to a source invalidates only its transitive dependents and recomputation is pull-based and glitch-free. This chapter fixes the cross-language behavior — the lazily-formal kernel and the lazily-rs / lazily-py / lazily-zig / lazily-kt implementations are the executable references.

The reactive graph is compute, not protocol: only resolved values cross IPC/FFI as ordinary cell payloads. Every binding that ships a reactive graph MUST honor this contract.

The reactive family

A cell is a value-bearing reactive node — a node with a readable value. There are exactly two kinds of cell and one sink:

KindHandleNodeRole
sourceSource<T, M = KeepLatest>SourceCellWritten from outside. set replaces (invalidating dependents on a == (PartialEq) change; an equal set is a no-op); merge folds an op under MergePolicy M. M defaults to KeepLatest, so Source<T> is a plain source cell and Source<T, M> with M ≠ KeepLatest is what used to be called a MergeCell.
computedComputed<T>ComputedCellComputed from upstream. Tracks its dependencies automatically, computes on first read, caches, and recomputes only when read after an upstream invalidation. Guarded, always: an equal recompute suppresses downstream invalidation (matches TC39 Signal.Computed).
effectEffectA side-effecting sink — no readable value, so nothing can ever depend on it. Reruns whenever a tracked dependency invalidates; an optional cleanup closure runs before each rerun and on dispose. It sits outside the cell hierarchy, by capability, not by current degree.

Cell is the value-bearing-node concept, never a handle. The word Cell names “a reactive node with a readable value”; the two kinds of cell are the SourceCell and the ComputedCell (the arena nodes Node::Source(SourceNode) and Node::Computed(ComputedNode)). A caller never holds a Cell — the two handles are the concrete types Source<T, M> and Computed<T>. There is no Cell<T, K> genus struct and no phantom kind parameter: Source’s policy M is a real parameter of the Source<T, M> handle, present only where writes exist. Reactive is the umbrella adjective — the reactive graph is cells plus effects — never a type and never a synonym for one kind.

The partition is one axis with both sides covered and no leftover: a cell’s value comes either from outside it (a SourceCell) or from upstream of it (a ComputedCell). Effect has no value to read, so it cannot be folded in — the sink position is a real boundary, not a naming accident.

A plain cell — Cell ≡ Source<KeepLatest> — is the keep-latest instance of the source kind; this is a default type parameter, not a spec assertion. A binding MAY implement Source<T> as that instance or keep it as a distinct fast path with identical semantics.

The eager construct is an eager Computed, not a distinct kind. The eager construction is computed(compute).eager(): a guarded Computed plus a puller Effect that reads it on creation and after every invalidation, so its value is materialized by the time the invalidating set/batch returns (readers never see an intermediate unset state — relaycell-backpressure-analysis.md §4.0). .eager() is declarative and idempotent and returns the same Computed handle, mutated — so the per-write puller of the old Signal cannot be constructed (see Eager computed cells under Semantics, and §9.2.2’s theorem that a writer is always a sink). .lazy() is the reverse transition; is_eager() is the predicate, so the bare verbs are never confused with a query.

The normative eager semantics are four observable clauses — materialize once at creation, fresh at mutator return, once per flush rather than once per write, and disposal that removes only the puller. They are stated under Eager computed cells below and fixtured in conformance/reactive-graph/signal_*.json. The computed-plus-puller construction is the recommended way to satisfy them, not itself a requirement.

Values are lazy by default; call .eager() on a Computed when eager push semantics are required. Handles are the two concrete types — Source<T, M>, Computed<T> — plus EffectHandle: lightweight, copyable ids over a shared node table (arena slots; see Handles and identity), usable only with the owning context.

Read is on every cell; write is on the source kind. Both handles expose get (auto-subscribing) — and only get; there is no subscribe, because observation is a declared dependency edge, never a registered callback (see Reactives have no observers). Writing is not a supertype/subtype relationship but a kind restriction: set (replace) and merge (fold under the source’s policy) live on the inherent impl for Source<T, M> alone, so computed.set(…) is a compile error — no method found, no trait in sight. A Computed reads and never writes; a Source does both. The payoff (relaycell-backpressure-analysis.md §4.0): a composite reader that needs to accept either kind takes a small per-binding read-only view (an enum, or Go’s Cell[T] interface below) and the backend chooses the impl — pull-computed, push-fed source, or polling-computed — behind one type, so ownership of the mutation (not the type) decides the invalidation source. Where a binding has no way to restrict methods by type parameter (Go), a read-only interface Cell[T] carrying Get is reintroduced, with SourceCell/ComputedCell as structs — the same compile error under a different mechanism (§4 of the design).

API surface

Two constructors and one transition, symmetric with the kernel. source / computed / .eager() replace the eight old constructors (cell, merge_cell, computed, memo, slot, signal, get_signal, dispose_signal) — and because every Computed is now guarded, the old computed-vs-memo distinction is gone: memo is removed and computed is the guarded derivation.

MethodDescription
source(value)Create a Source<T, KeepLatest> — a plain mutable source cell
source::<M>(value)Create a Source<T, M> whose write folds under policy M (was merge_cell)
get(handle)Read either kind — a Source or a Computed — computing/refreshing a Computed if necessary (auto-subscribes the running computation)
get_shared(handle) (binding-specific; Rust Context::get_rc)Optional unified shared-owner read of either kind, avoiding a value clone while preserving get value, refresh, and dependency-tracking semantics (#lzrsgetarc)
set(handle, value)Update a Source and invalidate dependents (no-op on ==) — a compile error on a Computed
computed(compute)Create a lazy derived Computed, guarded (an equal recompute suppresses downstream — the guard is never optional, and memo folded into this). T: PartialEq, the same uniform bound as source
computed(compute).eager()Make the Computed eager (attach a puller Effect). Declarative and idempotent; returns the same Computed handle (was signal)
computed(compute).lazy()Revert an eager Computed to lazy (removes the puller only, keeps the value) — replaces dispose_signal; exists only if a binding needs the reverse transition (§9.3.4)
is_eager(handle)Predicate: whether a Computed currently has a puller attached
merge(handle, op)Fold op into a Source under its policy (; routes through set, so the == guard + store-without-cascade apply) — a compile error on a Computed
effect(run)Register a side-effecting computation (a sink); run may return a cleanup closure
dispose_effect(handle)Deschedule, drop edges, run cleanup
dispose(handle)Tear down any node kind: detach edges in both directions, clear the node, recycle its slot id. Disposing an eager Computed also tears down its puller (§9.3.4)
scope()Open a teardown scope: scope.source, scope.computed, and scope.effect create nodes disposed together when the scope ends
scope.disarm()Disarm a scope — ending it disposes nothing; its nodes revert to context ownership
batch(run)Coalesce several source updates into one invalidation + effect flush

Semantics

  • Pull-based, glitch-free refresh. A Computed that reads other cells always observes values consistent with the current inputs. On get, it first refreshes its own dependencies (recursively, lazy pull), then recomputes only if any dependency actually changed — it never observes a half-updated graph.
  • Unified shared-owner reads (#lzrsgetarc). A context may expose a binding-specific shared-owner form of get (Rust’s local spelling is Context::get_rc). When exposed as a generic alternate read, it is a read mode, not another cell kind: it MUST accept both Source and Computed, MUST return the same current value as get, MUST refresh a dirty Computed in the same way, and MUST register the same dependency edge when invoked through a running computation. It MUST NOT add a Clone/deep-copy requirement merely to read the value. Allocation identity, reference-count operations, and inline value fallbacks are binding-level mechanics and are not wire-visible. The formal pins are Reactive.readShared_eq_readCell, Reactive.trackedSharedRead_eq_trackedRead, and Reactive.trackedSharedRead_registers_edge in lazily-formal.
  • Every cell is guarded — one rule, two sides. A cell suppresses an equal value. On the source side this is the == guard on set: setting an equal value is a no-op, no downstream cascade fires. On the computed side it is the equality guard on recompute: an equal recompute suppresses downstream invalidation — the Computed’s value version does not bump, so subscribers see no change (matching TC39 Signal.Computed). Equality is structural/value equality, not reference identity, so two distinct-but-equal values suppress invalidation. T: PartialEq is the uniform bound on every cell, source and computed alike — the guard is not a mode a caller opts into.
  • There is no unguarded cell, and no equals:false escape. The guard is never wrong: if it suppresses an update you wanted, the value did not encode the change. To always propagate, make the value genuinely PartialEq-distinct (encode the distinction you care about into the value), or use a merge policy to express accumulate/always-apply semantics. This is not a library toggle, and there is no unguarded constructor — memo (the old guarded form) has been removed because computed is now guarded and the two are the same thing.
  • Dynamic dependencies. A tracking stack auto-discovers edges on each recompute: every get read inside a running Computed/effect registers a dependency. Stale dependencies from a previous run are removed before re-registering; a Computed that reads a different set of inputs on rerun has its edge set updated to match. There is no manual subscribe/unsubscribe.

Implementation note. The dedup that keeps edge registration idempotent is an implementation concern, not an observable one — the contract fixes the edge set, not how membership is tested. A binding MAY dedup by linear scan while a node’s degree is small; above a wide-fanout threshold it SHOULD promote to a hash-indexed edge set, so registration stays amortized O(1) in node degree and a wide-fanout graph does not degrade to O(n²) per propagation. The threshold matters in both directions: below it the linear scan is measurably the faster of the two, so an unconditional hash set is a regression on the common low-degree case (#lzspecedgeindex).

The threshold is not portable — measure it per binding. Naively it is where a scan of contiguous ids crosses the cost of one hash lookup, so it moves with both, and the same number can be right or wrong in the same language depending on unrelated choices. In lazily-rs that crossover measured near degree 170 with the standard library’s SipHash and near degree 40 once ids were hashed with a multiply-shift finalizer — a 4x shift from changing the hash function alone. lazily-dart measured 60 under AOT and 96 under JIT, a 1.6x spread from compilation mode with no code change at all. Copying another binding’s constant is how a promotion threshold ends up making mid-degree nodes slower than the scan it replaced. Across bindings the measured thresholds so far are 32, 64, 128 and 160 — there is no family constant.

Measure the hybrid, not the pure crossover. The crossover between a pure scan and a pure index is the wrong number, because a list arriving at degree T pays the full scan plus the one-time index build, with no indexed insert at exactly T to amortize against. Every candidate threshold therefore parks a regression on its own width, and the pure crossover cannot predict where. Sweep the real implementation against the unfixed tree across candidate thresholds and take the knee — lazily-dart’s pure crossover said 60–96, but the hybrid sweep put the worst-case regression at 1.63x for T=64 and 1.31x for T=128, so 128 shipped.

Two ways to get this measurement wrong, both observed:

  • Comparing always-indexed against always-scanning inflates the crossover. It charges the index’s overhead on every one-element dependency list to the wide list. lazily-cpp first estimated 96 this way against a true 32 — a 3x error. Sweep the threshold constant alone, with everything else fixed.
  • A width ladder’s narrow rungs cannot answer the low-degree question. A few hundred registrations is noise; lazily-dart measured 1.4x run-to-run variance at widths 2–4 and nearly reported a phantom regression at width 96 from ladder data. Low-degree behavior needs a separate high-repetition harness, not the tail of the ladder.

Two further hazards, both observed:

  • Demotion needs hysteresis, or it thrashes. A dependent list oscillates by one on every recompute, because edges are removed and re-registered, so a single shared promote/demote boundary makes a list sitting at the threshold rebuild its index on every recompute. Demote well below the promote threshold, or do not demote at all.

    The cost is severe where the hazard exists and varies far more than the mechanism suggests: ~4x in lazily-rs, 7.67x in lazily-js, and 21.5x in lazily-kt — each at exactly threshold+1 and within noise at every neighbouring width, which is why a ladder must cluster rungs there or it will not see this at all.

    It is not universal, and the reason is structural: the hazard needs a list that oscillates by one. lazily-zig has no demotion path, so nothing can thrash. lazily-dart clears its dependent list wholesale during cascade rather than removing one edge at a time, and no thrash was observable at threshold±1 (0.99–1.05x) — it keeps hysteresis as cheap insurance on the detach path, not because measurement forced it. A binding SHOULD check which shape its own recompute has before assuming either result.

  • A recycled id must not inherit an index. Where the index is held outside the node — a side table keyed by owner — its entries have to be dropped whenever the list is cleared or the owner is torn down. A binding that recycles ids will otherwise alias a stale index onto an unrelated node.

  • Disposal is explicit. Handles are copyable ids, not owners, so dropping every handle to a node reclaims nothing: without an explicit disposal call the node and its edges live as long as the context. A binding whose nodes can outlive their usefulness — anything with subscribe/unsubscribe churn — SHOULD expose disposal for computed and source cells, not only for effects, or a workload whose live size is constant still grows without bound in both memory and propagation cost. Disposal detaches edges in both directions; reading a disposed node afterwards is an error, the same contract as disposing an effect (#lzspecedgeindex).

    Detection of a stale handle is bounded, deliberately. A binding that recycles ids and checks only the node’s kind catches a handle naming a disposed node of a different kind, but cannot catch one whose id has been reused by a new node of the same kind — the classic ABA case. Closing that requires generational ids or reference-counted handles, and this spec does not require either: both cost either handle size or the copyability that fan-out depends on. Conforming bindings therefore reject the cross-kind case and MAY admit the same-kind one. Callers must not rely on read-after- dispose failing.

    Garbage collection does not substitute for this. The reverse edge set is a strong reference to each dependent, so a long-lived source retains every node that ever read it — the same unbounded growth a manual binding has, arrived at by a different route. A tracing binding that wants reclamation to follow reachability MUST make its back-edges weak; until it does, its disposal story is exactly the explicit one above.

  • Teardown scopes (scope). A scope records what was created through it and disposes that set when it ends. It bounds teardown, not visibility: a scope’s nodes read parent- and sibling-owned nodes freely, and scoping never restricts what an edge may point at.

    Ending a scope MUST be observationally equal to disposing each of its members individually — a scope introduces no disposal semantics of its own, it only names a set and a moment. Proved as disposeScope_eq_disposeAll in the standalone lazily-formal LazilyFormal.Reactive module. A binding is therefore free to implement the scope as a bulk sweep rather than a loop, and SHOULD, since reading each node’s kind from the arena at teardown is cheaper than a per-node dispatch.

    The resulting graph state depends only on the set of members, not on their order or multiplicity (disposeAll_order_independent). Effect cleanups are a different matter: they are side effects, so the order they run in is observable, and a binding MUST tear a scope down in reverse creation order — dependents before what they read — so a scope never transiently dangles inside itself and every binding runs cleanups in the same sequence. Order is therefore free for the edge bookkeeping and fixed for the cleanups.

    A scope carries the same hazard as dispose: ending it tears down its nodes even if something outside the scope still reads them. A binding MUST NOT present scope teardown as safe against that; only reference-counted handles close it, and they cost copyable handles.

  • Scope and reachability are different questions. Disposal and teardown scopes answer “this work is over — free it now”, deterministically, at a point the program names. Weak back-edges and reference-counted handles answer “is anyone still using this?”, and answer it whenever the collector or the last release gets around to it. Neither subsumes the other, and a binding SHOULD offer both rather than picking one:

    binding classscopereachability
    non-GC, no destructors (zig)scopes, ended at an explicit deinit/defernone available; explicit disposal is the whole story
    non-GC, destructors (rs, cpp)scopes, ended by the scope’s own destructorreference-counted handles, opt-in — they cost copyable handles, which fan-out needs
    tracing GC (js, kt, py, go, dart)scopes, ended by an explicit callweak back-edges, which need no user-facing API

    Reference-counted handles are opt-in in every class because a source read by two dependents cannot be moved twice, so making the handle an owner forces a clone at every fan-out capture site. Per-node destructor ownership is not a third option: a node an arena-stored closure can capture must outlive the capture, so a handle that borrows the context can only ever be a leaf — the constraint that makes the scope, not the node, the right unit of teardown.

  • Cycle detection. A Computed that depends on itself (directly or transitively) is detected during refresh and throws — the graph is acyclic by construction.

  • batch coalesces. Multiple set calls inside batch(run) queue their invalidation roots; at the outermost batch exit the roots propagate and effects flush once. Mutation inside a batch is synchronous; only the invalidation propagation is deferred to the boundary.

  • Effects are scheduled, not inline. An effect rerun is scheduled when a tracked dependency invalidates and runs in the subsequent flush (which may be the same tick, at batch exit). A rerun does not start until the previous cleanup completes. Disposal removes pending reruns, runs the current cleanup, and unsubscribes all dependency edges.

  • Eager computed cells (.eager()). Eagerness is not a kind — it is a Computed with a puller Effect attached, produced by computed(compute).eager(): the effect reads it on creation and after every invalidation, forcing it to re-materialize. Because the puller runs inside the invalidating set/batch’s effect flush, the value is fresh by the time the mutator returns. .eager() is declarative and idempotent — a second call is a no-op, so a Computed never acquires two pullers and the per-write over-compute is structurally unrepresentable. .lazy() (or disposing the Computed) reverts it to lazy behaviour (the backing value stays readable but is no longer eagerly kept fresh); is_eager() reports the current state.

    Normative eager semantics. The four clauses below are what a binding conforms to. They are stated as observations a caller can make, so that any implementation strategy satisfying them conforms — see Composition is recommended, not required below. (The conformance fixtures still carry the historical signal filenames; the concept is an eager Computed.)

    1. Creation materializes once. computed(compute).eager() MUST run compute exactly once at creation and MUST NOT expose an intermediate unset state. A reader immediately after creation observes the computed value without triggering a compute of its own.
    2. Fresh at mutator return. After a set that invalidates the eager Computed’s dependency cone returns, its value MUST already equal what compute yields from the current sources, with no intervening read. This is the clause a lazy Computed does not satisfy, and it is the operational meaning of “eager”.
    3. Once per flush, not once per write. Inside batch(run), an eager Computed whose dependencies are written N times MUST re-materialize once, at the outermost batch exit — not once per write. The puller is an effect and obeys Effects are scheduled, not inline; a Computed that re-materializes during invalidation rather than during the flush violates this clause even though it satisfies (2). N writes inside a batch MUST produce exactly one compute.
    4. .lazy() removes only the puller. Reverting an eager Computed to lazy MUST dispose the eager puller and MUST NOT dispose the backing value. After .lazy() the value remains readable, remains correct on read (it reverts to lazy recompute-on-read), and MUST NOT re-materialize on write. This is why the operation is a state transition back to lazy rather than a teardown, and why the old dispose_signal naming was an inaccuracy.

    Composition is recommended, not required. Clauses 1–4 are observable. “A guarded Computed plus a puller effect” is the construction that satisfies them and is what every binding SHOULD use — it is five lines over the public API and needs no teardown special case. It is deliberately not a MUST, because which nodes exist internally is not something a caller can observe, and this specification does not mandate unobservable representation (the same rule that lets a binding choose weak back-edges). A binding that welds eagerness into its computed-cell invalidation path conforms if and only if it satisfies all four clauses — and clause 3 is the one such a binding is most likely to fail, because re-pulling during invalidation is earlier than the flush.

    A binding MUST NOT make eagerness a node kind in its graph representation: the kernel’s node enumeration is SourceCell, ComputedCell, Effect — the two value-bearing cell kinds plus the sink. An eager Computed is a ComputedCell with an Effect, not a fourth kind to dispatch on. This is now a MUST NOT rather than the old SHOULD NOT, because computed().eager() makes the composition the only way to build eagerness: the handle a caller holds is the Computed itself, read with ordinary get, and there is no Signal type left to ship. The kernel stays closed because the DAG positions are closed — the same reason this specification declines to mandate weak back-edges is why it need not police a construction that cannot be written.

    Conformance: conformance/reactive-graph/signal_*.json (historical filenames).

    Measured 2026-07-20. What replaying the three signal fixtures against every context each binding ships actually found. Recorded as measurements, not as inferences from which types exist — the discipline established by the capability table above.

    BindingClauses 1, 2, 4Clause 3Construction
    lazily-jspasspasscomposed
    lazily-ktpasspasscomposed
    lazily-cpppass (Context)passcomposed
    lazily-pypassfailed, fixedwelded → composed
    lazily-dartpassfailed, fixedwelded → composed
    lazily-gopassfailed, fixedwelded → composed → Memo-backed
    lazily-zig4 fails on AsyncContextfailed on ThreadSafeContext, fixedcomposed

    Clause 3 caught four bindings across two unrelated mechanisms, and every one of them produced correct values while doing 2–3× the computes — which is why none of the other 11 reactive-graph fixtures saw them.

    Mechanism A, welded eagerness (py, dart, go). Three bindings independently grew a signal-specific slot subclass whose invalidation handler re-pulled the signal inline. Re-pulling during invalidation is earlier than the effect flush, so the compute count scaled with the number of changed sources. No effect existed anywhere in the construction. Nobody coordinated this; the same wrong design was reached three times in three languages.

    Mechanism B, a batch that does not bound the flush (zig). set flushed effects unconditionally while batch only nested a depth counter, so a correctly composed puller still ran once per write. A composition is not sufficient if the batch boundary does not gate the flush.

    The async surface is the family’s weakest, systematically. lazily-cpp’s AsyncContext exposes no signal API at all and its async slots carry no dependency graph; lazily-zig’s async context has no lazy mode, so every derived slot is eager whether or not a puller is attached and clause 4 is unobservable there; lazily-py and lazily-dart ship no async signal constructor. A binding MAY omit signal from a context — clause conformance is per surface, and a context that does not offer the constructor is not non-conformant. What it MUST NOT do is offer it and diverge silently.

    lazily-go was the deepest case and is now resolved. Its Memo implemented the == guard by recomputing during invalidation, so a memo-backed signal could not satisfy clauses 3 and 4 while a slot-backed one lost equal-recompute suppression — neither could hold, because the cascade consumed reverse edges as it walked. Marking a dependent clean without recomputing meant it never re-registered, so its source could no longer reach it and the next write was lost at depth two, which is hybrid_serves_stale_value_at_depth_two.

    Its sync plane now uses a non-consuming mark-frontier walk with a pull-time guard, converged on lazily-rs’s model rather than a new design. Two details from that convergence are worth recording for any binding attempting the same migration, because both were initially judged redundant and both were forced back by the corpus:

    • Notifying dependents when a slot’s value actually changed is what makes the pull walk order-independent. Without it, a dependent that refreshes in an unlucky order observes “no dependency changed” and keeps a stale cache. Full-cone marking does not subsume it.
    • A force-run flag on effects. Without one the guard cannot reach effects at all: a transitively scheduled effect runs before the memo can suppress it. The pull-time re-mark must not re-schedule effects, or an effect re-runs itself mid-flush.

    Its AsyncContext was deliberately scoped out and still does value-only suppression where the sync plane suppresses downstream entirely. That divergence between a single binding’s two planes is recorded in the binding rather than resolved here.

Reactives have no observers (#lzdartobservercow)

No reactive exposes an observer API. Not a Source, not a Computed, eager or lazy — no kind of cell. No subscribe, no on_write, no on_change, no add_listener, no callback collection of any kind attached to a reactive node. A binding MUST NOT provide one, and MUST NOT carry per-node storage reserved for one.

The clause is stated on reactives rather than on Cell because it was first written too narrowly and a second registry survived it: lazily-py carried a Signal.subscribe — documented in its own docstring as “an external (non-reactive) change callback”, with the real graph edges tracked separately — that a Cell-only prohibition did not reach. It deduplicated by equality via a set, the same defect this section elsewhere calls a MUST NOT. If the rule is worth having it is worth having on every node kind, so it is written that way.

This is a MUST NOT about a mechanism rather than a behavior, which is unusual for this document. It is stated that way because the mechanism cannot be made safe by constraining it — every constraint below was tried, written down, and abandoned.

Observation is a graph edge, not a callback

Reading a cell inside a computation declares a dependency. The tracking stack records the edge, invalidation propagates structurally, and the graph decides when dependents run. Nobody registers anything. That is the whole design, and every guarantee the family makes — batching, glitch-freedom, coalescing, cone-settled consistency — follows from the graph knowing what depends on what.

A callback list attached to a cell knows none of that. It cannot batch, because it has no notion of a cone to settle. It cannot be glitch-free, because it fires mid-update by construction. It cannot participate in scope teardown, because it is not a node. It is the observer pattern, living inside the reactive primitive and bypassing it.

This is the substance behind a common objection: “isn’t a reactive just an observer with extra steps?” While Cell.subscribe existed the answer was embarrassing, because in four bindings it was literally true — there was an observer registry inside the cell, and callers could reach it. The honest answer is that a reactive is not an observer, and the way to be able to say so is to not ship one. Observation is declarative and structural; if a caller is registering callbacks against a value, they have left the reactive model, and the library should say so rather than provide a door.

How to tell an edge set from a registry

A binding auditing itself against this clause needs a test that does not depend on what a collection is called, because naming is exactly what hid these:

Anything that survives an invalidation is not a graph edge.

Dependency edges are re-discovered on every recompute — the tracking stack clears them and the next run re-registers whatever it actually reads. That is what makes dependencies dynamic. A collection that persists across invalidation is therefore not participating in dependency tracking, whatever its name, whatever its docstring, and whatever it sits next to.

This criterion is stated because it was arrived at expensively. lazily-py carried three registries — on Cell, on Signal, and on Slot — and every one of them was either labelled or assumed to be dependency-graph state. Two separate readers, with the source open, misclassified one each. In every case the registry sat beside a real edge set with a similar name (_subscribers next to _parents), and in every case the deciding fact was a single line: the edge set is rebound and cleared on invalidation, the registry is iterated and kept.

Apply the criterion mechanically rather than reading intent. It resolves all three without argument, and it is the only check here that does not require trusting a description.

Effect and observer are not two spellings of one thing

The distinction is worth stating flatly, because the two look interchangeable at the call site and are not:

EffectObserver (removed)
Registrationimplicit — reading a value inside the body declares the edgeexplicit — hand a callback to a node
Positiona node in the grapha callback list hanging off a node
Runsonce per settled coneonce per write
Batchhonours it — one run per batchignores it — one call per write
== store-guardsees the coalesced resultcannot see a suppressed write at all
Computed guardrespects it — no run on an equal recomputenot subject to it
Merge sees converged state — the only guaranteed valuesees a flush-timing artifact (below)
Glitch-freeyes — inputs are mutually consistentno — fires mid-update by construction
Dependenciesdynamic; re-discovered every runnone; bound to one node forever
Teardowndisposed with its scopemanual, and outlives its scope
Per-node cost when unusedzerostorage on every node

Coalescence is the row that matters most, and it is not one mechanism but five. This family coalesces at every layer: the == store-guard drops an equal write entirely; the computed guard drops an equal recompute so downstream never learns; batch folds many writes into one invalidation and one flush; store-without-cascade skips effect scheduling for a cell whose cone holds no effect; and the merge algebra folds a run of ops into one state through . Every one of those is the graph deciding that some change does not need to be propagated — which is most of what makes a lazy reactive graph cheaper than recomputing everything.

Merge coalescence is where an observer fails hardest, and it is worth spelling out because it is not a quality-of-implementation matter. Associativity is the irreducible law of MergePolicy, and what it buys is variable flush points: a bounded relay may flush at any post-merge watermark and converge identically. The converged state is guaranteed; the sequence of intermediate values is not. Two runs of the same program over the same ops may legitimately produce different intermediate values under different backpressure, buffer sizes, or transports.

An Effect reads the converged state, which is the value the algebra actually promises. An observer fires on intermediate writes — so what it receives is an artifact of flush timing rather than data, and it is non-deterministic by design, not by defect. A caller cannot write correct code against it, because there is no contract there to be correct against.

Last-writer-wins sharpens this to a point. KeepLatest is old ⊕ op = op — the new op annihilates the previous state — and Cell ≡ Source<KeepLatest>, so every plain source cell in the family is already an LWW instance. Under a timestamped LWW register (CrdtJoin<LwwRegister>) a losing write is dropped outright: after convergence, it never happened. An observer that fired on that write reported an event the system has since decided did not occur, and any side effect it took — a log line, a queue push, a paint, an outbound message — is now describing a state that no replica will ever agree existed. An Effect never sees it, because it observes only what survived the merge.

That is the general shape of the whole objection, stated at its most concrete: observers report writes; the system’s actual semantics are about values that survive. Those are different questions, and in a coalescing, converging, distributed graph they diverge constantly.

An Effect is downstream of all four. It sees what the graph decided was worth propagating, which is why it can be glitch-free and why an unobserved subgraph costs nothing to write to.

An observer is upstream of all four and outside all of them. It fires on the raw write, before coalescence has a chance to apply. That is not a different observation strategy; it means a caller holding an observer is looking at a system whose central optimization is invisible to them, and cannot tell the difference between a change that mattered and one the graph suppressed.

Use an Effect

Everything an observer expressed, an Effect expresses:

// instead of: cell.subscribe(cb)
ctx.effect(|ctx| cb(ctx.get(&cell)))

The effect is batched, glitch-free, participates in teardown scopes, and is disposed by handle. Where a caller wants both the previous and the new value — the state-machine on_transition shape — the effect captures the previous value in its closure; see lazily-rs state_machine.rs for the reference form.

The behavioral difference is real and intended: under a batch, an effect observes the settled value. Writing A → B → C inside one batch reports A → C. Intermediate states are not observable, because that is what a batch asserts. A caller who did not want that should not have opened a batch.

Use a Topic when you need every transition

A Cell is a value: latest-wins, batched, glitch-free. A stream of every transition is a different thing, and the family already has it — Topic, present in all eight bindings, with cursors and durability. Topic.subscribe keeps its name because a topic genuinely is a subscription: an ordered stream a consumer reads at its own position.

The design error this section removes was a Topic hiding inside a Cell. A consumer needing every write — a mutation log, a replication feed, a persistence tap — should publish to a topic. That also makes the cost honest: only machines that actually expose a stream allocate one, and a plain cell pays nothing.

Why not keep it, constrained

Recorded so the argument is not re-run from scratch. Cell.subscribe was specified in detail before being removed, and each clause below was a genuine attempt to make it safe:

  • Firing order is registration order, because a set- or hash-backed collection reorders on rehash and lazily-go’s map iteration is deliberately randomized. Four bindings had four answers.
  • No deduplication by callback identity or equality, because two components subscribing the same bound method silently share one registration and the first to unsubscribe cancels the second. lazily-py deduplicated by equality, lazily-zig by address.
  • Subscribe during notify is deferred, or a self-feeding observer extends the loop it is running in and never terminates.
  • Unsubscribe during notify takes effect immediately, because in a manually-managed binding unsubscribe is routinely the step before freeing the state the callback reads, so one more call is a use-after-free.
  • Disposers latch, or an unlatched second call removes a later registration belonging to a caller who never asked.
  • Delivery is per write and batch does not coalesce it — which put the mechanism in permanent conflict with the batching model it sat beside.

Six clauses, four bindings, and the specification was still wrong twice in a single day: it omitted a violated clause in one binding, and its central argument cited two bindings that had never implemented the mechanism at all. A primitive requiring six normative clauses to be safe, that still diverges across the family, and whose delivery discipline contradicts the surrounding model, is not under-specified. It is misdesigned.

Two things settled it: memory carried by the graph itself, and semantics that were footguns at the edges.

The memory is measured, not estimated. Removing the observer API from lazily-zig moved @sizeOf(Cell(u64)) from 168 bytes to 32 — 136 bytes, 81%, reclaimed from every cell in every program whether or not anything ever registered. It was not only the callback collections: the reentrancy counters (notify_depth, before_notify_depth, and two tombstone flags) and the monotonic registration counter were all unconditional per-node state that existed solely to make the notify loop safe. At cell-family scale that is the dominant memory term in the graph, paid by every reactive value to support a feature with one caller family-wide. A reactive graph’s whole value proposition is holding many nodes cheaply; a per-node cost multiplied across the graph is the one kind of overhead it cannot absorb.

The semantics were worse, because the failures were all at the edges where they are hardest to find. Delivery that ignores batch while everything beside it honours it. Firing order that depends on a collection’s rehash. Two components sharing a bound method and silently sharing one registration, so the first to unsubscribe cancels the second. A disposer that removes a later caller’s registration. An observer invoked once more after asking to stop, which is harmless under a tracing collector and a use-after-free without one. Each of these is fine in the common case and wrong in a case the caller cannot see coming, which is the definition of a footgun rather than a bug: correct code and broken code look identical at the call site.

Underneath all of them is the defect no clause could patch: an observer cannot distinguish batching from coalescence. It receives a flat sequence of callbacks with no framing. Three invocations may be three separate updates or one logical update whose writes were grouped, and nothing in the callback distinguishes them — there is no signal for where an update begins or ends. Nor can it see what the graph coalesced away: a write dropped by the == store-guard, a recompute dropped by the computed guard, a flush skipped because the cone held no effect. So “the value did not change”, “nothing was written”, and “the graph decided this did not need propagating” are all the same non-event to an observer. It is an event stream stripped of both its transaction boundaries and its elisions.

This is not a missing feature to be added. It follows from sitting outside the graph: the graph is what knows where an update starts and stops, and a callback list attached to one node is structurally unable to observe that. An Effect has the framing for free, because running once per settled cone is the transaction boundary. The two mechanisms are therefore not two ways of observing a value — one can express “this update is complete” and the other cannot, at any level of specification effort.

Conformance

There are no observer fixtures. The clauses above were removed along with the mechanism, and conformance/reactive-graph/observer_*.json no longer exists. A binding conforms to this section by not having the API.

lazily-rs, lazily-cpp, lazily-js, and lazily-kt never implemented one and require no change. lazily-py, lazily-dart, lazily-go, and lazily-zig carried one and remove it, re-expressing on_transition as an effect.

The reactive-graph fixtures that remain cover disposal, teardown scopes, and eager computed cells. They require TeardownScope (ctx.scope() / disarm()) and dependency-graph introspection (dependents_of, dependencies_of, cleanup_order). Every binding replays this corpus as of 2026-07-19.

The eager-computed fixtures (historical filenames signal_*.json) need one observable the rest of the corpus does not: computes_of. It maps a node id to the cumulative number of times its compute function has run, counted from the start of the scenario. A runner MUST count every invocation of the compute, including the one at creation, and MUST NOT reset it per step. This key exists because an eager Computed and a lazy Computed return identical values for every read sequence — the only caller-observable difference between them is when compute runs, so a corpus that asserts values alone cannot distinguish computed().eager() from computed() and will pass against a binding that implements the former as the latter.

Three ops are specific to these fixtures. The caller-facing transitions are .eager() / .lazy()Eager computed cells); the fixture ops retain the historical signal / dispose_signal names, which a runner maps to those transitions (create-eager and revert-to-lazy). A runner MUST accept these op names — the runner panics on an unknown op, so this acceptance is the contract:

opshapemeaning
signal{id, reads, offset}create a guarded Computed (compute is sum(reads) + offset, the same convention as computed) and make it eager — the computed(…).eager() construction
dispose_signal{id}revert an eager Computed to lazy (.lazy()) — the puller only, not a node teardown, see clause 4
batch{writes: [{id, value}, ...]}perform every write inside one batch; invalidation propagates and effects flush once, at the outermost exit

batch is a single op rather than a begin_batch/end_batch pair so that a runner need not carry nesting state. Bindings whose batch API is a closure take the writes as the closure body; bindings with explicit begin/end call them around the writes. Note that batch also appears in the reliable-sync and collections areas with unrelated semantics — these are per-area op vocabularies, not one global namespace.

Fixture shape is declared, not inferred. Every ReactiveGraph fixture carries a top-level "shape" field, either "steps" or "scenarios". A runner MUST switch on that field rather than probing for whichever key happens to be present, and MUST NOT special-case a fixture by filename — the first runner written against this corpus did exactly that, which goes stale silently the moment a second scenarios fixture is added. The schema suite cross-checks the declaration against the keys actually present, so shape cannot drift from the fixture it describes.

The two shapes are not interchangeable and the split is deliberate. A steps fixture asserts a single trace. A scenarios fixture asserts a relation between two op streamsscope_teardown_equals_fold_of_disposals.json claims that ending a scope is observationally equal to disposing its members individually, and a single steps array structurally cannot express “these two paths must agree.” It names the scenarios that must agree in expected.observationally_equal.

One trap in that fixture, stated here because every runner will hit it: cleanup_order is cumulative across a scenario, not per-step. The individual_disposal scenario spreads three disposals across three steps and pins the whole resulting order on the last of them, while scope_teardown produces all three from a single end_scope. A runner reading cleanup_order per-step will see the scenarios disagree and report a divergence that is not there.

The merge algebra and Source<T, M> (#relaycell)

A Source<T, M> is a source cell whose write is a merge rather than a replace: merge(handle, op) computes ⊕(current, op) under MergePolicy M and routes the result through set — so the == store-guard, store-without-cascade, and batch all apply unchanged. A plain Cell is exactly Source<KeepLatest> (the keep-latest instance, the default of the one source kind); a binding MAY implement it as that instance or keep it as a distinct fast path with identical semantics. The policy M is a real parameter of the Source<T, M> handle, so it exists exactly where writes exist and is absent on the computed side — never a third parameter every signature must spell.

Theorem — merge policies must be cheap (§9.2.1). MergePolicy::merge MUST NOT block and SHOULD be O(1)-ish in the size of old. The algebra is the convergence guarantee, so the fold stays synchronous; the async or expensive work that produces an op lives in the Effect that feeds the cell, never in the fold. In a ThreadSafeContext the fold runs under the mutex, so an expensive merge holds the lock for its duration. This constrains implementations rather than observable behaviour, so it is review-enforced, not fixtured — the same construction the observer prohibition uses.

Feeding a Source from another reactive

A recurring question, answered here because the answer is not obvious and the failure mode is one cycle detection cannot see.

A Source never acquires a dependency edge. That is what makes it a source: source and computed partition the graph by incoming edges, so a node fed by the graph would be both, and Computed → Source → Computed would become constructible. So “feed this source cell from that reactive” is not a new capability on the cell.

Theorem — a writer is always a sink (§9.2.2). Writing is not having a value: anything whose job is to write reads something and produces no readable value, which is incoming edges and no value — the Effect position. So a network-fed cell, a feedback writer, and an eager puller are each an Effect, not a new node kind. The kernel is closed because the DAG positions are closed. The family reached this answer three times from three directions — eager values (proposed as Signal), feedback (proposed as FeedbackEffect), and writer encapsulation — and each was an Effect composed with set/merge/.eager(). Stating it once retires the question; only the ergonomics (a named feed_async constructor) stay open, and those wait on two real call sites.

So feeding a source is an Effect that reads the reactive and calls merge:

effect(|ctx| { merge(acc, ctx.get(upstream)) })

The edge belongs to the effect. The cell stays edge-free, the partition holds, and the construction needs no new node kind — the same answer the family gave for eager values.

Delivery is per settled cone, not per write. The effect runs once per flush carrying the post-coalescence value, so 1 → 2 → 3 inside a batch produces one merge, of 3. This is a feature rather than a limitation: the intermediates were never materialized, because eliding them is what a lazy graph is for. An Effect is the only mechanism with the framing to do this correctly — it sees a settled value at a boundary, where an observer would see raw writes with neither.

Therefore merge granularity is flush granularity, and this MUST be stated wherever the construction is offered. With a non-idempotent policy (+, count, append-to-log) the accumulated result depends on how writes were batched: three unbatched writes fold three times, the same three writes inside a batch fold once. That is not a defect — it follows from Effects are scheduled, not inline — but the accumulator is the case callers reach for first, and it is the case where the difference is visible.

For an exact fold over every operation, do not drive it from a dependency edge. Drive it from explicit merge calls or from a Topic:

drivermerges performedfold is over
explicit merge() callsone per call, batched or notevery op — exact
Topic subscriptionone per eventevery op — exact
dependency edge, via an effectone per settled conesettled values — flush-granular

The first two are exact because the caller or the topic decides how many operations exist. The third is flush-granular because the graph decides, and deciding not to produce an intermediate is the graph working as designed. This is the same event-versus-state split recorded in relaycell-backpressure-analysis.md — retained events versus coalescible state — and it is why the answer to “I need every transition” is Topic, not a new capability on a reactive.

merge folds synchronously inside a batch; only propagation defers. This follows from merge routing through set and from Mutation inside a batch is synchronous, but it is stated explicitly because “does batching lose my merges?” is the first question the accumulator case raises. It does not. Every merge call folds when it is called. What a batch defers is the invalidation and the flush, so N calls inside a batch produce N folds and one invalidation.

Feedback: the construction cycle detection cannot see

An effect that reads R and merges into M, where M is upstream of R, closes a loop through the scheduler rather than through the graph. It is not a dependency cycle, so the acyclicity check will not fire.

Theorem — feedback is when an argument is also a dependency (§9.2.3). An Effect relates to a cell in two ways that look alike and behave nothing alike. A write target is an argument: passed in, captured by the closure, known statically, creating no edge — and, because set/merge are kind-restricted to Source, an effect cannot even take a Computed as a write-argument (it does not compile). A read is a dependency: discovered at run time by the tracking stack and re-discovered on every rerun. Feedback is exactly the case where an effect writes a cell it also reads — a write-argument that is also in deps(E). Nothing about that is a graph cycle (the argument is no edge), which is why acyclicity never fires and the loop closes through the scheduler. Prefer this phrasing over “an effect that writes into its own dependency cone,” which makes a scheduler property sound like a graph one.

This is deliberate and it is the family’s only way to express feedback: the dependency graph is acyclic by construction, so a cycle cannot be an edge. Closing it through the scheduler makes each iteration a flush, which bounds it in time and makes it observable — a discrete-time recurrence rather than an unbounded walk:

x_{n+1}  =  x_n ⊕ f(x_n)

Termination needs two properties, and the policy’s algebra supplies only one.

Where is a join on a semilattice — idempotent, commutative, associative, the properties a CRDT policy carries — every step satisfies x_{n+1} ⊒ x_n, so the state traces a non-decreasing chain. That is the ascent, and it is all the join gives you.

It does not give you termination. The chain stabilizes only if the lattice also satisfies the ascending chain condition — no infinite strictly increasing chains — and idempotence, commutativity and associativity imply nothing about chain height. A G-Set or OR-Set over an unbounded domain is a perfectly good join semilattice whose chain ascends forever if f keeps producing fresh elements; LWW over unbounded timestamps is the same. Most CRDT policies are not finite-height. A policy declaration therefore does not certify that a feedback loop over it terminates; ACC is a separate property of the value domain and must be established separately.

ACC and bounded height are different properties, and the weaker one is the one that matters. ACC says no ascending chain is infinite. Bounded height says there is a single constant H bounding the length of every chain. Bounded height implies ACC; the converse fails. A witness: take , a top , and for each n ≥ 1 a disjoint ladder (n,1) ⊏ … ⊏ (n,n), with joins across different ladders landing on . Every ascending chain is finite — so ACC holds — while chain lengths are unbounded, so no H exists. Termination needs only ACC. The distinction is recorded because a flag can only ever declare the stronger property (see below), so the two must not be written as synonyms.

Where ACC does hold, the == store-guard is the fixpoint detector: once the join stops moving the value, nothing invalidates and the cascade ends. f need not be monotone for the ascent, which is why this is not the classical monotone-framework result — Kleene iteration requires a monotone transfer function and delivers the least fixed point. This construction reaches a fixed point and offers no leastness guarantee.

The termination condition, stated correctly, is x ⊕ f(x) == x — the loop ends exactly when the store-guard suppresses. Two consequences that an earlier draft of this section got wrong:

  • An identity is sufficient, never necessary, and need not exist. If f(x) is the policy’s identity the step is a no-op, but any absorbing or saturating value is equally a fixed point — a Sum at its maximum absorbs every op. And a MergePolicy is only required to be an associative fold; nothing mandates a unit. KeepLatest is a right-zero band and has none, so “terminates when f yields the identity” is not merely false there, it is undefined.
  • Correspondingly, a non-idempotent policy does not diverge by construction. It has no guaranteed fixed point; whether it reaches one is a property of f and the value domain, and the caller owns that argument.

The three termination classes

The split is not two-way, and the default policy is in the third class.

recurrencetermination
join with ACCmonotone ascent, no infinite ascending chainalways halts
join without ACC — G-Set, OR-Set, LWW over unbounded domainsmonotone accumulationhalts iff the accumulated set is finite; semi-decidable
KeepLatest and other idempotent-but-not-commutative bandsx ⊕ op = op collapses it to x_{n+1} = f(x_n)undecidable — unrestricted iteration of an arbitrary function
non-idempotent — +, append, countersaccumulatesno guarantee; halts at an absorbing or saturating value

KeepLatest is the case a caller will actually hit, because Cell ≡ Source<KeepLatest> — an effect that reads a Computed and writes back to a plain source cell is the most reachable feedback loop in the family. Under a right-zero band the merge discards prior state entirely, so there is no ascent, ACC is irrelevant, and the lattice framing does not apply at all. What remains is x_{n+1} = f(x_n) over unbounded state with arbitrary f, which is Turing complete: no analysis can decide in general whether such a loop halts.

A caller in that class cannot appeal to the algebra. They must supply one of: a decreasing measure, an iteration bound, or a cancellation observed in the effect body (see below).

Accordingly:

  • A caller building a scheduler-closed feedback loop MUST be able to state why it terminates. The policy’s declared algebra is sufficient evidence only for a join over a domain satisfying ACC. In every other class the caller owes an argument the specification cannot supply.

  • Bindings MUST bound the effect-drain iteration count within a single flush, and report exhaustion rather than spinning. Note this is not re-entry depth: every binding surveyed guards re-entrant flushes and returns immediately, so the loop is a flat unbounded drain at constant stack depth, and a re-entry-depth bound would be pinned at 1 and could never fire.

    The bound’s value is deliberately unspecified and is not part of the contract — it is a binding’s tuning parameter, and pinning a number would make a legitimately long cascade non-conformant. What is normative is that the drain has an exit other than an empty worklist, and that taking that exit is observable: exhaustion MUST surface as a distinguishable outcome (a raised error, a reported diagnostic, a context-level status) and MUST NOT be a silent truncation of the cascade. A silently truncated flush leaves the graph in a state no clause of this specification describes — dependents marked dirty with their effects never run — which is worse than the livelock it replaced, because it is indistinguishable from convergence.

    Why this was raised from SHOULD. Three flush loops were read directly (lazily-rs Context::flush_effects and ThreadSafeContext::flush_effects, lazily-js flushEffects). All three are re-entrancy-guarded flat drains whose only exit is an empty worklist; none bounds anything. A SHOULD that no surveyed binding implements is not a requirement, and the failure it permits is a silent livelock on a thread that has starved whatever would diagnose it. The bound is also the enabling condition for fixturing this section at all: a conformance runner cannot replay a divergent feedback loop against a binding whose only exit is convergence, because the fixture hangs the runner. This section has stood as unfixtured normative prose precisely because there was no bounded failure mode to assert against.

Where a cancellation can be observed. In a synchronous context the flush does not return to the caller between iterations, so the calling thread cannot observe an interrupt and in fact starves whatever would deliver one. The available yield point is the effect body, which is caller code running once per iteration; declining to write ends the cascade, because an empty worklist is the drain’s only exit. Async contexts additionally suspend between iterations. Unmeasured: whether any binding’s async drain observes cancellation between iterations has not been checked.

Exhaustion SHOULD say what was cycling. A bound that reports only that it was hit sends the reader to a debugger on a thread that has just stopped livelocking. A binding SHOULD report the nodes that re-ran repeatedly and, where cheap, the repeating values — the difference between “drain exhausted after N iterations” and “drain exhausted; acc and total alternated”. Only the second is actionable. Representation is a binding’s choice.

Termination state belongs in the graph, not in a closure

The stop predicate that ends a feedback loop SHOULD be a Computed, read by the effect, rather than a branch buried in the effect body:

done = computed(|ctx| ctx.get(acc) >= threshold)
effect(|ctx| { if ctx.get(done) { return } merge(acc, f(ctx.get(upstream))) })

This is a recommendation about where to put a condition, not new surface — it is a Computed and an Effect, both already kinds. What it buys:

  • It is inspectable. Other nodes may read done; a UI can show it; another effect can react to it.
  • It is fixturable. A fixture can assert done transitions and that merging stops. A predicate inside an effect body is opaque to the corpus, which is part of why this area went untested.
  • It is reviewable. The termination argument the clause above requires a caller to state becomes a named node rather than a comment.

The shape is legal under the source/computed partition, which is worth checking explicitly because it looks circular: acc → done → effect → acc. The final hop is a write, not a dependency edge, so acc acquires no incoming edge and no dependency cycle exists. It remains scheduler-closed, one iteration per flush.

It does not make termination decidable. A done that never becomes true diverges exactly as before; the drain bound remains the backstop.

Accumulators

An accumulator is a Source<M> under an accumulating policy. That is what the merge algebra is for.

A Computed cannot be an accumulator, for two independent reasons, and the second is a live hazard:

  1. No self-edge. A Computed‘s value is a function of its dependencies’ current values; accumulation is a function of history. Reading its own previous value would be a self-dependency, which the acyclic graph forbids.
  2. Recomputes are elidable, so an impure workaround silently loses data. A compute body closing over a mutable counter compiles everywhere and looks correct. But the graph is free not to run it: the guard suppresses a recompute whose value is unchanged, a Computed with no readers never runs, and a batch coalesces N invalidations into one recompute. Each of those drops an increment. A binding MUST NOT document or example this pattern.

The structural rule, stated once because it decides these questions generally:

Memory of its own past implies a Source. A Computed must be recomputable from its dependencies, and a value determined by history is not. This is why a Source<M> is a source even though it computes — ⊕(current, op) reads its own prior state, which no Computed may do.

The exception that confirms it: a Computed folding a Topic’s retained events is an ordinary derivation, because the history is materialized in the topic and the fold is a pure function of current state.

patternwhere history lives
Source<M> + accumulating policyin the cell (a source)
Computed folding a Topic’s retained eventsin the topic✅ the fold is pure
Computed closing over a mutable counternowhere reachable❌ loses increments under elision

Recall also that driving merge from a dependency edge is flush-granular, so an accumulator fed that way counts flushes, not writes. For an exact count, drive it from explicit merge calls or from a Topic.

There is no fixpoint construct, and this records why

A termination construct was designed for this section and declined. Recorded briefly so it is not re-derived; this is rationale, not specification.

The proposal was a fixpoint restricted to policies declaring a new BOUNDED_HEIGHT, plus a fixpoint_with(cell, f, measure) taking a strictly-decreasing measure. Three reasons it failed:

  1. The flag would be unverifiable. COMMUTATIVE and IDEMPOTENT are honest because they are refutable by sampling — one counterexample disproves a claim; neither is confirmable, and this document does not pretend otherwise. ACC is not even refutable: every finite chain observed is consistent with it. The stronger bounded-height-with-constant is refutable and is vacuous — Max over a 64-bit integer has height 2^64. A declared property that is unverifiable or vacuous is worse than no property, and that test generalizes past this proposal.
  2. It could not cover the reachable case. Cell ≡ Source<KeepLatest>, and a right-zero band has no ascent, so an ACC-restricted construct is by construction absent from the class callers actually hit.
  3. A measure parameter adds no capability, and the construct is a composition promoted to a primitive. A caller who can compute a decreasing measure over their own state can compute a stop predicate over that same state — which is the Computed pattern above, written with kinds that already exist. Signal was retired for exactly this shape.

What the proposal genuinely offered was diagnosis, which is retained above as a SHOULD on exhaustion reporting.

Conformance for this section

Three fixtures are specified; they are not yet written.

fixtureasserts
feedback_drain_bound_reports_exhaustionA divergent loop — c a source cell, m a Computed of c + 1, an effect merging get(m) into c under KeepLatestterminates the flush and surfaces the exhaustion outcome. Asserts no iteration count, which is not contract.
feedback_converges_below_the_boundThe discriminating negative. The same shape with m = min(get(c) + 1, 3) under Max reaches its fixed point and must not report exhaustion. A binding that reports exhaustion for every feedback loop passes the first and fails this.
feedback_declining_to_write_terminatesPins the caller-side exit: an effect that stops merging ends the cascade as ordinary convergence, not an exhaustion report.

Deliberately absent: the bound’s value, the exhaustion outcome’s representation, and any assertion about how many times a given effect ran.

Unfixtured and unverified, stated explicitly. Until these exist, treat cross-binding agreement on this whole section as unverified — the eager-computed semantics sat in exactly this state for months while a binding shipped a different construction under a green checkmark. Separately unmeasured: whether any binding’s async drain observes cancellation between iterations. And the MUST above is currently satisfied by no surveyed binding; it describes required behavior, not observed behavior.

A MergePolicy is an associative fold ⊕ : T × T → T. The properties it satisfies are selected by the transport contract, not fixed (relaycell-backpressure-analysis.md §2):

PropertyRequirementPurpose
AssociativityAlways — the irreducible lawRegrouping a run of merged ops never changes the converged state, which is what licenses variable flush points: a bounded relay may flush at any post-merge watermark and converge identically. Not a flag; a law every policy MUST satisfy.
CommutativityPer policy (const COMMUTATIVE)The reordering tax — required only when ops may be applied out of order (concurrent producers / replicas / pages).
IdempotencyPer policy (const IDEMPOTENT)The durability tax — required only for at-least-once / crash-replay. For an idempotent , re-applying an op is a no-op, which is exactly the == store-guard one layer up: free dedup.

Policies are open. A binding MUST permit application-defined policies. The list below is canonical, not exhaustive: a MergePolicy is any associative fold with honestly declared flags, and a binding that ships a closed enum of policies has implemented a subset rather than the algebra. Merge semantics are domain knowledge — a text register, a set of tags, a running quantile, a per-tenant precedence rule — and the family cannot enumerate them in advance. A binding MUST expose the policy interface publicly, and SHOULD accept a custom policy anywhere a canonical one is accepted.

With one obligation attached, and it is the whole cost of the openness. An application-defined policy MUST declare COMMUTATIVE and IDEMPOTENT truthfully, and a binding MUST ship the law-test harness that checks a policy against its own declarations — the same harness that verifies the canonical policies, exposed for application use.

A policy that misdeclares its flags does not fail loudly. It converges correctly in every single-replica test, every in-order test, and every test without redelivery — and diverges only under the conditions the flag was supposed to license: a claimed-commutative policy that isn’t will disagree between replicas that saw the same ops in different orders, and a claimed-idempotent one that isn’t will drift under at-least-once delivery. Both surface as replicas that silently stop agreeing, arbitrarily far from the code that caused it. That is why the flags are verified rather than trusted for the canonical policies, and the reasoning does not weaken for a policy an application wrote. Shipping the checker is therefore part of shipping the extension point — an unverifiable declaration is worse than no declaration, because the relay acts on it.

The canonical policies (each names its algebraic structure and flags):

PolicyStructureCommIdem
KeepLatestold ⊕ op = opright-zero band
Sumold + opcommutative monoid
Maxmax(old, op)semilattice (total order)
SetUnionold ∪ opgrow-only semilattice
RawFifoold ++ opfree semigroup (concat)
CrdtJoin<C>C::merge_fromjoin semilattice

A replicated cell that accepts writes from more than one authority MUST use a commutative policy. This follows from the reordering tax above, but it is stated separately because the default is the trap: Cell ≡ Source<KeepLatest>, and KeepLatest is not commutative. A plain cell replicated to peers that also write it will converge differently depending on arrival order — the exact defect the algebra exists to prevent, arrived at by writing no policy at all. The multi-writer replicated case wants CrdtJoin<LwwRegister> or another commutative policy, chosen deliberately.

“More than one authority” is about who may write, not about topology. A replicated derivation pushed to a peer that only reads it has one authority and needs no commutativity — on that peer it is read-only, exposing only get without the Source write methods, which is the read-only replicated cell shape. Commutativity becomes mandatory the moment the receiving peer may also write.

Derived default with user override. A recurring shape worth naming, because it looks like it needs a new primitive and does not. Where a value has a computed default that a user may override:

  • Co-located — use the graph, not a policy. A Source holding the override (absent = defer), a Computed holding the derivation, and a Computed selecting override ?? derived. Glitch-free, self-documenting, and “reset to default” is clearing the override cell. No arbitration exists to get wrong.

  • Distributed — a merge policy is required, and a valid one exists: tag ops with provenance and fold by precedence (user outranks derived, LWW within the user rank). It satisfies all three properties — associative because a higher rank absorbs a lower one regardless of grouping, commutative because rank comparison is order-free and ties resolve by timestamp, idempotent because re-applying an op of equal rank and stamp is a no-op.

    The hazard is reset-to-default, and it is not obvious. Clearing an override is an op that lowers precedence, and a naive “a clear beats the user rank” rule is not commutative — a clear arriving after a newer user edit would wrongly win. The clear MUST carry a timestamp and lose to any user op newer than itself, which makes it an ordinary participant in the LWW rank rather than a special case. A binding that special-cases the clear will pass single-replica tests and diverge only under reordering.

KeepLatest (positional last-writer-wins, not commutative) is distinct from a timestamped LWW register (CrdtJoin<LwwRegister>, commutative): both conflate, they differ only on commutativity — the CRDT-vs-LWW branch. RawFifo cannot conflate (order and multiplicity are meaning); its only bounded-lossless option is Spill (Phase 3+). CrdtJoin<C> wires the existing cell CRDT units (Merge mechanisms) into the algebra without reimplementing their join.

Verification form. The three properties are algebraic identities over T values ((a⊕b)⊕c == a⊕(b⊕c), (a⊕b)⊕c == (a⊕c)⊕b, (a⊕b)⊕b == a⊕b), so a binding pins them with property-based law-tests (associativity for every policy; commutativity/idempotency asserted exactly when the flag is set, plus a counterexample proving a cleared flag does not lie) — lazily-rs uses tests/merge_laws.rs. The cross-language converged-state determinism invariant (same op multiset, any grouping → same egress) is additionally pinned by the mergecell_algebra.json compute fixture.

Invalidation propagation

When set changes a value (post-==-guard):

  1. The cell’s dependents are marked dirty (computed cells) or scheduled (effects).
  2. Dirty marks propagate transitively through computed dependents — a dirty Computed marks its own dependents, and so on. A Computed that recomputes to an equal value stops the propagation (the computed guard).
  3. On the next get of a dirty Computed, it refreshes: it pulls each dependency (recursively refreshing/ recomputing as needed), and recomputes only if a dependency actually changed value.

This is a push-invalidated, pull-recomputed graph — invalidation travels downstream eagerly (so effects fire), but the new value is computed lazily on read (so untouched branches do no work).

Store-without-cascade (the write-side dual of lazy reads). When set changes a value whose transitive dependent cone contains no Effect, the new value is stored (step 1’s dirty-marking of lazy computed dependents still happens, so a future subscriber reads the current value glitch-free — late-subscribe correctness) but no effect flush is scheduled — there is no active reactor to run. A binding MAY skip the flush machinery entirely in this case. Combined with demand-driven derivation on the read side, an unobserved reactive node — pull-derived or push-populated — costs approximately its raw storage: the merge cost law tiers the write cost by dependent kind (none → store only; lazy-only → store + O(deps) dirty-mark, no flush; active → store + dirty + flush). A burst of N value-changing writes with no interleaved active read pays the transitive dirty-mark once (dirty-marking is idempotent and monotonic — an already-dirty Computed is not re-walked), i.e. N·(==/⊕) + one dirty-propagation. See relaycell-backpressure-analysis.md §4.0.

Handles and identity

A handle is a Source<T, M> or Computed<T> (or EffectHandle) carrying a SlotId — and here Slot is the storage concept, not a reactive value: a slot is the arena position that holds a node, and it holds any kind (SourceCell, ComputedCell, or Effect). SlotId, SlotValue, and the slab vocabulary are accurate under this storage meaning and are unchanged by the kernel rename; the reactive-value node the arena holds is a SourceCell or a ComputedCell. The slot persists across recycling; its occupant does not (recycled_id_inherits_nothing).

Slot ids are minted monotonically and recycled on dispose. A disposed handle is inert: reads on a disposed Computed return its last cached value if any; reads on a disposed Source, or on a Computed whose eager puller was disposed, are undefined (the caller MUST NOT retain a handle past disposal). Re-entrancy of a disposed effect is prevented by removing it from the schedule before running cleanup. Detection of a stale handle is bounded (see Disposal is explicit): a binding that recycles slot ids MAY admit a same-kind ABA read unless its SlotId carries a generation tag.

Context layers

  • Single-threaded — the base context (mirrors lazily-rs Context). The graph is not Send/Sync; it lives on one thread/executor. Unconditionally required of every binding (it is the reactive core).
  • Thread-safe — a lock-backed counterpart (mirrors lazily-rs ThreadSafeContext); handles are clonable and the transition function and state are Send + Sync. Effects are scheduled and flushed within the invalidating send/batch, preserving glitch-free pull-based ordering. Required of any binding whose platform exposes preemptive multi-threading or shared-memory concurrency — see Wire Protocol § Concurrency layers are required.
  • Async — a separate reactive surface for future-returning computations; see Async Reactive Context. Required of any binding whose platform exposes an async/future runtime — see Wire Protocol § Concurrency layers are required.

The single-threaded context is the unconditional base; the thread-safe and async layers are required conditionally. A platform that structurally lacks either primitive (a strictly single-threaded runtime, a process/actor-isolation model, or a platform with no suspendable async computation) declares the matching thread_safe / async capability as none and advertises it, never silently. The flat State Machine and State Charts compose with whichever reactive context a binding ships, with identical send semantics.

Conformance

A reactive context conforms when:

  1. The kernel’s two cell kinds — nodes SourceCell / ComputedCell, handles Source<T, M> / Computed<T> — and the sink Effect are implemented; an eager Computed (computed().eager()) is the eager construct — not a fourth kind.
  2. Every cell is guarded on T: PartialEq: set is ==-guarded (equal value is a no-op) and computed is guarded on recompute (an equal recompute suppresses downstream). There is no unguarded mode.
  3. Refresh is pull-based and glitch-free: a Computed observes consistent inputs; untouched branches are not recomputed.
  4. Dependencies are tracked dynamically through a tracking stack (edges re-registered each recompute; no manual subscribe).
  5. Cycles are detected and throw.
  6. batch coalesces into one propagation + effect flush at the outermost exit.
  7. Effects fire scheduled (not inline), cleanup runs before each rerun and on dispose, and disposal unsubscribes edges.
  8. An eager Computed is materialized by the time the invalidating set/batch returns (eager push).
  9. Read on every cell, write on the source kind. Reads (get) are on every cell — Source and Computed alike. Any generic alternate read surface (for example the optional get_shared / Rust Context::get_rc) MUST cover the same two kinds and preserve ordinary-read value, refresh, and dependency-tracking semantics. Writes (set/merge) are on Source<T, M> alone — a compile error on a Computed, enforced by the type rather than a trait (a read-only interface in Go, §4 of the design). There is no subscribe — see Reactives have no observers.
  10. The merge algebra (#relaycell). merge(handle, op) folds under an associative MergePolicy and routes through the ==-guarded set (so an idempotent policy’s no-op merge fires no cascade). Cell ≡ Source<KeepLatest>. Every policy is associative; the COMMUTATIVE and IDEMPOTENT flags match the policy’s algebra (verified by law-tests); the converged egress state is independent of merge grouping/order for a commutative policy (verified by mergecell_algebra.json).

Declared context capabilities

A binding MUST declare, per context it ships, where that context sits on two independent axes. The declaration exists so the conformance harness can decide which fixtures apply. It has no other purpose, and the constraints below are part of the requirement rather than commentary on it.

Axis 1 — read discipline. Either blocking (reads return values) or async (reads return futures/promises). Mutually exclusive per context.

Axis 2 — concurrent access. One of:

LevelMeaning
noneSingle-threaded; no concurrent access contemplated.
serializedConcurrent callers are linearized against per-thread or per-realm graphs. No graph is shared.
shared-graphOne reactive graph is genuinely accessed from multiple threads.

The three-way split on axis 2 is not pedantry — it was added because a two-way thread-safe: yes/no flag would have been satisfied by four bindings offering materially different guarantees. Measured 2026-07-19: lazily-rs and lazily-py share one graph across OS threads; lazily-js shares an Atomics-backed lock across worker realms while each realm keeps its own graph; lazily-dart is a single-isolate reentrancy guard with no cross-isolate anything. Every one of those is individually honest and documented, but a fixture asserting concurrent-access behavior is meaningful only at shared-graph and vacuous at serialized. A flag that cannot tell them apart produces exactly the failure this chapter exists to prevent: a suite that passes while testing nothing.

Feature-group rule. A serialized context is a meaningful execution flavor when it supplies a distinct linearization boundary for reentrant or concurrent logical callers, even though its graph is realm-local. It joins every portable Core feature test that its surface supports (ordering, membership, atomic move, materialization). It does not join shared-graph stress/model-check tests. A binding may instead declare the context absent; then it is staged out of that peer group. Merely publishing a duplicate type is not support: an incomplete wrapper remains partial/absent, and a test runner MUST NOT fill its gaps with a base-context object, ad hoc mutex, or ordinary dictionary.

The axes are independent. Read discipline does not imply a concurrency level and vice versa. Single-threaded async is fully concurrent while requiring no thread-safety at all, and a blocking context may be shared-graph. A binding MUST NOT infer one axis from the other.

The declaration is not wire-visible. Context capability MUST NOT appear in any protocol message, influence sync behavior, or be observable by a peer. The reactive graph is compute, not protocol; only resolved values cross IPC/FFI, and convergence is a property of the merge algebra rather than of any replica’s local execution model. Associativity already licenses variable flush points, so local scheduling is unconstrained by the protocol — which is precisely why the execution model producing that scheduling is irrelevant to a peer. A replica running an async context and one running a shared-graph context converge because says so, not because they agree about threads.

This constraint is stated because the failure mode is quiet. Nobody sets out to leak the execution model into the protocol; it happens when a sync path needs a decision and a capability declaration is conveniently in scope. The declaration answers exactly one question — which fixtures run — and the moment it answers a second, the layering is broken.

Thread-safe context conformance

The lock-backed context (Context layers) is required of any binding whose platform exposes preemptive multi-threading or shared-memory concurrency. A binding’s thread-safe context conforms when it holds these deterministic properties under concurrent access:

  1. Handles are clonable, and the transition function and source/computed node state are Send + Sync; one reactive graph is shared across OS threads.

    This clause applies only to a context declaring shared-graph (see Declared context capabilities). A serialized context — lazily-js, whose graph is per-realm, or lazily-dart, whose guard is single-isolate — cannot satisfy it and is not required to. Conformance for those is the linearization property alone: concurrent callers are ordered, and none observes a half-updated graph. Requiring a shared graph of every “thread-safe” context would have made two bindings permanently non-conforming for having chosen a design their platform actually permits.

  2. Effects run glitch-free under concurrent access, preserving the same ordering as the single-threaded context — a concurrent reader never observes a half-updated graph. This mandates glitch-free ordering (every effect that runs sees the fully settled cone, never an intermediate state), not literal in-lock dispatch. A threaded binding MAY defer effect dispatch out of the graph lock, so a callback may re-enter the context, provided the ordering invariant holds: effects are delivered in dependency order and none observes a mixed state.

    (This clause previously governed observer callbacks, under #lzspecobserverclarify. Observers were removed from the family — see Reactives have no observers — and the requirement now attaches to effects, which are the only remaining way user code runs inside an invalidation wave.)

  3. The == (PartialEq) source guard and the Computed equality guard both hold under concurrent mutation: an equal write invalidates nothing, an equal recompute suppresses downstream work.

  4. The graph lock is released before user compute/effect/cleanup callbacks run, so callbacks may re-enter the same context without deadlock.

  5. An in-flight recompute is parked on a per-slot generation/condvar sidecar; a stale completion (the slot was invalidated during compute) is discarded and the waiter retried against a fresh value rather than observing a mixed state.

Verification form. Concurrent interleaving is not a deterministic load/replay sequence, so it is not pinned by a portable conformance fixture in lazily-spec. Each binding verifies its lock-backed context with a synchronization-model checker over the same semantics above — lazily-rs uses Loom (tests/thread_safe_loom.rs, behind the loom feature). The five properties above are the contract that model check exercises; a binding with no such tooling MUST at minimum exercise 1–4 under a threaded stress harness.

Implementation status

The single-threaded reactive context is required of every binding that advertises the reactive core. lazily-rs, lazily-py, lazily-zig, lazily-kt, and lazily-js implement it. The thread-safe and async counterparts are required of any binding whose platform supports them (see Wire Protocol § Concurrency layers are required); a platform that structurally lacks either declares the matching thread_safe / async capability as none and advertises it, never silently.

Measured 2026-07-19. Shipping a type named AsyncContext or ThreadSafeContext is not the same as implementing the capability, so this records what was found by reading each implementation rather than by counting type names:

Bindingasyncthread_safeNote
lazily-rsfullshared-graphreference; the only binding replaying the reactive-graph corpus
lazily-pyfullshared-graphAsyncContext added 2026-07-19 (d115000); it was the last binding without one
lazily-dartfullserializedsingle-isolate reentrancy guard; isolates share no memory
lazily-jsfullserializedAtomics/SharedArrayBuffer mutex shared across worker realms, graph is per-realm
lazily-gofullunmeasured
lazily-ktunmeasuredunmeasured
lazily-zigfullunmeasuredcascade rides the publish path rather than the invalidate path
lazily-cppstubunmeasuredsee below

lazily-cpp’s async context is a stub and MUST NOT be counted as implementing the capability: AsyncSlotNode carries no dependents or dependencies fields, get_async unconditionally recomputes on every call, and the synchronous get() returns a cached value that nothing ever invalidates. Depth tests “pass” through get_async only because nothing is ever memoized. Per the rule above it should declare async: none until it has a dependency graph, or implement one.

The unmeasured entries are deliberately not guesses. Absence of a finding is not a finding — that lesson is recorded in Reactives have no observers, where three bindings’ omission from a divergence table turned out to mean nobody had looked.

lazily Cell Model Specification

Normative source of truth for cell kinds and the multi-write merge mechanism.

This chapter defines the cell-kind model that the Wire Protocol serves. It is upstream of every transport: IPC, FFI, signaling, and the distributed plane all carry cells whose convergence semantics are fixed here. The Distributed: CRDT Cell Plane section specifies merge: crdt — the first multi-write merge mechanism defined below.

The single axis: writer count

A lazily graph is a set of cells. The only axis that determines how a cell’s value converges is how many writers can concurrently produce a value for itnot whether those writers are local or remote, in one process or many.

KindConcurrent writersConvergenceMerge
Single-writer (local / direct)exactly onedirect reactive push/pullnone
Multi-writepotentially manymerge: <mechanism> ingressmechanism-defined

The kind is a static property of what the cell represents, chosen at definition time. There is no dynamic per-write mode switching: a multi-write cell stays multi-write even when only one writer is currently live, and a single-writer cell never becomes multi-write because a value happens to arrive over a wire.

Single-writer cells (local / direct)

A single-writer cell has exactly one writer:

  • this runtime’s own derivation graph (a derived cell is always single-writer — see Derived cells), or
  • a single owning runtime that one-way mirrors the cell to other runtimes via a delta projection (today’s lazily-rs → lazily-kt #lazilystatesync push is exactly this shape).

Propagation is direct reactive push/pull over the IPC or FFI channels. A mirror is still single-writer: the receiving side observes, it does not write back. No merge step exists, and none is permitted — a single-writer cell that receives a concurrent remote write is a conformance error, not a merge.

A cell mirrored one-way to N readers is single-writer. Locality does not make a cell multi-write; a second concurrent writer does.

Multi-write cells

A multi-write cell admits concurrent writes from multiple replicas. New values arrive as remote ops merged through the cell’s declared mechanism; the merged result is fed into the reactive graph as an ordinary cell update, after which propagation is identical to a single-writer cell.

A multi-write cell carries a merge: <mechanism> attribute. The mechanism is a parameter of the cell, not a separate cell kind:

Cell      = SingleWriter
          | MultiWrite { merge: MergeMechanism }

This framing is deliberate. An implementation MUST model multi-write as one cell category parameterized by a merge mechanism, and MUST NOT hardcode a single crdt cell kind. New mechanisms slot in without a new kind, and every mechanism shares the ingress boundary, the merge-unit granularity, and the downstream propagation rules below.

Merge mechanisms

crdt is the first mechanism this spec defines and the only one with a normative wire schema today (distributed.json). The mechanism slot is open by construction so later mechanisms slot in alongside it. Every mechanism MUST be deterministic — replicas applying the same op set MUST reach the same value regardless of arrival order or lag.

mergeStatusConvergence strategy
crdtnormative (first)Conflict-free replicated data type (yrs/Yjs-family registers); converges without coordination. See Distributed: CRDT Cell Plane.
lwwreservedLast-writer-wins by HLC/Lamport timestamp.
otreservedOperational transform (server-ordered op rebase).
leasereservedLease/lock-serialized single-live-writer; degenerate-concurrency convergence.
customreservedApplication-supplied deterministic merge function.

crdt is chosen as the first mechanism because it converges without coordination — no central ordering authority, no lock round-trip — which is the property the editor-as-replica use case needs. Reserved mechanisms are named to fix the extension shape; an implementation MAY reject any mechanism it does not implement, but MUST reject it explicitly (capability negotiation) rather than silently treating it as crdt.

A multi-write cell with a single live writer degenerates to a near-free merge (no concurrent ops to reconcile) under every mechanism. This is why the kind is chosen statically by representation: a shared cell that is usually edited by one replica still declares merge: so that the moment a second writer attaches, convergence is already guaranteed.

Merge is an ingress operation on root cells only

The merge mechanism is an ingress step at the boundary where remote ops enter a replica. It applies only to root (input) cells.

Derived cells are never multi-write

A derived cell is a deterministic function of its inputs. It has exactly one writer — the derivation — and therefore is always single-writer. Replicas converge on a derived cell because they converge on its roots, never by merging the derived value itself. An implementation MUST NOT replicate or merge a derived cell directly.

Propagate guard (computed / computed_ripple_when / slot)

A derived cell carries a propagate guard: after it recomputes, the guard decides whether the recompute is forwarded to its dependents. The guard governs propagation, never computation — a derived cell is always recomputed when it is dirty and read (laziness is the only compute gate), so a reader MUST always observe a value current with the inputs. The guard only decides whether dependents are invalidated. This is the glitch-suppression that lets a diamond re-converge without re-running consumers whose input did not meaningfully change.

Three constructors expose the guard; all always compute, and differ only in the guard predicate changed(old, new) (propagate iff true):

  • computed(f) — the default. Guards by the value’s natural equality: changed = (old ≠ new). An equal recompute MUST NOT invalidate dependents. This is the #lzcellkernel guarded computed; equality semantics are the host language’s (PartialEq / == / equals), so for reference-typed values (e.g. a fresh array each recompute) suppression is host-defined and MAY not fire — this is permitted (an unsuppressed cascade is a missed optimization, never incorrect).
  • computed_ripple_when(f, changed) — the same guard with an explicit, pure changed(old, new) predicate: a cheaper/custom equality (dedup a large value by a version/hash field; epsilon compare; hysteresis; monotonic gate; or “propagate every N” when the counter lives in the value). changed MUST be a pure function of (old, new); value-carried state is a permitted input, external mutable state is NOT (it would key off recompute/read frequency and break determinism). computed(f)computed_ripple_when(f, ≠).
  • slot(f)pass-through: no guard (changed ≡ true). Every recompute propagates. The escape for values with no cheap equality (!PartialEq, non- comparable collections) that a binding is fine re-firing. This is a derived constructor and is distinct from the internal storage-sense Slot that a computed node occupies.

The guard is proved in lazily-formal as recomputeSlot_equal_preserves_dependents (equal recompute leaves dependents’ dirty flags untouched) and its specialization recomputeSlot_ripple_when_false_preserves_dependents for a custom changed.

Dependency tracking (the fortified compute view)

A derived cell discovers its dependencies dynamically: on each recompute it runs its compute function and records every cell that function reads. The recorded set is re-bound every recompute (not accumulated), so a conditional read (cond ? a : b) drops the branch it did not take — an implementation MUST NOT retain a dependency a recompute did not read.

The identity that a read must attribute to — which node is being recomputed — is carried into the compute function as a value, through a per-recompute compute view (lazily-rs: Compute, the sole implementor besides Context of the ComputeOps operations subset). It is NOT ambient (thread-local / module global). This is normative because ambient state is clobbered across suspension: an async compute that reads a dependency after an await would attribute it to whatever else ran on the executor. A value threaded through the closure survives suspension (it is captured), so it is the only mechanism that tracks correctly post-await — and the only one that works where no ambient carrier exists (browser JS has no AsyncLocalStorage). Bindings whose runtime does provide a suspension-surviving ambient carrier (Python contextvars, Dart Zone, Node AsyncLocalStorage) MAY use it; all others MUST thread the value.

The compute view SHOULD be fortified so misattribution is prevented by construction, not convention:

  • Sole tracking surface — a tracked read is available ONLY through the compute view; reading through the owning context registers no edge (the explicit untracked escape). A normal read therefore cannot silently miss tracking.
  • Non-escapable — the view MUST NOT outlive the recompute (lazily-rs binds it by lifetime and makes it !Send), so it cannot be stored and later replayed to register an edge against the wrong node.
  • Generation-stamped — a read against a node disposed/recycled mid-recompute is detected, never misattributed.

Edge-attribution invariant (normative): every dependency edge registered during the recompute of node n has n as its dependent. Because the node is a value parameter of the compute view, this holds by construction. Proved in lazily-formal as registerReads_dependent_is_recomputing_node.

Effects stay single-writer

Effects (irreversible external actions — send email, charge card, fire webhook) are not multi-write. State convergence does not authorize an effect to fire on every replica. Effects MUST be gated behind a single-writer authority (a designated peer or small consensus group) that decides when the effect fires, at-most-once. See Single-writer effect authority.

                 remote ops
                     │
                     ▼
        ┌──────────────────────────┐
        │  merge: <mechanism>      │   ← ingress, ROOT cells only
        │  (crdt | lww | …)        │
        └──────────────────────────┘
                     │  merged value as ordinary cell update
                     ▼
        root cell ──► derived cells ──► effects
                    (single-writer)   (single-writer authority)
                     └─ direct reactive propagation, identical for all kinds ─┘

Cell = merge unit

The cell is the unit of merge. Each multi-write cell converges independently; a merge mechanism operates within one cell’s value and MUST NOT move content across cell boundaries.

This is normative because the cell graph already supplies the natural merge boundaries: making the cell the merge unit makes cross-cell contamination impossible by construction. A whole-document merge that splices one logical region’s content into another (for example an agent’s console output bleeding into a queue region) is a conformance violation — each region is a distinct cell and converges on its own.

An implementation MUST scope every merge to a single cell. Coarser-grained merge (whole snapshot, whole document) is permitted only as an optimization that is observably equivalent to per-cell merge; if it can produce a result no sequence of per-cell merges could, it is non-conforming.

The per-cell merge is characterized as an associative fold by the merge algebra (#relaycell): a MergeCell<T, M> is a cell whose write folds under a MergePolicy M, and a plain Cell is MergeCell<KeepLatest>. The merge mechanisms below are the semilattice policies (CrdtJoin<C>) of that algebra; associativity is the invariant that lets a bounded relay flush at any watermark and still converge (Phase 2+).

Liveness vs mechanism

Whether a cell is multi-write (merge: present) is static. How many writers are live against it at a given moment is dynamic, governed by an attach/detach authority state machine (e.g. an editor plugin attaching or detaching). The authority SM governs only liveness — it never changes a cell’s kind or mechanism:

  • No live writer → the cell still declares its mechanism, but with zero concurrent ops the merge is inert and a durable replica MAY be ephemeral (rebuilt from a checkpoint on demand).
  • One or more live writers → ops flow and the declared mechanism reconciles them.

Mechanism is a property of the cell’s meaning; liveness is a property of the current session. Conforming implementations MUST keep these independent.

Cross-process liveness as a CRDT cell. When session liveness itself must cross a process boundary — “editor pid X has doc Y open”, “pid X holds the owner lease” — it is modeled as an ordinary multi-write cell on the CRDT plane, not as out-of-band state: an OR-set for open-set membership (observed-remove, so a re-open wins over a lagging close) and an LWW register for the per-pid alive flag / lease. The derived “is this doc live” aggregate is then a plain reactive memo over that liveness keyed map (the #lzfamilysync derived-aggregate contract), and an OS process-exit event is just the highest-stamp write to alive[pid]. This keeps liveness on the same convergent, idempotent, frontier-resumable substrate as every other replicated cell. Normative semantics: protocol.md § Liveness cells.

Conformance summary

An implementation conforms to the cell model when:

  1. Every cell is classified single-writer or multi-write, statically, by representation.
  2. Multi-write cells carry a merge: <mechanism> attribute; multi-write is not modeled as a hardcoded crdt cell kind.
  3. crdt is implemented as the first mechanism; unimplemented reserved mechanisms are rejected explicitly, never silently aliased.
  4. Merge is an ingress step on root cells only; derived cells and effects are never merged.
  5. Every merge is scoped to a single cell (cell = merge unit); no cross-cell content movement.
  6. Downstream reactive propagation is the same direct mechanism regardless of cell kind.
  7. The attach/detach authority governs writer liveness only, never a cell’s kind or mechanism.
  8. A keyed cell collection (ReactiveMap — the SourceMap/ComputedMap specializations) is implemented — entries are ordinary cells, a dedicated membership cell tracks the key set, and the value / set-membership / order reactivity-independence, stable-handle, and atomic-move invariants below hold. Collections are required of every binding, not optional.
  9. An ordered keyed tree (SourceTree) is implemented, inheriting the per-cell merge and atomic-move guarantees node-by-node (required of every binding).
  10. Keyed reconciliation emits the move-minimized {insert, remove, move, update} op set (LIS over prior indices preserved), and a stable entry is not invalidated by a sibling reorder (required of every binding).
  11. A reactive queue (QueueCell) is implemented — a FIFO collection whose reactive shell invalidates by reader kind (head / length / empty / full / closed), backed by a pluggable QueueStorage backend. The shell / storage split, closure observable contract, bounded-queue backpressure, and ordering guarantees below hold (required of every binding).
  12. Materialization is eager by default — a ComputedMap’s derived entries are pre-minted over the keyset; lazy materialization (get_or_insert_with mint-on-access) is opt-in, keyed, and observationally transparent — identical read values, allocation deferred only (see Materialization). It is a behavior, not a mode flag. Lazy evaluation (bounded-viewport recompute) is provided either way and is never conflated with lazy materialization.

Materialization (a caller-provided recipe)

Cell kind (above) fixes how a cell converges. Materialization is an orthogonal axis: it fixes when a derived cell’s backing node is allocated — not what it computes, not how it converges, not how it merges. It trades memory and first-touch latency against cold full-scan cost, and it MUST NOT be observable through the value of any cell.

Why a behavior, not a mode. Materialization was first pinned as a bespoke ReactiveFamily type carrying an eager/lazy mode — a reaction to one binding (lazily-zig) implementing lazy materialization in the spreadsheet benchmark and thereby diverging from the others. Standardizing a type with a mode invites re-divergence: each binding builds it slightly differently, and most never need it. What must agree is the observable behavior the benchmark measures — transparency (a lazy read equals an eager read) and deferral (an unread lazy entry costs nothing) — not any type or flag. So materialization is normative as a behavior of the keyed primitive: it is simply what ComputedMap does — get_or_insert_with mints a derived slot on first access (lazy); a pre-mint loop over the keyset is eager. There is no materialization mode and no family type — the family types (ReactiveFamily / CellFamily) are removed; ComputedMap (a ReactiveMap specialization) is the vehicle.

The materialization recipe

Materialization is caller-provided: a keyed collection (a SourceMap or any keyed address space) plus a per-key factory whose return type is the materialization choice. Nothing new is required beyond the cell/slot/signal primitives a binding already has:

  • Eager entry — the factory yields an input cell or an eager signal (a memo-slot + puller effect): the node is allocated/pulled up front; a read is a direct node access.
  • Lazy entry — the factory yields a lazy slot: the node is allocated on its first observe, addressed by key. A never-observed lazy entry is never allocated.

So the caller provides materialization through two levers it already owns — whether it observes a key (unread lazy entries stay unallocated) and what the factory returns (slot ⇒ lazy, signal ⇒ eager) — with no mode flag and no per-read toggle. This is the same “lazy by default, eager when asked (via signal)” model lazily already uses at the single-cell level, applied per key. Entry kind is the pinned axis:

  • Cell entries (H = Source) are input nodes — always materialized; an input has no derivation to defer. Minting an input on first get is a collection concern, not materialization.
  • Slot entries (H = Computed) are derived — the ones deferral governs: an eager factory allocates them up front, a lazy factory defers each to first observe.

Entry kind is orthogonal to the materialization choice (proved in lazily-formal’s Materialization module as cell_entries_materialized_in_every_mode / slot_entries_deferred_under_lazy): lazy defers only slot entries, never cell entries.

Normative rules on the recipe:

  • Eager is the default. Absent an explicit lazy opt-in, derived entries are eager — a read is a direct node access and a full recompute pays only compute. A binding MUST make eager the default.
  • Lazy is an explicit opt-in overlay on the eager core, addressed by key, never the default and never a per-read toggle on an eager handle. The first observe of key k constructs the same node the eager build would have, then caches it — a keyed overlay, not a second graph engine. A binding that offers lazy MUST expose it as an explicit opt-in (e.g. a keyed factory / keyed-context constructor).

Observational transparency (normative)

For every node and every read, the observed value MUST be identical under either mode. Materialization mode is not observable on the value axis — it changes allocation timing and memory, never results:

observe(build(eager, spec), id) = observe(build(lazy, spec), id) = spec.val(id)   ∀ id

This is proved in lazily-formal’s Materialization module (observe_canonical, eager_lazy_observationally_equivalent). An implementation MUST preserve these consequences:

  1. Same values. A lazy read returns the value an eager read would (observe_canonical).
  2. No churn from allocation. Materializing one node MUST NOT change any other node’s observed value (materialize_preserves_observe).
  3. Deferral, not de-allocation. Lazy materialization only grows the materialized set; a materialized node is never silently dropped, and the lazy set is a subset of the eager set (materialize_present_monotone, lazy_present_subset_eager).
  4. Reactivity is orthogonal. Lazy evaluation — leaving off-viewport derived cells dirty and never recomputing them (the microsecond bounded-viewport read) — is required of both modes and is independent of materialization. Eager materialization still evaluates lazily; lazy materialization additionally defers allocation. An implementation MUST NOT conflate the two.

Execution-context flavors (thread-safe / async)

ComputedMap runs against a context, and the context is a third axis orthogonal to both entry kind and materialization: it fixes where and how the graph executes, not what it computes. The materialization laws hold over each context a binding provides — the ReactiveMap line has one flavor per context, and each carries the context-specific law below:

  • Single-threaded (ComputedMap, over the base Context) — the reference semantics above.
  • Thread-safe (ThreadSafeComputedMap, over a lock-backed context) — a Send + Sync map that can live in a cross-thread owner (e.g. a hub behind a global mutex, where an Rc-based map cannot go). It carries the same materialization laws, plus materialization confluence: the present set and every observed value are independent of the order in which keys are materialized. This is what makes lock-serialized concurrent materialization safe — any order the lock admits yields the same observable map. Proved in lazily-formal’s Materialization module (materialize_present_comm / materialize_observe_comm).
  • Async (AsyncComputedMap, over an async context) — derived (slot) entries resolve asynchronously, so a non-blocking read returns an optional value (None while pending, Some(v) once resolved). Observational transparency weakens to eventual transparency: once a node resolves, its observed value is the canonical value — identical to what the synchronous ComputedMap observes. Input cells are resolved at build. Proved in lazily-formal’s AsyncMaterialization module (eventual_transparency, async_resolved_matches_sync; a pending read is never a stale value, observe_pending_is_none).

A binding SHOULD keep the per-key factory uniform across flavors, so the flavors differ only in execution context — a derived async slot wraps that factory in a ready computation. The factory SHOULD receive the entry’s own tracking view (Fn(&Compute, &K) -> V, not Fn(&K) -> V): without it a derived entry cannot read another reactive and have that read register a dependency edge, which silently reduces ComputedMap to a cache.

A flavor MUST preserve the entry-kind and materialization laws above, MUST expose the Core surface defined under “Core surface vs. binding extensions” below, and adds the context-specific guarantee (confluence for thread-safe, eventual transparency for async). Ordering and atomic move are not exempt: they are Core, and they bind every flavor the binding advertises. A binding that advertises a thread-safe or async map but ships Core only on its single-threaded map is non-conforming on that advertised flavor. A context declared absent is staged out of that peer group; a runner-local lock, dictionary, or synchronous stand-in MUST NOT be used to make the missing flavor appear covered.

When to opt into lazy

Lazy pays off only for sparsely-touched large keyed address spaces — e.g. a 10,000,000-cell spreadsheet where a session reads ~1% of the derived cells: it lowers peak memory and makes “open” cost O(inputs) rather than O(derived cells). It costs a keyed-cache lookup per read instead of a handle dereference, and a cold full scan pays allocation and compute together (eager_materializes_all vs lazy_defers_slots). Handle-based graphs that read most of what they build SHOULD stay eager. The choice is a per-context construction decision, not a per-cell or per-read one.

Keyed cell collections

A keyed cell collection is a composition of cells, not a new cell kind. It maps keys K to per-entry reactive nodes and adds a dedicated membership cell tracking the set of keys. There is one keyed primitive, generic over the entry’s handle kind:

  • ReactiveMap<K, V, H> — a mutable reactive keyed dict: reactive membership + order, get_or_insert_with (mint-on-access), remove, move. H is the entry handle kind. Its two specializations are the concrete types a binding exposes:
    • SourceMap<K, V> = ReactiveMap<K, V, Source>input-cell entries. Adds set(key, value) (an input is settable). Minting is eager-by-value.
    • ComputedMap<K, V> = ReactiveMap<K, V, Computed>derived-slot entries. get_or_insert_with(key, factory) mints a slot on first access (lazy materialization); a slot’s value is derived, so ComputedMap has no set. Eager materialization is a pre-mint loop over the keyset; lazy is mint-on-access — there is no eager/lazy mode flag.

Deprecated spellings. SourceMap and ComputedMap were previously CellMap and SlotMap, with ThreadSafeCellMap / ThreadSafeSlotMap and AsyncCellMap / AsyncSlotMap as their per-context variants. The rename finishes the v2 kernel migration: the node kinds became Source and Computed, and the map names now say which kind they hold instead of naming a vocabulary the kernel no longer uses. A binding SHOULD keep the old names as deprecated aliases of the new ones rather than removing them, so a caller that has not migrated still compiles. Runners MUST accept the old model spellings in a fixture ("CellMap", "SlotMap") alongside the new ones — the same dual-accept the corpus already uses for the signal/eager and dispose_signal/lazy op names. The fixture FILE names keep their historical spelling; a file name is not a type name, and renaming them would invalidate every binding’s replay ledger at once for no semantic gain.

set(key, value) is therefore cell-only (lives on the SourceMap specialization); the shared surface — get_or_insert_with / remove / move / membership / order — lives on the generic ReactiveMap. There are no family types: the “keyed materialized family” is ComputedMap + the mint recipe, and the “auto-mint keyed default” is get_or_insert_with — neither needs a separate type (see § Materialization).

Required. The keyed cell collections layer is normative for every lazily binding — it is not an optional lazily-rs extension. A conforming binding MUST implement ReactiveMap (at least its SourceMap specialization; ComputedMap where the binding supports derived slots), the ordered keyed tree (SourceTree), and keyed reconciliation, and MUST validate against the canonical fixtures in conformance/collections/. The single-writer / multi-write classification, merge: mechanism, and ingress rules below are exactly those defined above — the collection adds no new merge unit.

It conforms to the cell model when:

  1. Each entry is an ordinary cell — its single-writer / multi-write classification, merge: mechanism, and ingress rules are exactly those above. The collection adds no new merge unit; cell = merge unit still holds per entry.
  2. Value, set-membership, and order reactivity are independent: writing one entry’s value MUST NOT invalidate membership or order readers; adding/removing a key MUST NOT invalidate readers of unrelated entry values; and a pure reorder (atomic move) MUST NOT invalidate set-membership readers (len / contains) — only order readers (keys).
  3. A key resolves to a stable handle for the key’s lifetime; membership and order changes are signalled by their dedicated cells, never by mutating sibling entries.
  4. Atomic ordered move (move_to / move_before / move_after): reordering a key MUST keep the entry’s same cell handle, dependents, and lineage (not remove + re-mint) and bump only the order signal once.

Exact-key dependency availability (#lzdependencyavailability)

A non-minting observe(key) correctly reports that an entry is absent, but it cannot register an edge to a node that does not exist. A computed that must react when an exact key is published later therefore uses a dependency reactive:

DependencyReactive<T> = Unavailable | Available(T)

DependencyMap<K, T> is the SourceMap<K, DependencyReactive<T>> specialization whose lifecycle is explicit:

  • observe_dependency(key) materializes one stable input cell for the exact key, seeded with Unavailable, and observes it. This materializes only the dependency handle; it does not fabricate T or run an expensive value factory.
  • publish(key, value) is an ordinary source transition to Available(value). If observation happened first, publication invalidates exactly that key’s consumers. Publishing an unrelated key does not invalidate them.
  • unpublish(key) transitions the same source back to Unavailable; it does not remove or re-mint the handle. A later publish reuses the same identity.
  • Multiple first observers, including concurrent observers on thread-safe and async flavors, converge on one stored handle without orphan nodes.
  • The dependency handle lives until its owning graph scope is torn down. Logical unavailability is a value transition, not disposal.

This API is deliberately separate from observe(key): the existing Core observation remains pure and non-minting. Implementations MUST NOT approximate exact-key availability by observing the whole membership cell, polling/retrying, sleeping, or consulting durable storage. Those mechanisms either invalidate unrelated readers or move live transition authority outside the graph.

Core surface vs. binding extensions

The clauses above are laws. This section says which methods a binding must expose for those laws to be observable, and — just as importantly — which methods are not spec surface at all.

The distinction exists because the family drifted without it. A survey of all nine bindings’ single-threaded maps found method counts from 11 to 32, and the drift splits cleanly in two: methods present in 8 or 9 bindings are laws with a fixture behind them and one binding that never implemented them, while methods present in 7 or fewer are conveniences with no fixture and no law — is_empty is len() == 0, len_untracked is a tracking-discipline escape hatch, reconcile belongs to a higher layer, insert is set under another name. Without this split, “kt is missing seven required methods” and “cs also ships a reconcile helper” are indistinguishable: both read as divergence, and the coverage matrix marks both green.

Core — REQUIRED

A conforming binding MUST expose all of the following, on every flavor it ships (single-threaded, thread-safe, async), spelled in the binding’s own naming convention:

groupmethods
constructionconstruct bound to a context
materializationget_or_insert_with (lazy mint-on-access); materialize_all (eager pre-mint)
kind specializationset (SourceMap only); materialize_all (ComputedMap only)
entry readobserve(key) -> Maybe<V>pure and non-minting; handle(key) -> Maybe<H>
reactive readskeys (order-tracked), len and contains_key (membership-tracked)
materialization planepresent_keys, present_count, is_present — non-reactive
orderposition, move_to, move_before, move_after
membershipremove
introspectionentry_kind

Two Core entries are load-bearing in ways that are easy to miss:

  • observe MUST NOT mint. A read that takes a factory and allocates is a different operation with a different contract; it cannot express “is this key absent” and it cannot be called from a reader without a write side effect.
  • handle is not optional. Clause 3’s stable handle and clause 4’s “same cell handle” are unassertable without a way to observe entry identity. A map whose entries are plain cached values rather than reactive nodes cannot satisfy this, and MUST be recorded as divergent rather than marked green.

The kind specializations bind per entry kind, not per flavor: set is SourceMap-only on all three flavors, materialize_all is ComputedMap-only on all three. Ordering, by contrast, binds every flavor — it touches no entry handle and awaits nothing, so it is neither thread-coloured nor async-coloured.

Extended — OPTIONAL, non-normative

The following ship in some bindings and are explicitly not spec surface. A binding MAY offer them, MAY name them differently, and MAY omit them entirely; their absence is not a conformance gap and MUST NOT be scored in the coverage matrix.

entry / entry_with (eager value-minting convenience), is_empty, len_untracked, get_or_insert_handle, reconcile, insert, Try*-prefixed naming variants, and any binding-local accessors for the membership/order signals themselves.

Ordered keyed tree

An ordered keyed tree (SourceTree) is a further composition: each node is (stable id, value cell, ordered keyed child collection). It conforms when per-node value reactivity holds (editing a node invalidates only that node’s readers), per-level membership/order reactivity holds (a sibling subtree or descendant change MUST NOT invalidate an unrelated level’s child readers), and child reorder inherits the atomic-move guarantee. The tree is still a composition of cells — not a new cell kind — so per-cell merge applies node-by-node.

This is the runtime substrate for stable keyed/wire addressing of collection entries (see the protocol spec’s node-key addressing) and for keyed reconciliation of document trees (minimal {insert, remove, move, update} ops per item → per-cell CRDT merge).

Keyed reconciliation

Reconciling a level diffs two keyed sequences by stable key, not position, emitting the minimal {insert, remove, move, update} op set. It conforms when reordering is move-minimized (keys already in relative order — the longest-increasing-subsequence over their prior indices — MUST NOT move; only the remainder emit move), and when applied to the reactive collection a stable entry (unchanged value, in the LIS) MUST NOT have its value cell invalidated by a sibling reorder. Applying this minimal op set per-cell is the enabling step for per-cell CRDT merge of a document tree — it replaces whole-subtree replacement with proportional-to-the-diff work.

Memoized semantic tree

The syntactic tree holds input cells; a semantic tree (unresolved prompts, drainable heads, summaries) is a layer of memoized computeds derived from it — one memo slot per node folding (node value, child derived values). It conforms when the derivation is incremental and glitch-free: editing one node recomputes only its ancestor chain (a sibling subtree’s derived value stays cached), and a node edit that does not change the folded result MUST NOT re-run a downstream consumer (memo equality guard). Semantics are derived, not materialized eagerly — cost is proportional to the diff, not the document.

Manufactured identity for text

Markdown has no inherent node ids, so reconciliation keys are manufactured from text in three layers: in-band anchors (exact, survive a body rewrite), content-derived hashes of normalized text (survive reflow/reorder, change on edit), and alignment by similarity (word-LCS ratio) to distinguish an edit (key inherited from the matched predecessor → an update) from a genuine insert. A true rewrite legitimately reads as insert+remove. This is why the controlled skeleton uses in-band markers: stable identity is the linchpin that keeps keyed reconciliation from degrading to whole-document replacement over unstable text.

Free-text CRDT + re-parse

For anchorless prose under concurrent edits the merge unit drops to characters: a Fugue/RGA-style character CRDT (each char an element with a unique id + left origin; deletes tombstoned) whose order is a pure function of the element set, so merge is commutative, associative, idempotent and concurrent same-point inserts converge with both preserved. The structural tree is then a projection of the merged text (re-parse → manufactured-identity keys → reconcile), not the merge unit. Honest floor: a true rewrite is a replace — no character identity survives it. The anchored layer keeps per-node lineage; the free-text layer’s guarantee is “merge the text, re-derive the tree.”

Delta sync (#lztextsync)

Whole-replica merge requires transporting the entire element set; a conformant binding also exposes delta synchronization so replicas converge by exchanging only what a partner lacks. Three operations, over the same element set:

  • version_vector(){peer → counter} — the greatest [OpId] counter this replica holds per originating peer, taken over both insert ids and tombstone (delete) ids. It is the compact frontier a replica publishes; an op (c, p) is unknown to a partner iff c > their_vv[p] (absent peer = 0).
  • delta_since(their_vv)[TextOp] — the ops this replica holds that their_vv has not observed: elements whose insert id is newer, plus elements whose tombstone id is newer (a fresh deletion of an already-shared element). Each TextOp is the transport form of one element — { id, ch, origin, deleted }. A whole-state snapshot is delta_since(∅).
  • apply_delta(ops) — applies a delta op list with the same algebra as merge: a new id adds its element (preserving that id); an incoming tombstone is merged sticky-minimally (concurrent deletes keep the smaller delete id); the local Lamport counter advances past every observed id. It is therefore commutative, associative, and idempotent, and re-applying a delta is a no-op.

Identity preservation is the load-bearing property: rebuilding a replica by apply_delta-ing a snapshot onto a fresh buffer keeps every character’s OpId, so a later concurrent edit merges without duplication — unlike re-parsing the text, which would mint fresh ids and double content on the next merge. This is what lets a canonical replica fork per-member replicas from an encoded snapshot and keep them converged by bidirectional delta_since/apply_delta.

The three operations and their convergence/idempotence/identity invariants are pinned by the compute fixture conformance/collections/textcrdt_delta_sync.json (#lztextsync): version-vector shape, bidirectional exchange, whole-snapshot fork identity, and no-op re-apply.

Move-aware sequence order

Sibling order under concurrency is a separate composition above per-cell value merge: a move-aware sequence CRDT (fractional-index positions tiebroken by peer). It conforms when a move is a single LWW reassignment of an element’s position — not delete + reinsert — so two concurrent moves of the same element converge to the later one without duplication, and a concurrent move + value-edit of one element both apply (position and value are independent registers). Removal is an LWW tombstone. This is the order layer beneath keyed reconciliation; it lives only at the multi-writer boundary, leaving the single-producer Snapshot/Delta mirror unchanged.

Forking a replica (#lzzigforkhlcpeer). A fork copies the element set under a new peer identity, and the two halves of its clock are governed separately. The forked replica MUST inherit the source’s causal position — the HLC’s last observed wall time and logical counter — because it has already observed everything the source holds; a fork that restarts its clock at zero mints stamps behind state it already carries the moment its next local op supplies a backwards-skewed now, and since every register adopts only on a strictly greater stamp, that replica’s own write is silently rejected rather than applied. The forked replica MUST NOT inherit the source’s peer id: the peer is the stamp’s final tiebreaker, so two replicas stamping under one id can mint an identical (wall, logical, peer) triple, neither adopts the other, and they diverge permanently. Both halves are pinned by conformance/collections/seqcrdt_convergence.json (fork_carries_the_clock_so_a_backwards_skewed_write_survives, fork_stamps_with_its_own_peer_so_equal_wall_edits_converge). A same-peer copy — clone — inherits both, because it is the same replica continuing rather than a new one.

Tombstone garbage collection

Tombstones (both the sequence-CRDT LWW flag and the character-CRDT sticky delete, which carries the delete’s own id) accumulate without bound — the standard set-CRDT memory-bloat cost. Conformant GC is causal-stability-gated: a tombstone is collectable only once every replica has observed the deletion (the version-vector frontier supplied by the distributed plane, never a single replica’s clock). The sequence layer drops a stable tombstone directly (observationally inert: order/contains already skip it, re-merge re-adopts it as a tombstone, a genuine resurrection wins by LWW). The character layer is conservative: it collects a stable deleted element only when nothing references it as a left origin, so removal never orphans a survivor; interior tombstones are reclaimed bottom-up. Bloat is bounded to the multi-writer plane — the single-producer Snapshot/Delta mirror accrues none.

Scheduling (#lzspecgcdefer). The safety contract fixes which tombstones are collectable; it does not mandate when collection runs. A binding MAY defer GC to idle, batch it under memory pressure, or reclaim incrementally per anti-entropy round, provided no collectable tombstone is re-examined after reclaim and unbounded accrual is surfaced via instrumentation. Deferral changes only memory footprint, never observable values.

Cascade depth is scheduling, not safety (#lzspecgcreferencedtombstone). “Bottom-up” describes the ORDER reclamation is safe in, not how much of it one call must perform. Collecting a leaf can unreference its origin, making that origin collectable in turn; a binding MAY run a single pass and leave the newly-unreferenced tombstone for the next call, or iterate to a fixpoint within one call. The family is split roughly evenly between the two and both conform — the difference is memory footprint at a point in time, which the clause above already places outside the contract. Conformance fixtures therefore assert which tombstones survive a collection, never how many passes it took, and a runner that pinned a cascade count would be encoding one binding’s schedule as a requirement.

What is not optional is the conservative rule itself: a stable tombstone that any surviving element still names as its left origin MUST be kept, or removal orphans a survivor. That half is pinned by conformance/collections/textcrdt_convergence.json (gc_keeps_a_tombstone_that_is_still_a_left_origin) — deleting an INTERIOR character, since a scenario that deletes the last one has no referenced tombstone to keep and so cannot tell a conforming collector from one that ignores origins entirely.

Reactive queues

A reactive queue (QueueCell) is a FIFO collection composed of cells — not a new cell kind — that adds queue semantics (push to tail, pop from head) to the reactive graph. Like the keyed collections above, it adds no new merge unit; each element’s value is an ordinary cell subject to the same single-writer / multi-write classification.

The distinguishing property of a reactive queue is that invalidation is scoped to reader kind, not to individual positions: a push invalidates length/empty/full readers (and the tail signal); a pop invalidates head/length/empty/full readers. The head reader observes the current head value — after a pop, the head reader sees the next element (or empty), not a stale value. There is no random-access queue[N] reader; per-position reactivity is the domain of SourceMap, not QueueCell.

QueueCell — SPSC primitive with MPSC usage rule

QueueCell is specified as a single-producer, single-consumer (SPSC) primitive: one writer owns the tail, one reader owns the head. The producer is the natural FIFO sequencer (push order = delivery order).

MPSC (multi-producer, single-consumer) is a usage rule on the same primitive, not a separate type. Multiple producers push to the same tail inside a batch(); the batch boundary serializes the pushes into a deterministic order. A conforming implementation MUST document the MPSC usage rule and MUST NOT introduce a separate MPSCQueueCell type.

Naming discipline. The cardinality of producers/consumers is not a type parameter. SPSCQueueCell would imply MPSCQueueCell / SPMCQueueCell / MPMCQueueCell siblings — but those shapes differ in semantics (invalidation model, handoff exclusivity), not cardinality. See § Future queue primitives for the genuinely distinct primitives (TopicCell, WorkQueueCell).

The queue family — two axes (semantics defines the primitive)

QueueCell, TopicCell, and WorkQueueCell are one family of reactive cursor-stream cells, separated by two orthogonal axes. Only the first defines the primitive; the second is a usage tier every primitive shares.

Axis 1 — consumer delivery semantics (the primitive axis). Where does each pushed element go?

Delivery semanticsEach element goes toConsumptionPrimitive
singlethe one consumerdestructive popQueueCell
competingexactly one of N consumersdestructive, exclusive handoffWorkQueueCell
broadcastevery subscribernon-destructive cursor readTopicCell

Axis 2 — topology / ordering (a usage tier, not a type). Producer and consumer counts — fan-in and fan-out — never change which primitive you have; they only set the ordering guarantee and its cost. Opt into exactly the tier you need:

Ordering tierGuaranteeMechanismCost
per-producer FIFO (default)each producer’s substream in push order; interleave arbitrarynone — ≡ multiplexing N single-producer channels at the consumerfree
agreed total orderone global order across producersa single leader sequencerone hop, single point of failure
agreed total order + HAtotal order surviving node death / partitionconsensus (Raft/Paxos)quorum latency

Why not name by cardinality or by fan-in/fan-out. Both are the same mistake — naming a primitive by topology instead of by delivery semantics:

  • SP*/MP* mixes two axes: QueueCell already covers SPSC and MPSC (both single-consumer; MPSC is the multi-producer usage). WorkQueueCell and TopicCell are each usable single- or multi-producer, so SPMC/MPMC cannot tell them apart.
  • FanOutCell/FanInCell fails identically. Fan-in (many producers → one consumer) is just QueueCell used multi-producer — the ordering tier, not a distinct primitive; a FanInCell collapses back into QueueCell. Fan-out (one → many consumers) is ambiguous between broadcast and competing — TopicCell and WorkQueueCell both fan out, and one FanOutCell name cannot say whether an element goes to all or to one. And fan-in and fan-out are not mutually exclusive (a topic or work queue may be many-producer and many-consumer at once), so they are orthogonal descriptors of wiring, not primitive identities.

So fan-in and fan-out are real and useful — but as the topology axes describing how a primitive is wired, not as names: fan-in = the multi-producer ordering tier (Axis 2); fan-out = multi-consumer, which subdivides into broadcast (TopicCell) vs competing (WorkQueueCell) by Axis 1. A primitive is always named by its Axis-1 delivery semantics, which carries meaning a topology name cannot.

Naming rationale (suffix). The shared family suffix is Cell; Queue appears only where consumption is destructive and exclusiveQueueCell and WorkQueueCell. A TopicCell is a non-destructive broadcast log (reading removes nothing; every subscriber reads every element), so it carries no QueueTopicQueueCell would conflate the deliberately contrasting messaging terms queue (consumed once) and topic (delivered to all). Parity is the Cell suffix + this family section, not a forced Queue infix.

What multi-producer actually buys (fan-in): a single merged fan-in aggregation stream, one shared backpressure / capacity bound, and — only at the total-order tier — an agreed order. Consensus is the price of agreed total order under partition only, never of multi-producer itself; per-producer FIFO needs no sequencer (it is multiplexed single-producer channels). Full cross-replica retention (every replica holds each item until all consumers ack + compaction) is a replication/HA cost, orthogonal to producer count — a non-replicated queue retains each item once.

Distribution cost differs by primitive (Axis 1 decides it):

PropertyQueueCellWorkQueueCellTopicCell
Each element →the one consumerexactly one of Nevery subscriber
Consumptiondestructive popdestructive, exclusive handoffnon-destructive cursor read
Distribution costleader-election HA (fence one consumer)assignment consensus (quorum)per-subscriber cursors, no consensus¹
Slow consumerfills queue → backpressurerouted to others; fine until all saturatedgrows its retention; evict on lease expiry
Consumer deathqueue stalls until failoverin-flight reassigned; no stallthat subscription grows; others fine
Orderingtotal FIFOassignment-FIFO; processing unorderedper-subscriber FIFO¹
Deliveryexactly-once local / effectively-once distributedat-least-once + idempotency = effectively-onceat-least-once per subscriber
Consensus?only for HAyes (assignment)only for agreed-order broadcast¹

¹ A topic is consensus-free only at the per-producer-FIFO tier; total-order broadcast (all subscribers agreeing on one order) is atomic broadcast ≡ consensus — the same cost as an ordered WorkQueueCell. Quorum-intersection safety for WorkQueueCell assignment is proven ReliableSync.majorities_intersect. Detailed semantics follow in § Future queue primitives.

Reactive shell vs storage backend

A QueueCell factors into two layers:

  ┌─────────────────────────────────────────────────────────┐
  │                   Reactive shell                         │
  │  head version cell │ tail version cell │ closed cell     │
  │  len / is_empty / is_full / head — reactive reads       │
  │  invalidation scoped by reader kind                     │
  └────────────────────────┬────────────────────────────────┘
                           │ QueueStorage trait
                           │  try_push(v) → Result<(), Full|Closed>
                           │  try_pop() → Result<T, Empty|Closed>
                           │  len() → usize
                           │  capacity() → Option<usize>
                           │  is_closed() → bool
   ┌───────────────────────┼───────────────────────────────┐
   │                       │                               │
   ▼                       ▼                               ▼
 VecDequeStorage      RaftQueueStorage              KafkaStorage
 (local default)      (embedded consensus;          (external broker;
                       per distributed-queue PRD)    via adapter)

The reactive shell owns the version cells and invalidation logic; it is storage-agnostic and is what the formal model (QueueCell.lean) pins. The reader-kind plane is pinned separately by QueueReaderKinds.lean: the invalidation set is proven to be a function of the transition alone (the bound, the pre-op length, emptiness, and the closed flag) and never of the elements, which is what licenses running the identical rule on all three flavors. It also proves the atomicity requirement above, and — as the queue-family instance of the thread-safe confluence obligation — that invalidation is order-independent and idempotent, because it consults membership in a set rather than a sequence. QueueFamilyReaderKinds.lean carries the same obligation to the other two primitives: the WorkQueueCell rule invalidates a reader kind iff that kind’s value moved (so it neither over- nor under-invalidates) and reads only the before/after counts; a TopicCell publish changes the observed suffix of exactly the connected subscribers; a cursor read depends on that subscriber’s own record alone, which is why the independence law holds structurally rather than by discipline; and safe GC provably changes no reader’s value, which is what licenses gc taking no context on any flavor. The storage backend owns the actual FIFO data structure and is pluggable via the QueueStorage trait (Rust) / concept (C++) / interface (Py/JS/etc.).

An implementation MUST split the shell from the storage. The shell MUST NOT assume a specific storage type (VecDeque, ring buffer, broker client). A binding MAY ship multiple backends; the default MUST be an unbounded VecDeque-backed storage.

Storage backend contract

Minimal required contract. A QueueStorage backend MUST implement exactly try_push / try_pop / len / is_closed / close. peek and capacity are optional capabilities with a default of “absent” (None): a backend that satisfies only the five required methods — a raw channel, a consuming stream, a Go channel — is fully conforming. It simply has no head reader (no peek) and no is_full reader (unbounded, capacity() → None), exactly as an unbounded backend has always had no is_full. A backend that can cheaply inspect its head MAY expose peek to gain a reactive head; a backend that is bounded MUST expose capacity() → Some(n). head was never in the MUST-reactive set (see “Named observables” below), so removing peek from the required contract removes no required reader.

Footnote — LookaheadShim. A caller that wants a head reader over a non-peekable backend MAY opt into a shell-level lookahead shim that prefetches (early-pops) one element into a one-slot buffer. This is SPSC-local only — early-popping is incorrect for competing-consumer or consensus backends, where an element must not be committed to one consumer before assignment. The shim is not part of the core contract.

A conforming backend MUST also satisfy:

  1. FIFO order: try_pop returns elements in the order they were try_push-ed. A backend that reorders or silently drops elements is non-conforming.
  2. Cardinality compatibility: the backend’s native producer/consumer shape MUST be a superset of the shell’s required shape. (SPSC shell = any backend; MPSC usage requires a backend that accepts multi-writer pushes.)
  3. Bounded contract (optional): a bounded backend exposes capacity() → Some(n) and try_push returns Full when at capacity. The overflow policy (block / drop-oldest / drop-newest / reject) is a backend property — the shell’s observable contract only distinguishes Full from Empty/Closed.
  4. Position identity: invalidation is phrased over reader kind (head/len/empty/full), not over storage indices. A ring-buffer backend whose slot index wraps MUST NOT cause spurious invalidations; the shell layers its own logical reader-kind derivations above the storage.

Closure and lifecycle

Closure is an observable contract, not a mechanism:

  1. try_pop on a closed, non-empty queue returns the next element (drain continues).
  2. try_pop on a closed, empty queue returns Closed — a signal distinct from Empty.
  3. try_push on a closed queue is an error, regardless of capacity.
  4. Close is idempotent (closing an already-closed queue is a no-op) and terminal (once closed, a queue cannot be reopened).

The mechanism (a dedicated closed cell, a flag in storage, a sentinel value) is a binding-level choice. The formal model pins closure as a monotonic flag: Closed_then_stays_Closed.

Bounded queue and reactive backpressure

When the storage backend is bounded (capacity() → Some(n)), the reactive shell exposes is_full as a reactive read. A consumer’s pop that transitions the queue from full to not-full MUST invalidate is_full readers (true → false), enabling push-side effects to react to capacity recovery without polling. This is the backpressure signal: a producer observes is_full and backs off; a consumer’s pop invalidates the producer’s is_full subscription and the producer resumes.

An implementation MUST expose is_full as a reactive cell when the backend is bounded. The unbounded default (capacity() → None) has no is_full reader to invalidate.

Ordering guarantee

ShapeGuarantee
SPSCTotal FIFO — pop order exactly matches push order. The producer is the single sequencer.
MPSCPer-producer FIFO — messages from each producer arrive in that producer’s push order. Inter-producer interleaving is deterministic within a batch() but implementation-defined across batches; under distribution it converges.

A consumer MUST NOT assume total-FIFO across multiple producers. If total order across producers is required, route all pushes through a single producer or use a consensus-backed storage backend (per the distributed-queue PRD).

Wire and snapshot shape

The QueueCell shell’s version cells have no own IPC schema — the head/tail/closed counters are trivial and not independently serialized. A queue reconciles by two complementary wire forms, chosen by plane:

  • Snapshot plane → storage-snapshot form. Full queue state is the storage backend’s snapshot form. The reference VecDequeStorage backend serializes as a JSON array (element order = FIFO order) for conformance fixtures; bindings MAY choose a more efficient binary encoding (bincode, postcard) for production. Cross-backend interop (e.g. VecDeque-backed on one peer, RaftQueueStorage on another) requires explicit storage-format agreement; the shell does not mandate a canonical storage snapshot.
  • Delta plane → op-log form. Incremental change is the ordered shell op-log — QueuePush / QueuePop / QueueClose — carried in a Delta like any other DeltaOp (protocol.md § QueueCell op-log delta form). These are shell ops (storage-agnostic append/remove-head/close), so they need no storage-format agreement; the op-log is the form reliable-sync fuses under backpressure (a queue cannot state-supersede coalesce — order, multiplicity, and the receiver’s pop position forbid it). The op-log delta form is normative here; a distributed storage backend (per the distributed-queue PRD) remains a separate v1 non-goal.

Distribution

Distribution of a QueueCell is a storage-backend property, not a shell property. A QueueCell is distributable iff its storage backend provides a distributed synchronization mechanism. The shell itself is sync-mechanism-agnostic.

v1 does not specify any distributed backend. The Native Distributed Queue PRD covers the future consensus-based RaftQueueStorage backend (Phase 1+) and the positioning relative to external brokers (Kafka, RabbitMQ, Redis Streams, SQS) via the QueueStorage adapter. CRDT-based distribution is explicitly out of scope for queues — destructive pop requires agreement, not merge (see the PRD’s § “Background: Why Consensus, Not CRDT”).

Threading, permissions, and instrumentation

  • Threading contract: QueueCell inherits the Context threading model. MPSC on the same Context uses batch(); cross-thread MPSC requires a ThreadSafeContext-bound flavor; cross-process requires a distributed storage backend.

    This sentence was aspirational prose for the whole v1 line: it named ThreadSafeContext as the way to reach cross-thread MPSC while no binding had a constructor that accepted one, so the only documented path to the guarantee was unimplementable. It is now normative — a thread-safe flavor is Core (see § “Core surface vs. binding extensions (queue family)”) and a binding that ships the family on a single-threaded context only is non-conforming on the other two flavors, not merely unfinished. Recording a capability as absent is fine; documenting a route to it that does not exist is not.

  • Permissions: over the distributed plane, push and pop are distinct capabilities under PeerPermissions (distributed.json). A peer MAY be granted push-only, pop-only, or both.

  • Atomicity: push and pop are individually atomic. Multi-op transactions (e.g., “enqueue N items then close”) MUST use batch() so a concurrent observer never sees a partial state.

  • Instrumentation: a binding SHOULD expose depth / push-count / pop-count metrics via the standard instrumentation surface (parallel to effect_queue_pushes / max_effect_queue_depth).

  • Named observables: is_empty and len are reactive reads dual to is_full. All three (is_empty / len / is_full) MUST be reactive when their respective conditions can change.

  • Demand-driven derivation (#lzspecdemanddriven): a reader-kind MUST be observable-consistent — a get returns the value consistent with all preceding ops — and a binding SHOULD defer its derivation until it has a subscriber (the measured collapse is ~32× per op — 327 ns → ~10 ns — per docs/relaycell-backpressure-analysis.md §5). A reader-kind is a derived value (a Slot), not an eagerly-written cell; an op with no subscriber to a given reader-kind SHOULD only mark it stale (O(1)) and derive it lazily on the next get, provided the derived value is consistent with all preceding ops. This preserves the observable contract (conformance fixtures read the values and MUST stay green) while an unsubscribed QueueCell collapses toward raw-storage cost — the reactive shell is charged only along a path an effect actually observes. A binding that eagerly derives reader-kinds remains conformant (values stay green by construction) but forgoes the write-cost win. See docs/relaycell-backpressure-analysis.md §5 (demand-driven reader-kinds) and §4.0 (the merge cost law).

Queue family extensions

QueueCell covers SPSC and MPSC. TopicCell is the broadcast member of the family; WorkQueueCell remains the competing-consumer extension. They differ in invalidation model and handoff semantics, not merely producer/consumer cardinality:

TopicCell (broadcast)

A broadcast topic: every subscriber receives every pushed element. Invalidation is “all subscribers,” not “head reader.” Each subscriber holds its own cursor; the topic retains an element until all durable cursors pass it (or a TTL expires). Reading is non-destructive — a subscriber’s advance never removes the element for others.

Relationship to QueueCell: not a multi-consumer queue. A QueueCell consumer destructively pops; a TopicCell subscriber reads by cursor and removes nothing. Different invalidation models in kind.

Normative state and operations

A TopicCell<T> state is the tuple (base_offset, elements, subscriptions):

  • base_offset: u64 is the absolute offset of elements[0]; end_offset is base_offset + elements.len.
  • elements: [T] is the retained append log in per-producer FIFO order.
  • subscriptions: Map<SubscriberId, Subscription> is keyed by stable subscriber identity. Each record contains an absolute cursor (the next offset to read), durability (durable or ephemeral), and connected. Every stored cursor MUST remain in base_offset..=end_offset. An ephemeral record MUST be connected: disconnecting it removes the record instead of persisting an offline ephemeral cursor.

The following operations and observables are required:

  • subscribe(id, durable) creates a new cursor at end_offset. Reconnecting an existing durable id MUST reuse its persisted cursor; it MUST NOT silently jump to the tail. Disconnecting an ephemeral subscription removes its record, so a later subscription with that id starts at the then-current tail.
  • publish(value) appends exactly one element and leaves every cursor unchanged. Every connected subscriber whose cursor is now behind end_offset is invalidated independently. An offline durable subscriber is not scheduled, but the element remains readable after it reconnects.
  • read(id) observes the element at that subscriber’s cursor without changing the log or any cursor. Reads from an unknown or disconnected subscription are unavailable. advance(id) moves only a connected subscription’s cursor by one; it is a no-op for unknown/disconnected subscriptions and at end_offset. Therefore base_offset ≤ cursor ≤ end_offset is preserved by every non-TTL operation, and an offline durable cursor is frozen until reconnect.
  • The retention frontier is the minimum cursor across durable subscriptions, or end_offset when there are none. Safe GC removes only offsets below that frontier, advances base_offset, and leaves absolute subscriber cursors unchanged. It MUST preserve every durable subscriber’s future read stream. Ephemeral subscriptions never hold the frontier.
  • GC is observational maintenance and invalidates no subscriber when it stays below the frontier. TTL-forced truncation beyond a durable cursor is a separate gap/resync policy and is not part of the v1 semantic core.

The canonical replay fixtures are topiccell_broadcast_cursor_isolation.json, topiccell_durable_replay_gc.json, and topiccell_ephemeral_lifecycle.json, with connected-session boundary behavior pinned by topiccell_offline_tail_bounds.json. The executable universal reference is lazily-formal/LazilyFormal/TopicCell.lean.

Distribution is cheap — no assignment consensus. Every subscriber gets every element, so there is no “who gets this” decision to arbitrate. A distributed topic is N independent per-subscriber cursor-queues — the per-peer DurableOutbox fan-out already pinned for reliable sync. At-least-once fan-out + idempotent apply per subscriber = effectively-once, no quorum. Caveat: this holds only at the per-producer-FIFO tier; total-order broadcast (all subscribers agree on one order) is atomic broadcast ≡ consensus.

Durable vs ephemeral subscriptions. A durable subscription persists its cursor and replays elements missed while offline; an ephemeral subscription sees only elements published while connected (fire-and-forget). Durable subscriptions drive retention.

Backpressure is per-subscriber, and the answer is the state-vs-event split. Each subscriber’s delivery buffer is itself a bounded QueueCell, so a slow subscriber’s is_full fires locally; what happens then is a per-subscription policy, and the right one depends on message semantics — the same dichotomy as outbox coalescing:

  • State topic (broadcasting a value) — old elements are worthless once superseded, so a lagging subscriber conflates to latest (drop intermediates, keep the newest): the LWW/last-value coalesce applied to a subscriber. Memory-bounded and effect-lossless — the laggard simply gets current state, and the producer feels nothing.
  • Event/log topic (each element a distinct event) — no conflation is possible (order and multiplicity are meaning), so a lagging subscriber resolves overflow one of three ways:
    • Drop (lossy, isolated)drop-oldest / drop-newest for that subscription; fast subscribers and the producer are unaffected. This is the broadcast default — coupling defeats fan-out.
    • Couple + backpressure (lossless, slowest-paces) — the subscriber withholds ack, retention grows, and the producer throttles to the slowest durable subscriber. Opt-in, for “all-must-receive-losslessly” topics; the operator accepts one slow subscriber pacing the whole topic.
    • Evict (bounded) — on sustained lag past a liveness lease, drop the whole subscription (§ Partition & eviction); it full-resyncs (durable) or resumes from now (ephemeral) on return.

So a TopicCell producer feels backpressure only if a subscription opts into coupling; by default a slow subscriber conflates (state) or drops/evicts (event), and failure is isolated — a dead subscriber grows only its retention, never stalling other subscribers or the producer. This is the opposite of QueueCell, where the single consumer is the backpressure path.

Consumer groups = a topic of work queues. The “Kafka consumer group” shape is a composition, not a fourth primitive: WorkQueueCell semantics within a group (competing) and TopicCell semantics across groups (each group gets the full stream). Model it as a TopicCell whose subscribers are WorkQueueCells.

Status: the local semantic core, conformance fixtures, and Lean proofs ship in v0.31.0. The consensus-backed distributed storage integration remains in the distributed-queue PRD Phase 3.

WorkQueueCell (competing consumers)

A work queue: N consumers compete for elements from a shared FIFO; each element is delivered to exactly one consumer (exclusive handoff).

Why pure CRDT cannot do this. A queue pop is not idempotent-commutative: two consumers concurrently popping the same head both survive a CRDT merge → duplicate delivery, and there is no “un-pop.” Exclusive handoff therefore needs a single serialization point for the assignment decision — a designated leader assigning each element to one consumer, or a consensus-committed assignment log. This is the queue’s CP nature made concrete.

Safety via quorum intersection. With consensus-committed assignment, “element X → consumer W” is a majority-committed log entry. Because any two majorities of an n-voter set intersect in ≥1 voter (ReliableSync.majorities_intersect / majorities_overcount), two conflicting assignments of the same element can never both commit — no double-delivery, ever — and a minority (no quorum) cannot commit an assignment, so it blocks rather than risking a duplicate.

Three populations — do not conflate.

  • Workers/consumers — any count; clients of the queue, not voters. Scale freely.
  • Replication peers (per-peer outbox) — any count; transport, not voting.
  • Voting replicas (order + commit assignments) — want an odd count: 2f+1 replicas tolerate f failures; odd maximizes tolerance-per-node and keeps quorum unambiguous. Make an even data set odd with a witness/arbiter (votes, holds no data).

Partition behavior. Only a partition holding a majority makes progress; the rest block (safe, no double-pop). A 2–2 split of 4 voters gives neither side a majority → both stall (the even-group hazard, fixed by a witness). A 2–2–1 split of 5 voters leaves no side with 3 → all block until partitions heal enough for some side to reach a majority. Odd counts protect two-way splits; they never guarantee a majority under multi-way fragmentation. Halting is correct — safety over liveness.

Delivery & lifecycle (#lzworkqueue):

  • Assignment IDs + ack/nack. Each handoff carries a delivery ID; the worker acks (done, remove) or nacks (requeue). Exactly-once commit is the consensus assignment; exactly-once effect needs an idempotent worker or a causal receipt on completion; delivery is at-least-once (redelivery on failure) — so effectively-once = at-least-once + idempotency.
  • Visibility-timeout / lease. An assigned-but-unacked element is leased with a TTL; on expiry it reassigns (worker presumed dead) — the per-item analog of the liveness lease.
  • Dead-letter queue + poison detection. An element exceeding a max-redelivery count (repeatedly crashing its worker) routes to a DLQ instead of redelivering forever.
  • Fairness / dedup. Assignment policy (round-robin, weighted, pull-based) + producer/consumer dedup keys are policy extensions; the portable core uses pull-based FIFO assignment.

Portable local-authority contract. Every binding ships the same in-process WorkQueueCell state machine. The owning instance is the serialization point; hosts MUST serialize calls using the binding’s ordinary context/threading boundary. A local instance does not claim to be a distributed consensus implementation. A cross-process or HA backend MUST put the claim decision behind a leader or consensus-committed assignment log while preserving these operations and outcomes:

  • push(value) -> item_id appends a fresh, monotonically increasing item to the pending FIFO.
  • claim(worker, now) -> delivery? removes the oldest pending item and creates a fresh, monotonically increasing delivery_id. The returned lease contains the stable item_id, value, worker, 1-based attempt, and deadline = now + visibility_timeout. Empty claim is a no-op.
  • ack(worker, delivery_id) -> bool removes only that worker’s matching in-flight delivery. Unknown, stale, or wrong-worker acknowledgements are no-ops. Re-acking is therefore idempotent.
  • nack(worker, delivery_id) -> bool validates ownership, then either appends the item to the pending tail or moves it to the dead-letter list when the configured max_deliveries has been reached.
  • reap_expired(now) -> count expires leases whose deadline < now (the deadline itself remains live), in ascending delivery-ID order. Each expired item is requeued at the pending tail or dead-lettered under the same attempt limit. A redelivery receives a new delivery ID while keeping its item ID and value.

visibility_timeout is a positive logical-clock duration and max_deliveries >= 1. Requeue-at-tail means retries do not block newer work. The portable reactive reader kinds are pending_len, is_empty (pending only), in_flight_len, and dead_letter_len; a mutation invalidates only readers whose values may have changed, and these reader-kinds SHOULD be demand-driven (#lzspecdemanddriven, see § “Demand-driven derivation” above). The canonical fixtures are conformance/collections/workqueue_competing_delivery.json and workqueue_lease_deadletter.json.

Assignment-FIFO ≠ processing-FIFO. Competing consumers trade ordering for parallelism: even if elements are assigned FIFO, N workers process concurrently, so completion order is unordered. A slow worker holding element 1 does not block element 2 (it routes to another worker) — the point of the primitive — but a consumer MUST NOT assume total processing order. For total order, use one consumer (QueueCell) or route through a single worker.

Pull beats push for balancing. A pull model (workers request when ready) is naturally load-balancing and backpressure-friendly — a fast worker pulls more, a saturated worker stops pulling. A push model needs the assigner to track worker capacity. Pull-based competing consumers are the simpler, self-throttling default.

Backpressure & failure — most resilient of the three. A single slow worker does not fill the queue (its items route to others); the queue backpressures only when all workers are saturated and depth grows. A worker death does not stall the queue — its in-flight (unacked) items reassign on lease expiry; throughput degrades, delivery continues. Opposite of QueueCell, whose single consumer dying stalls until failover.

Status: the portable local-authority state machine, reactive reader kinds, Lean safety model, and cross-language conformance are shipped. Distributed/HA assignment still requires the consensus adapter from the distributed-queue PRD Phase 2; the local shell must not be mistaken for that adapter.

Core surface vs. binding extensions (queue family)

The clauses above are laws. This section says which methods a binding must expose on QueueCell / TopicCell / WorkQueueCell for those laws to be observable, and which methods are not spec surface at all. It mirrors § “Core surface vs. binding extensions” for ReactiveMap, and exists for the same reason: without the split, “this binding ships no ordering on two flavors” and “that binding also ships a convenience helper” both read as divergence and both score green.

The split is drawn from a survey of the bindings that ship the family. When it was first written that was eight, lazily-cs excepted; lazily-cs has since landed the single-threaded family and satisfies every Core entry below, so the survey is now 9 of 9 — unanimous, therefore law — and the rest are conveniences.

Core — REQUIRED

A conforming binding MUST expose all of the following on every flavor it ships (single-threaded, thread-safe, async), spelled in the binding’s own naming convention:

primitivegroupmethods
QueueCellconstructionconstruct bound to a context; bounded and unbounded forms
mutationtry_push, try_pop, close
reader kindshead, len, is_empty, is_full, is_closedall reactive
boundcapacity
TopicCellsubscriptionsubscribe, reconnect, disconnect
broadcastpublish
cursor readread (one element at the cursor), read_stream (the cursor’s tail), advance
retentioncursor/GC observables sufficient to assert the retention law
WorkQueueCellmutationpush, claim, ack, nack, reap_expired
reader kindspending_len, in_flight_len, dead_letter_lenall reactive

Three Core entries are load-bearing in ways that are easy to miss:

  • is_empty is Core here, unlike on ReactiveMap. On the map it is Extended, because it is just len() == 0. For a queue it is a named reactive observable that § “Threading, permissions, and instrumentation” already requires — “is_empty and len are reactive reads dual to is_full; all three MUST be reactive when their respective conditions can change”. The reader-kind independence law is stated over that triple, so dropping one makes the law unassertable.
  • A reader kind MUST be reactive, and polling a counter is not reactive. A version counter the caller must poll registers no dependency edge, so no reader can be invalidated by it and the independence law cannot be observed. A binding whose reader kinds are polled counters, or which derives them by diffing a snapshot, MUST be recorded as divergent rather than marked green.
  • try_pop MUST distinguish empty from closed. The closure lifecycle law needs Closed to be observably different from Empty; a pop returning a bare optional collapses them.

Ordering-style reasoning applies to the whole family: none of the Core surface touches an entry handle and none of it awaits, so it is neither thread-coloured nor async-coloured and binds every flavor. Confirmed empirically — across all nine bindings the queue-family sources contain no async fn, suspend fun, Future, Task, Promise, or await anywhere. A flavor is not permitted to colour pop.

Which flavors each binding actually ships, and how that is proved

Auditing this table against the nine bindings found it wrong in both directions at once (#lzcoverageaudit), which is worth stating because the two failure modes hide from different reviewers:

  • Understated. lazily-go and lazily-cpp ship and replay all three flavors of all three primitives, and lazily-py / lazily-kt / lazily-dart ship and replay the thread-safe and async WorkQueueCell and the async TopicCell. All of it read absent. An understated table is the more dangerous of the two: nobody files a bug about coverage they are told they lack, so a “project this family into the bindings” task reads as unstarted work that is in fact done.
  • Overstated. lazily-js was marked as shipping the thread-safe and async QueueCell and the thread-safe TopicCell. It ships none of them — its own flavor ledger records them unshipped and no such type exists in src/.

A mark is only clearable by replay, never by the type existing. Every binding’s suite carries a flavor ledger checked against its own sources in both directions, so a row cannot claim a flavor with no type and a type cannot exist unreplayed; and the replays are fed by a runtime fixture manifest, so “this binary opened these bytes” is observed rather than grepped. lazily-zig is the case that proves the distinction matters: it ships TopicCell and WorkQueueCell and had tests named after the canonical fixtures whose expected values were transcribed into the source, with the fixtures themselves listed as known-uncovered. The types shipped, the corpus did not run, and the honest fix was to add the replay — not to flip the cell.

Reader-kind derivation — one design, pinned

A conforming binding MUST implement reader kinds as memoized derivation with explicit invalidation: each reader kind is a derived node with no graph dependencies, and a successful op clears exactly those whose value provably changed, in one atomic frontier walk.

Three designs were in the field. This one is pinned because:

  • it is what § “Demand-driven derivation” already specifies, and the only one that realises its cost win — a version-cell design charges a graph write on every op even with no subscriber, which is precisely what that clause exists to avoid;
  • it is the shape the whole family converged on for ReactiveMap, where all nine bindings now hold a graph-agnostic ordering/bookkeeping core beside per-flavor version cells minted on each flavor’s own graph. Forking the queue family onto a different reader-kind design would split the family’s own structure for no gain.

A binding MAY keep a version-cell implementation and remain conformant only if reads still register a dependency edge and the invalidation set is exact; it forgoes the demand-driven win. Polled counters and snapshot diffing are not conformant under any reading.

Per-flavor obligation

A flavor MUST preserve the reader-kind independence law, the closure lifecycle, and the single-frontier-walk atomicity requirement — one op invalidates all of its changed reader kinds together, so a subscriber never observes len bumped while is_full has not yet flipped. On top of that it adds exactly its context-specific guarantee: confluence for thread-safe, eventual transparency for async.

Two flavor questions are settled here rather than left to each binding:

  • TopicCell fan-out needs nothing extra under a thread-safe context. Each subscriber owns its cursor and a cursor only advances, so concurrent subscriber reads commute and the retained-until-all-cursors-pass rule is a monotone function of the cursor set. Broadcast is confluent by construction.
  • WorkQueueCell lease expiry stays caller-driven on every flavor. reap_expired takes the current time as a parameter; a flavor MUST NOT substitute an owned timer. Time as an input is what makes lease expiry deterministic and fixture-replayable, and an async flavor that grew its own clock would be unreplayable for no gain.

Extended — OPTIONAL, non-normative

The following ship in some bindings and are explicitly not spec surface. A binding MAY offer them, MAY name them differently, and MAY omit them entirely; their absence is not a conformance gap and MUST NOT be scored in the coverage matrix.

push / pop as panicking or asserting variants of try_push / try_pop, peek, elements (whole-buffer snapshot), base_offset, snapshot / from_snapshot, restart, gc as a caller-visible entry point when retention is already automatic, reader_handle accessors for the reader-kind nodes themselves, and any binding-local accessors for the version cells.

RelayCell — Algebra-Backed Backpressure

RelayCell is the algebra-typed conflating relay: a stream transform that sits on an edge, adapts a fast ingress to a bounded/slow egress by an algebra-typed merge, a reactive backpressure policy, and an optional paged durable spill. This chapter is the normative surface of the phased plan in relaycell-backpressure-analysis.md; the merge algebra (Phase 1) is its foundation and the lazily-formal Lean model (LazilyFormal.Merge, LazilyFormal.Relay) is the executable reference for the invariants below.

Status. Phases 1–8 are complete across all eight bindings. The lazily-rs reference implementation and Lean pins validate the API shapes: Phase 1–4 (merge algebra, RelayCell core, SpillStore, Transport — merge.rs, relay.rs, spill.rs, relay_transport.rs), Phase 5 (Inbox/Outbox role facades — relay_roles.rs), Phase 6 (the extra Rate/Window/Expiry/Priority/keyed policies — relay_policy.rs), and Phase 7 (example systems as integration tests — relay_examples.rs). The Phase 6 policies are formally corollaries of the core theorems (window → flush-grouping, priority/sharding → reorder), and every binding exercises the Phase 2–6 behavior in its relay tests.

Invariant above all else (§9). Policy and transport are local mechanism; the converged egress state is independent of binding and mechanism whenever the merge is associative. Every RelayCell fixture asserts this, and the Lean relay_converges theorem pins it. Break it — a non-associative merge, or a non-lattice merge on a reordering transport — and portability, determinism, and the formal pin all fail together.

1. Not a new node — a composite (Phase 2)

RelayCell<T, M> is not a new Axis-1 delivery primitive and not a new reactive node category. It decomposes into the reactive family:

  • the hot head is a SourceCell<T, M> (write = under policy M);
  • its reactive reads (depth / bytes / pending_keys / is_full / is_spilling / is_draining / lag) are FormulaCells — demand-driven, so an unobserved relay costs N·⊕ and nothing more (the merge cost law);
  • an ingress Effect drives the merge from the transport.

It composes onto QueueCell / TopicCell / WorkQueueCell edges and subsumes today’s scattered backpressure logic: DurableOutbox.coalesce_to_snapshot is a RelayCell<_, CrdtJoin<Lww>> + a SpillStore; per-subscriber TopicCell conflation is a per-sub RelayCell; op-log outbox fusion is a RelayCell<_, RawFifo> + Spill (no conflate).

      ingress          ┌──────────── RelayCell ─────────────┐         egress
 ───(Transport)───────►│ hot head : accumulating merge (⊕)  │───────►(Transport)──►
    ops arrive fast    │ BackpressurePolicy (watermarks)     │  drained on egress
                       │ overflow: Block|Drop|Conflate|Spill │  readiness (credit)
                       │ SpillStore (paged durable tail)     │
                       │ reactive reads: depth/bytes/keys     │
                       └─────────────────────────────────────┘

2. BackpressurePolicy — reactive limits (Phase 2)

Every limit is a reactive cell, so an operator or an adaptive controller retunes it live and every dependent relay reacts:

FieldTypeMeaning
dimensionCell<BoundDim>Count | Bytes | Keys | Age — what the bound measures
high_waterCell<u64>gate ingress at/above this level
low_waterCell<u64>re-open ingress at/below this level
overflowCell<Overflow>Block | DropOldest | DropNewest | Conflate | Spill

Hysteresis is required: high_water ≠ low_water so a relay riding the bound does not flap open/closed. An adaptive controller is an ordinary formula cell that observes depth/lag/downstream latency and drives high_water — the control loop is reactive policy driving reactive limits, not a config struct.

3. Overflow actions and policy-flag validation (Phase 2)

The merge algebra, not the relay, decides which overflow is sound. A RelayCell MUST reject an (overflow, transport) pair the policy’s flags forbid, at construction:

OverflowLossRequiresRejected when
Blocklossless— (propagates backpressure via is_full)never
Conflatelossless for converged state associative (always)RawFifo (order/multiplicity are meaning)
Spilllossless idempotent or dedup keys (crash-replay is at-least-once)non-idempotent without spill dedup
DropOldest/DropNewestlossycaller opts into lossa lossless policy (e.g. Sum, GCounter) declares Drop forbidden

A relay MUST additionally reject reordered pages / shards for a non-commutative policy (the reordering tax), and MUST log its overflow action and expose dropped / conflated counters — no silent truncation.

Merge swap (can_reconfigure): M MAY be swapped only when the hot head is empty (changing mid-stream changes meaning); the relay exposes can_reconfigure (true iff head empty), and re-validates overflow/transport against the new policy’s flags at swap time.

4. SpillStore — paged durable tail (Phase 3)

SpillStore generalizes DurableOutbox: a hot page in RAM (actively merged) plus immutable cold pages on durable store, a bounded manifest (page_id → location, watermark, bytes), an egress cursor, and ack-before-reclaim. Memory is O(hot page) + O(manifest) for any algebra. Retrieval is pull-metered by egress credit; associativity lets a lagging consumer receive a conflated catch-up. Reconstruction (cold pages in order, then hot head) reproduces the flat fold — paged spill loses nothing (Lean spill_lossless). Crash recovery replays the last unacked page; because a page is one coalesced summary op at the egress, replay is a no-op for an idempotent policy (Lean spill_replay_idempotent) — at-least-once delivery converges.

SpillPolicy (all reactive): page_size; mode = CompactOnWrite (keep-latest, minimizes disk) | AppendCompact (LSM-style, preserves increments for accumulating semilattices); retention = BoundedDisk | UnboundedLog | Ttl; rehydrate = Sequential | Parallel. Parallel rehydrate requires a commutative merge (pages merge out of order).

5. Transport seam (Phase 4)

Transport abstracts ingress/egress delivery so the mechanism is pluggable and per-binding: InProc (direct) | CrossThread (native mpsc or shared ThreadSafeContext) | IpcTransport | WsTransport. RelayCell is written once against Transport; a binding drives it on a goroutine, an async task, or an evented IO interface. The merge algebra — not the transport — guarantees converged state (Lean transport_independent), so transports may differ across bindings and still converge to the same egress.

A channel is a Transport, not a QueueStorage. With peek optional (QueueCell) a native channel satisfies the minimal storage contract directly, but its real role is cross-thread delivery: the idiomatic form is CrossThread, where a worker owns the channel (its mailbox), drains it, and merges into the hot-head SourceCell (the push-fed regime — the pop message carries the value). Crossing a thread boundary requires the thread-safe context.

6. Inbox / Outbox — directional roles (Phase 5)

RelayCell is direction-neutral. Inbox and Outbox are role facades (typed constructors with direction-appropriate defaults), not reimplementations — mirroring “MPSC is a usage of QueueCell, not a subtype”. They earn names because they differ in the backpressure-propagation contractwho you can backpressure and how:

RoleEdgeBackpressure targetDefault overflow
Outboxapp → transport (send)the local producer (directly blockable via is_full)Conflate(state) / Spill(event)
Inboxtransport → app (receive)the remote peer — only via transport flow control (withhold credits/acks, TCP window)Conflate(inbound) / Drop / Credit-meter

A network link is Outbox → Transport → Inbox, and end-to-end backpressure is a chain of relays: the local producer’s is_full ← Outbox fullness ← (credits / TCP window) ← remote Inbox fullness ← remote app’s consumption. Both ends share one RelayCell core so the signal propagates through the link as one continuous reactive edge. Fan-out reuse: per-subscriber RelayCells wire into TopicCell (the state-vs-event choice becomes a MergePolicy choice); WorkQueueCell stays as-is (competing consumers, no conflate).

7. Extra reactive policies (Phase 6)

Optional reactive stages composed onto the relay egress, each covering a row of the backpressure case matrix:

PolicyCaseBehaviorSoundness
WindowPolicy8 (debounce/throttle)coalesce on a time window, not just fullnessassociativity (a window is a flush group — flushGroupingIrrelevant)
RatePolicy9 (token bucket)pace egress; ingress backpressurespacing re-chunks flushes; converged state unchanged
ExpiryPolicy10 (TTL)drop elements older than a deadlinelossy-by-age (explicit)
PriorityStorage11egress by priority, not arrivalreordering — requires Commutative (Lean reorder_adjacent)
keyed sharding18N relays by key; merge across shardscommutative merge across shards

WindowPolicy and RatePolicy only change where the relay flushes, so the converged state is invariant (relay_converges). PriorityStorage and keyed sharding reorder ops, so they are sound exactly when the policy is commutative.

8. Conformance

A RelayCell implementation conforms when:

  1. Composite, not a new node. The relay is built from a SourceCell hot head, demand-driven FormulaCell reads, and an ingress Effect — no new node category.
  2. Converged-egress invariance (§9). For an associative M, the egress state after any flush schedule equals the lossless flat fold of the delivered ops. Pinned per-binding by replaying mergecell_algebra.json through a relay and by the Lean relay_converges / transport_independent theorems.
  3. Overflow validation. An (overflow, transport) pair the policy’s flags forbid is rejected at construction; Conflate on RawFifo, Drop on a lossless policy, and reordered pages/shards on a non-commutative policy all fail fast. Lossy actions log and increment dropped/conflated counters.
  4. Spill losslessness + idempotent replay. Reconstruction from cold pages + hot head reproduces the flat fold; crash-replay of the last unacked page converges (idempotent policy or spill dedup). Lean spill_lossless / spill_replay_idempotent.
  5. Reordering tax. A commutative policy is invariant under reordering (priority egress, keyed sharding, out-of-order pages); a non-commutative one preserves arrival order. Lean reorder_adjacent.
  6. Reactive policy. BackpressurePolicy/SpillPolicy limits are reactive cells with hysteresis; retuning a limit live re-drives dependent relays.

Verification form. The converged-state and algebra invariants are pinned by the Lean model (LazilyFormal.Relay) and the mergecell_algebra.json compute fixture; per-binding behavior (overflow validation, spill, transport) is exercised by each binding’s relay tests. Concurrency/transport interleaving is not a deterministic replay, so — like the thread-safe context — it is verified by each binding’s model checker / stress harness, not a portable fixture.

State Machine

A state machine is the flat finite-state-machine primitive — a reactive cell holding the current state plus a pure transition function. It is the kernel a single-region State Chart compiles down to, and the Lean formal model in lazily-formal (LazilyFormal/StateMachine.lean) fixes its semantics normatively.

A state machine is compute, not protocol. Like a chart, it is never serialized as a distinct wire kind — only its current state crosses IPC/FFI as an ordinary cell Payload (a CellSet op on the active-state cell). Each binding implements it natively; this chapter fixes the behavior so a machine defined in one language means the same thing in another.

The kernel

The pure transition core is:

Transition : State -> Event -> Option State
Machine    = { current: State, transition: Transition }
send(m, e) = match m.transition(m.current, e) of
             | Some next => { current: next, ... }   // accepted
             | None      => m                         // rejected (guard)

A binding wraps the current State in a reactive cell and exposes send as the mutator. The transition function is pure: given the current state and an event it returns the next state (Some) or rejects the event (None, a guard).

API surface

MethodDescription
new(ctx, initial, transition_fn)Create with an initial state and a pure transition function
send(ctx, event) -> boolEvaluate the transition; true if accepted, false if rejected (None)
state(ctx) -> StateRead the current state
state_handle() -> SourceCell<State>The underlying active-state cell, for reactive dependencies
on_transition(ctx, old_new_callback) -> EffectHandleObserver firing on each state change with (old, new)
state_is(ctx, target) -> FormulaCell<bool>Eager (driven) formula: true while in target

Semantics

  • PartialEq guard (no-op suppression): a transition to an equal state is accepted (send returns true) but does not invalidate dependents — the active-state cell’s equality guard suppresses the no-op update. This is the flat analogue of the chart self-transition rule. To force re-entry, clear the cell’s dependents before send.
  • Reactive integration: any formula, driven formula, or effect that reads state_handle() automatically recomputes or reruns on a real transition.
  • On-enter / on-exit: model with an effect that has a cleanup closure — the body is on-enter, the returned cleanup is on-exit (runs before the next rerun). on_transition provides a single (old, new) observer instead.
  • Batch atomicity: a batch coalesces multiple send calls — effects fire once after the batch settles.
  • Deterministic transition function: the transition function MUST be pure and deterministic. send never changes the transition function — a machine is fully described by its transition function and current state.

Proven invariants

The Lean kernel (lazily-formal/LazilyFormal/StateMachine.lean) proves the properties that are easiest to blur across bindings:

  • Guard rejection preserves state — a rejected event (None) leaves current unchanged.
  • Accepted transitions advance state — an accepted Some(next) sets current = next.
  • Self-transitions are no-opsSome(current) leaves current unchanged and sends returns false.
  • Changed transitions send trueSome(next) with next != current reports a real change.
  • send preserves the transition function.

Relationship to state charts

A single-region chart refines this send: the chart’s active leaf is the State, the chart event is the Event, and the chart’s walk-up + LCA + descend-initial logic is a pure function returning Some(new_leaf) or None. The confluence/determinism of a single-region chart is inherited from this kernel (see single_region_refines_flat_machine in lazily-formal/LazilyFormal/StateChart.lean). A flat StateMachine is the degenerate chart with no nesting.

Context layers

The flat kernel is context-agnostic. A binding offers it over each reactive context layer it implements, with identical semantics:

  • Single-threaded — backed by the single-threaded reactive context.
  • Thread-safe — a lock-backed counterpart over the thread-safe context; the transition function and state are Send + Sync, the machine is a clonable handle to the same state cell, and observers fire synchronously within the invalidating send/batch call preserving glitch-free pull-based ordering.
  • Async — backed by the async context; send and state stay synchronous (cells are the synchronous input layer), while reactive observers use the async effect/signal APIs and settle on the runtime rather than within send.

Implementation status

The flat machine is required of all bindings that ship a reactive graph. The thread-safe and async counterparts are required of any binding whose platform supports them (see Wire Protocol § Concurrency layers are required); a binding MAY omit one only when it has declared the matching thread_safe / async capability as none. In every case the single-threaded machine and its PartialEq no-op suppression MUST be present wherever a chart is supported (a chart depends on this kernel). lazily-rs, lazily-zig, and lazily-py implement it; the Lean model is the executable reference.

State Charts

A state chart is a Harel/SCXML hierarchical state machine. This chapter specifies the full cross-language subset all lazily bindings implement: compound (nested) states, orthogonal (parallel / AND) regions, history states (shallow and deep), entry/exit/run actions, internal/external transitions, guards, and extended state (context). It also fixes the declarative chart form the conformance fixtures use and how a chart relates to the reactive Cell Model and the flat FSM kernel in the Lean Formal Model.

A state chart is compute, not protocol. It is never serialized over the wire as a distinct kind — only its converged active configuration crosses IPC/FFI as an ordinary cell Payload. Each binding implements the chart natively; this chapter fixes the behavior so a chart defined in one language means the same thing in another (validated by the conformance fixtures).

The declarative chart form is normatively defined by schemas/statechart.json (JSON Schema, Draft 2020-12). The prose below fixes its semantics.

Why a chart reduces to existing machinery

The Lean kernel (formal/lean/LazilyFormal/StateMachine.lean) defines a pure transition State → Event → Option State. A single-region chart compiles to exactly this kernel: the chart’s active leaf is the State, the event is the Event, and the chart’s transition logic (walk-up + LCA + descend-initial) is a pure function returning Some(new_leaf) or None.

Orthogonal regions generalize State from a single leaf to an active configuration (a set of leaves, one per region). The transition is still a pure Configuration → Event → Option Configuration: enabled transitions, conflict resolution, and exit/enter-set computation are all deterministic functions of the chart definition and the current configuration. Because of this, a chart cell is single-writer by the Cell Model rules — it is either a direct cell (reactive bindings) or a derived cell keyed off event cells. It is therefore never multi-write: replicas converge on the event stream, and each replica’s chart recomputes deterministically. No merge: mechanism is defined or needed for charts.

Reactive vs. projection bindings

BindingChart backingWhy
lazily-rs, lazily-py, lazily-zig, lazily-kt, lazily-js, lazily-dartreactive Cell (active configuration)these run their own reactive graph; the chart composes with slots/signals/effects

A chart is never exposed over FFI. Every binding implements the transition as pure logic with zero system dependencies; routing it through JNA/koffi to a Rust Context would be circular (the only thing FFI buys a consumer is an authoritative projection, which a local chart is not).

Declarative chart form

Conformance fixtures and cross-language chart definitions use a flat id map. States form a tree via parent. The full form:

{
  "initial": "<state-id>",
  "context"?: { … },
  "states": {
    "<id>": {
      "parent"?: "<id>",
      "kind"?: "atomic" | "compound" | "parallel" | "history" | "final",
      "initial"?: "<child-id>",          // compound only
      "parallel"?: true,                  // AND-state: children are concurrent regions
      "history"?: "shallow" | "deep",     // history pseudo-state
      "default"?: "<target-id>",          // history: resume target before any recording
      "on"?: { "<event>": <transition>, … },
      "entry"?: [<action>, …],
      "exit"?:  [<action>, …],
      "run"?:    [<action>, …]
    }
  }
}

kind is optional and inferred: history when history is set; parallel when parallel is true; compound when the state has children; otherwise atomic. When kind is present it is authoritative, not an annotation: it MUST be one of the five values above and MUST agree with the structural fields and child relation. final is the only kind that cannot be inferred; it MUST have no children, initial, parallel, or history. A contradictory or unknown declared kind makes the entire chart malformed.

Every id-valued reference MUST name a state declared in the same states map before the chart can execute. This includes parent, compound initial, history default, transition target, and the chart-level initial. Implementations MUST reject an unresolved reference while building the chart; they MUST NOT infer an undeclared id as an atomic leaf or allow it into the active configuration.

A <transition> is a bare target-id string (shorthand for {"target": id}) or an object:

{ "target": "<id>", "guard"?: <guard>, "action"?: [<action>], "internal"?: false }

Structural rules

  • Exactly one state has no parent — the root.
  • A compound state (one with children) MUST declare initial, which MUST resolve to a leaf by descending compound initials.
  • A parallel state MUST NOT declare initial; its children are the concurrent regions, and all of them are active whenever the parallel state is.
  • A history state MUST NOT declare initial or parallel; it MUST declare history and SHOULD declare default. Its parent is the region whose configuration it records.
  • An atomic state has no children, no initial, and no parallel.
  • A final state signals completion of its parent region (see Completion).

Active configuration

The active configuration is the set of states that are currently active:

  • A state is active if any of its children is active (compound), or all of its region children are active (parallel), or it is an active leaf.
  • For a single-region chart, the configuration is the path root → active leaf and contains exactly one leaf.
  • For a chart with parallel regions, the configuration contains one leaf per region plus all their ancestors (including the parallel state itself).

The set of active leaves is the subset of atomic active states.

Transition selection

send(event) runs run-to-completion: it computes the next configuration from the current one with no interleaving of external events.

1. Enabled transitions

For each active leaf, walk up its ancestor chain. At each ancestor, collect every transition whose on[event] matches event and whose guard passes. A guard passes when it is absent, resolves true (see Guards), or — for a context-expression guard — evaluates true against context. A single event may enable transitions in more than one region (parallel charts fire concurrently).

2. Conflict resolution

Two enabled transitions conflict if their exit sets (computed next) intersect — i.e. one would exit a state the other needs. Resolve conflicts by:

  1. If one transition’s source is a descendant of the other’s source, keep the descendant (innermost wins).
  2. Otherwise keep the one that appears first in document order (the order states and their on entries are declared).

The surviving set is taken atomically.

3. Exit and enter sets

For each taken transition with source s and target t:

  • lca = the lowest state that is an ancestor of both the active leaf in s’s region and t. (For a transition internal to one parallel region, the lca stays inside that region and the sibling regions are untouched.)
  • Exit set = active states strictly below lca, restricted to s’s region subtree.
  • Enter set = the path lca → t, then:
    • if t is compound, descend via initial to a leaf;
    • if t is parallel, enter every region child and descend each;
    • if entering a region whose recorded history exists, descend via the recorded configuration instead of initial (see History);
    • if t is a history state, resume its parent region per its recorded configuration (or default if none).

The total exit set is the union over taken transitions; the total enter set is the union, plus any parallel siblings forced active by entering a parallel state. Exit actions run innermost-first (reverse document order within the exit set); then transition actions run; then entry actions run outermost-first (document order within the enter set).

Internal vs external. A transition with internal: true whose target is the source or a descendant of it does not exit/re-enter the source — only its action runs and descendants below the target are reconfigured. The default is external (exit/re-enter the source).

4. Apply

The new configuration is the old one minus the total exit set plus the total enter set. If no transition was enabled, the event is rejected: return false, state unchanged.

Single-region specialization

With no parallel regions the algorithm collapses to the SCXML single-region rule: walk up from the one active leaf, take the first passing transition, and compute exit/enter through the lca. lca makes sibling-substate transitions (e.g. a.x → a.y) cheap (exit only x, enter y) and cross-subtree transitions (a.x → b.y) correct (exit a.x…, enter b.y…).

Queries

  • active() → the active leaf id. Defined for single-region charts; for charts with parallel regions it is undefined (use configuration()).
  • configuration() → the full set of active state ids (leaves plus all active ancestors).
  • activeLeaves() → the set of active atomic state ids (one per region).
  • matches(id)true iff id is in configuration() (the hierarchical “state-in” predicate).

History

A history pseudo-state records its parent region’s active configuration whenever that region is exited, and resumes it when the region is re-entered by a transition targeting the history state.

  • Shallow ("history": "shallow"): records/restores the direct child of the parent region that was active. Nested configuration below that child is re-derived from the child’s own initial.
  • Deep ("history": "deep"): records/restores the full nested leaf configuration of the parent region, across all descendant compound and parallel levels.
  • First entry: when a region with a history child has never been exited (no recording), a transition targeting the history state enters default instead. default SHOULD be declared; if absent, the region’s initial is used.

A region records history on every exit, including exits caused by a transition that leaves the region without targeting its history, so a later re-entry via history resumes the most recent configuration.

Actions

Actions are host-resolved side effects, never part of the configuration. A binding accepts an action handler name → effect; for conformance replay, each step asserts the ordered trace of action names that fired.

  • entry — fired when the state enters, after its ancestors’ entries.
  • exit — fired when the state exits, before its ancestors’ exits.
  • run — ongoing (do) actions: started on entry, cancelled on exit. Host-managed; not part of conformance replay.
  • transition action — fired after the exit set and before the enter set.

Order, restated: exit actions (innermost-first) → transition action → entry actions (outermost-first), per the exit/enter set computation.

Guards

A transition may name a guard. A guard is either:

  • a bare string — a named guard resolved by the caller’s guard resolver (name → bool); when no resolver is supplied or the name is unknown it is treated as false (fail-closed); or
  • an object {"expr": "…"} — an extended-state expression the host evaluates against context.

Guards are pure predicates over caller-supplied state, never over the chart’s own configuration. For conformance replay, each step supplies its guard outcomes explicitly by name, so every binding reproduces identical behavior without shipping a guard evaluator.

Extended state

context is optional host-resolved caller state over which context-expression guards evaluate. It is never serialized as part of the active configuration; it lives outside the chart. A binding that supports context exposes it to guard expressions and transition/entry actions; a binding that does not MUST reject charts that use {"expr": …} guards explicitly rather than silently treating them as passing.

Completion

A final state marks its parent region complete. When every region of a parallel state reaches a final child, the parallel state itself is complete and a completion (done) event is raised for the parent. This is the SCXML automatic transition on completion. (Bindings MAY defer final/completion to a later revision, but MUST reject final explicitly if unsupported.)

Reactive binding

In reactive bindings the active configuration lives in a Cell. On send(event) the pure transition produces a new configuration; the cell’s != (PartialEq) guard suppresses downstream invalidation when the configuration is unchanged (e.g. a no-op self-transition or a parallel-region transition that doesn’t change the leaf set). Any Slot / Signal / subscriber reading active(), configuration(), or matches() is invalidated on a real transition.

Context-layer variants (non-normative)

A binding MAY expose the chart over each concurrency layer its context stack supports, exactly as it does for the flat StateMachine — one chart type per context layer, sharing a single context-free transition engine so the Harel semantics (and hence conformance) are identical across layers. lazily-rs ships StateChart (single-threaded Context), ThreadSafeStateChart (ThreadSafeContext), and AsyncStateChart (AsyncContext); lazily-kt ships StateChart + ThreadSafeStateChart (it has no async state machine); projection-only bindings without a reactive graph (lazily-js) expose a single plain-Set chart with no thread-safe/async variant. Which layers exist is a binding property; the transition function they compute is the same one this spec fixes.

Typed builder (non-normative)

fromChart (the declarative JSON form above) is the normative, conformance- tested definition path. A binding MAY additionally offer a typed native builder that constructs the same ChartDef — e.g. lazily-rs ChartBuilder, lazily-kt ChartBuilder, lazily-js ChartBuilder. A chart assembled by the builder MUST be behaviourally identical to the same chart parsed from JSON (both derive the children / root / depth structure through one shared assembly step); the builder is a source-ergonomic convenience, not a distinct semantics.

Self-transitions

A transition whose resulting configuration equals the current one is a no-op: accepted (true) but the cell’s PartialEq guard suppresses downstream invalidation, identical to the flat StateMachine self-transition rule. (For an external self-transition that must re-run entry/exit, the configuration object is still replaced; whether that invalidates dependents is the binding’s documented choice — conformance asserts the action trace, not invalidation counts.)

Conformance fixtures

Canonical charts live in conformance/statechart/. Each binding loads them, asserts initial_active (and initial_actions when present), replays the steps, and asserts accepted, active, matches, and (when present) actions after each step. For parallel charts active is an array of active leaves (sorted); for single-region charts it is a single leaf id.

Fixture schema

{
  "description": "…",
  "kind": "StateChart",
  "initial_active": "leaf" | ["leaf", …],
  "initial_actions"?: ["action", …],
  "chart": { …per schemas/statechart.json… },
  "steps": [
    {
      "event": "START",
      "guards"?: { "name": true },
      "accepted": true,
      "active": "leaf" | ["leaf", …],
      "matches"?: { "state-id": true },
      "actions"?: ["action", …]
    }
  ]
}
  • initial_active — expected active leaf (or leaves for parallel) after descending chart.initial, asserted once before any step.
  • initial_actions (optional) — ordered action names fired during initial entry.
  • event — the event sent.
  • guards (optional) — per-step named guard outcomes for this send.
  • accepted — expected send return value.
  • active — expected active leaf (single) or active-leaf set (parallel) after the step.
  • matches (optional) — { state-id: bool } expectations for matches().
  • actions (optional) — ordered action names fired during the step (exit → transition → entry).

Current fixtures:

FixtureCovers
flat_cycle.jsonflat (single-level) transitions, rejection, cycle
hierarchical_player.jsonnesting, walk-up transition resolution, LCA across levels, matches()
guarded_door.jsonnamed guards, fail-closed rejection, guard pass
parallel_regions.jsonorthogonal (AND) regions: per-region transitions, matches() across regions, exiting all regions
history_shallow.jsonshallow history: resume last direct child on re-entry; first-entry default
history_deep.jsondeep history: resume full nested leaf configuration; sticky across cycles
entry_exit_actions.jsonentry/exit/transition action ordering across LCA boundaries

Implementation status

The single-region subset (compound states, walk-up resolution, LCA, guards, self-transitions) is required of all bindings. Orthogonal regions, history, actions, and extended state are specified here in full; a binding MAY implement a subset, but MUST reject any feature it does not implement explicitly (never silently ignore a parallel, history, entry/exit/run, or {"expr": …} guard). Each binding’s conformance run selects the fixtures that match its implemented subset.

Async Reactive Context

An async context is a separate reactive surface for computations whose values are produced by async/future-returning functions. It is not an overload of the synchronous or thread-safe context; it is a distinct graph with its own handles, because futures introduce in-flight state, cancellation, stale completion, and dependency tracking across suspension points that the synchronous graph does not have.

This chapter fixes the cross-language contract. An async context is compute, not protocol — only resolved slot values cross IPC/FFI as ordinary cell payloads, exactly like the synchronous graph.

Why a separate surface

A synchronous reactive context tracks dependencies through a thread-local stack touched on every read, and a slot’s value is either present or unset. An async computation can be in-flight (suspended at an .await) when its inputs change, can complete after those inputs are gone, and can be canceled mid-flight. Those states require:

  • an explicit per-slot state machine (not just present/absent),
  • revision tracking so a stale completion is discarded,
  • dependency edges registered before the read is awaited, and
  • a cancellation contract that is safe under waiter drop, supersession, and context disposal.

A binding gates this surface behind a separate feature flag so downstream users do not accidentally accept the larger semantic surface.

Handles

An async context exposes its own copyable, id-only handles, distinct from the synchronous handles:

HandleWraps
AsyncSource<T>A mutable input cell (the synchronous input layer)
AsyncComputed<T>A computed/memoized async slot
AsyncEffectHandleAn async effect

Handles are id-only and copyable; they are usable only with the owning async context.

AsyncSource and AsyncComputed are the canonical public value kinds in every binding. Published pre-v2 spellings such as AsyncCellHandle and AsyncSlotHandle may remain only as deprecated aliases of those exact types. Constructing through an old spelling must still return the canonical type.

API surface

MethodDescription
source(value)Create a mutable AsyncSource (value type equality-comparable, cloneable, Send + Sync)
get(source)Read an AsyncSource value synchronously
set(source, value)Update an AsyncSource and invalidate dependents
computed / computed_async(compute)Create an AsyncComputed; the exact spelling follows the binding’s async conventions
computed_with_equals(compute, equals)Create a guarded AsyncComputed with an explicit equality policy
get(computed) -> Option<T>Read an AsyncComputed cache synchronously; Some(T) if resolved, None otherwise (warm-path fast path)
get_async(computed) -> TAwait an AsyncComputed; uses get() for resolved values, otherwise spawns async compute
memo / memo_async(compute, equals)Deprecated compatibility constructor forwarding to guarded computed; memo is not a third node kind
effect_async(effect)Create an async effect with an async cleanup
dispose_async_effect(handle)Dispose an async effect and await its cleanup
batch(run)Synchronous batch boundary; schedules async reruns at batch exit

Sources are the synchronous input layer: source, get, and set are synchronous. Only computed evaluation and effects are async. The constructor and read/write vocabulary is deliberately the same as the local and thread-safe contexts; only the async handle types and get_async are execution-model-specific.

The public state projection is AsyncComputedState. The formal theorem/module name remains AsyncSlotState, because it describes the storage state machine, not a public value-handle kind. Bindings with a published AsyncSlotState API keep it as a deprecated alias of AsyncComputedState; formal references are not renamed mechanically.

Async computed state machine (formal AsyncSlotState)

Each async slot tracks its state through a finite state machine:

            first get_async /         future Ok,          dependency
   Empty ──────────────────► Computing ──────► Resolved ──────► Computing
     ▲         (spawn)        │    │                              │
     │                         │    │ future Err                   │
     │                         │    ▼                              │
     │                         │  Error ──────────────────────────┘
     │                         │   retry get_async
     │                         ▼
     │      dependency invalidation      revision mismatch on
     └──── during in-flight compute ──► (stale) Computing ──► complete discards;
      hard clear                                                new future spawned
StateMeaning
EmptyNo cached value, no in-flight computation. Entered on creation and after a hard clear.
ComputingA handle tracks the in-flight future for the current revision. Concurrent get_async callers attach as waiters to the same in-flight result instead of spawning duplicate futures.
ResolvedThe cached value is fresh, until dependency invalidation transitions back to Computing.
ErrorThe last computation failed. Every caller waiting on that attempt receives its error. The error is not a cached value: the next get_async MUST re-spawn (Error → Computing) rather than replay the stored error.

Revision tracking is load-bearing: a computation records the slot revision at start; at publish time the graph accepts the value only if the revision is still current. This is what makes stale completion safe.

Transitions:

  • Empty → Computing — first get_async or invalidation with no cached value.
  • Computing → Resolved — future completes Ok and the recorded revision still matches.
  • Computing → Error — future completes Err and the recorded revision still matches.
  • Computing → Computing (stale) — invalidation advances the slot revision during an in-flight computation. The completing future finds its revision no longer matches and discards the result; a new future is spawned for the updated revision.
  • Resolved → Computing — invalidation marks the cached value stale and spawns a new computation.
  • Error → Computingget_async retry after an error. This transition is mandatory, not optional: a slot in Error holds no cached result, so the next read re-spawns for the current revision. A binding that replays the stored error to every later reader is non-conforming — it makes a transient failure (a timed-out fetch, a disconnected peer) permanent for the lifetime of the slot, with no read path that can recover it. invalidate on an Error slot is therefore a no-op rather than a repair: the retry is owned by the read, not by a dependency change. The executable form is LazilyFormal.AsyncSlotState.step … SlotEvent.retry, which maps error → computing and is a no-op from every other state.

Cancellation contract

A conforming async context MUST honor all five of:

  1. Waiter cancellation is safe. Dropping one get_async future does not cancel the shared in-flight computation while other waiters still need it. Each waiter holds a shared handle; dropping a receiver does not abort the in-flight task.

  2. Stale completion is discarded, not published. When invalidation advances the slot revision during an in-flight computation, the completing future finds its recorded revision no longer matches and discards the result. Waiting callers are retried against the new revision or attached to the newly spawned future.

  3. Explicit cancellation. A hard clear, invalidation, or context disposal may mark the in-flight revision as canceled; if an abort handle is available, the task is aborted. User futures MUST be cancellation-safe, because aborting drops them at an .await boundary.

  4. Context disposal. Dropping the async context cancels all in-flight computations via their abort handles and awaits completion of all active cleanup futures before returning.

  5. Effect cleanup is triggered by rerun or dispose, and completes before the next body. An effect’s cleanup future MUST run only when the effect reruns or is disposed — the trigger fixed by reactive-graph.md § Conformance (“cleanup runs before each rerun and on dispose”) — and when it runs on a rerun it MUST complete before the next body starts. Disposal removes pending reruns before awaiting cleanup.

    A binding MUST NOT run cleanup at the end of the flush that ran the body. This clause previously stated only the ordering (“cleanup completes before the next body starts”) and left the trigger to reactive-graph.md, which turned out to be enough rope: lazily-py’s async effect awaited cleanup at flush end whenever no rerun was queued, and satisfied the ordering vacuously because there was no next body to be ordered against. Found 2026-07-19 by disarm_disposes_nothing, which asserts cleanup_order: [] at a step where nothing was disposed or invalidated, and observed a cleanup.

    The reason the trigger matters rather than just the ordering: the canonical effect acquires a resource in the body and releases it in the cleanup. Running cleanup at flush end releases the resource while the effect is still live and will rerun later — an effect that subscribes and returns an unsubscribe would unsubscribe itself immediately. lazily-go and lazily-dart both retain the cleanup between runs and execute it at the start of the next one; that is the family behavior.

get_async re-resolve contract

get_async MUST treat the slot state as authoritative and re-resolve rather than assert, because the slot can change between its lock acquisitions and a notifier can close under it. It runs an outer loop that, each pass, re-reads the slot via the get() fast path and then re-locks to attach to or spawn a computation. Two concurrency windows are benign (not data inconsistencies — the published value is always correct):

  1. Resolved-since-get(): the slot can transition Computing → Resolved between the fast-path get() (which releases the lock) and the re-lock. Observing Resolved at the re-lock is expected; the cached value is read directly. It is not an unreachable state.
  2. Notifier dropped: the per-computation waiters can all close without a final Resolved signal when an in-flight compute is superseded by a newer revision (the stale Computing → Computing transition early-returns) or the slot is invalidated. A “the world changed” error means re-resolve from current slot state — return the now-published value, attach to the new in-flight compute, or respawn. It MUST NOT panic.

Dependency tracking

Async compute and effect callbacks do not use a thread-local tracking stack (a thread-local does not survive executor thread migration or suspension/resume across .await). Instead each callback receives a compute context:

MethodTracking
get_async(computed)Records the accessed computed as a dependency before awaiting its value
get(source)Records the accessed source as a dependency synchronously

Edges register immediately, so source invalidation while the future is suspended can cancel or supersede the in-flight computation before it publishes stale data. On rerun, stale dependencies are removed and new ones registered; the dependency set is carried by the compute context, not a thread-local.

Async effects

An async effect runs an effect body returning an optional async cleanup:

  • Serialized reruns: reruns are serialized per effect — a rerun does not start until the previous cleanup future completes.
  • Cleanup trigger and ordering: cleanup runs on rerun or dispose and at no other time — not at the end of the flush that ran the body. When it runs on a rerun, the previous run’s cleanup completes before the next body starts; disposal awaits the current cleanup before removing the node. The cleanup is therefore retained between runs rather than executed eagerly. See § “Conformance” item 5 for why the trigger is normative and not merely the ordering.
  • Auto-tracking: the body receives a compute context and tracks dependencies through get_async / get.
  • Scheduled, not inline: dependency invalidation schedules an async rerun after the current invalidation pass; the rerun runs on the runtime executor, not inline within send/batch.
  • Disposal: removes pending scheduled reruns, awaits the current cleanup future, and unsubscribes dependency edges.

Batch support

batch(run) is a synchronous boundary. Cell updates queue invalidation roots; at batch exit, queued roots trigger propagation. Async slots and effects are scheduled for rerun but do not execute inside the batch callback — async reruns execute after the batch returns, on the runtime executor. Mutation semantics stay synchronous at the graph boundary: invalidations schedule async reruns only after the outermost batch exits.

In-flight deduplication & fast path

  • One in-flight computation per revision: each async slot has one published cache and at most one in-flight computation for the current revision. Concurrent get_async callers await the same in-flight result instead of spawning duplicate futures.
  • Synchronous fast path: get() returns the cached value synchronously when the slot is Resolved, avoiding async overhead. get_async() calls get() first; only unresolved or dirty slots enter the async spawn path.

Async state machine

A flat State Machine over the async context keeps send and state synchronous (sources are the synchronous input layer) while reactive observers use the async APIs: on_transition returns an async effect handle and state_is returns an async signal handle. Because resolution is asynchronous, eager recomputation settles on the runtime rather than synchronously within send.

Conformance

An async context conforms when:

  1. The slot state machine (Empty/Computing/Resolved/Error) and its transitions, including the stale Computing → Computing discard, are implemented exactly.
  2. Revision tracking discards every stale completion; a stale value is never published.
  3. All five cancellation properties hold, and disposal awaits cleanup.
  4. get_async re-resolves through both benign-race windows without panicking.
  5. Dependencies are tracked through the compute context (not a thread-local) and registered before the awaited read.
  6. Async effect reruns are serialized, cleanup-before-body ordered, and executor-scheduled rather than inline.
  7. Batching is synchronous at the mutation boundary; async reruns fire only after the outermost batch exits.

Implementation status

The async surface is required of any binding whose platform exposes an async/future runtime (async/await, promises, coroutines, a suspending executor) — see Wire Protocol § Concurrency layers are required. A binding on a platform with no notion of suspendable async computation declares the async capability as none and advertises it, never silently. A binding that ships the async context MUST honor the full cancellation and re-resolve contract above. API-shape checks for the canonical async-v2 names are informational only: they do not admit a peer to transport, CRDT, queue, or sync feature groups. Network-suite membership continues to follow the binding’s advertised production capabilities, channels, codecs, and variants. Concurrency-window coverage is pinned by targeted deterministic tests rather than exhaustive interleaving exploration, because the async resolve loop runs on a real async executor whose primitives a synchronization-model checker cannot shim.

Lossless Tree CRDT

LosslessTreeCrdt is a single rooted concrete-syntax tree whose leaves own every rendered byte, so the tree itself is the lossless wire authority — no separate flat text CRDT floor is required for capable sessions. The defining invariant is losslessness:

render(tree) == source_text

for valid, invalid, and unknown source alike. This chapter specifies the M1 syntax-agnostic core: the tree state model, node/op id formats, the leaf and element node variants, the dotted non-contiguous version frontier, the op vocabulary, the byte-offset policy, and the compute-fixture contract every binding replays. The Lean model of the render algebra, convergence, and frontier soundness lives in lazily-formal (LazilyFormal.LosslessTree, LazilyFormal.LosslessTreeSync).

M1 scope. Create / tombstone / intra-parent reorder / leaf-edit / split-leaf / merge-adjacent-leaves, plus op-based delta sync, plus the op-delta wire schema (see Wire schemas). Deferred to later milestones: cross-parent move (single-parent + acyclicity enforcement), metadata/kind mutation, subtree replace, snapshot/GC and its wire schema, and full SessionHandshake capability negotiation (the capability name is reserved below). The Kotlin and JS bindings land in M2. In the create-only + intra-parent-reorder algebra, single-parent and acyclicity hold by construction.

Tree state model

The document is one rooted tree. Every node has a stable identity and belongs to exactly one parent’s ordered child list.

  • DocumentRoot — the sentinel root element (TreeNodeId {counter: 0, peer: 0}).
  • ElementNode — an internal semantic node with a kind and an ordered list of children. Owns structure only, never text.
  • Leaf — owns one exact source span, classified by a LeafKind:
    • Token — a syntax delimiter or marker;
    • Trivia — whitespace, blank lines, indentation, comments, separators;
    • Raw — valid text the adapter deliberately keeps opaque;
    • Error — invalid or ambiguous text that must still round-trip exactly.

render is a depth-first concatenation of the live leaves’ text in child order. Because every rendered byte belongs to exactly one live leaf and elements own no text, unknown or invalid spans round-trip exactly as Raw/Error leaves rather than being discarded — the requirement that keeps a semantic AST from being mistaken for a lossless tree.

Identity and clock

  • TreeOpId {counter, peer} — a dotted operation id: a Lamport counter tiebroken by peer, totally ordered by (counter, peer). The counter advances past every observed op, so a causally-later write wins last-writer-wins and concurrent ops tiebreak deterministically by peer. (No HLC is needed; tree anti-entropy is op-based.)
  • TreeNodeId — a node’s identity is the id of the op that created it, so a node keeps its id through reorder, edit, and (future) move.
  • SortKey {frac, peer} — a fractional-index child position: orderable bytes tiebroken by the minting peer, so concurrent inserts into the same gap get a deterministic total order. Positions travel inside create/reorder ops, so both replicas store byte-identical keys.

Operation vocabulary

Every mutation is one op carrying everything a remote replica needs to converge deterministically (positions and seed text travel inside the op):

OpEffect
CreateNode {id, parent, sort, seed}materialize an element shell or a text leaf seeded from exact text
Tombstone {node}tombstone a node (sticky; smaller op id wins concurrently)
Reorder {node, sort}LWW position reassignment within the parent (identity + payload preserved)
LeafEdit {node, prev, ops}apply an embedded text-CRDT delta to one leaf
SplitLeaf {node, new, sort, at_char, prev}split a leaf at a char boundary into two adjacent leaves of the same kind
MergeLeaves {left, right, prev_left, prev_right}merge two adjacent leaf siblings; total text unchanged

Split and merge reseed a leaf’s text destructively, so per-leaf text ops form a causal chain: each LeafEdit/SplitLeaf/MergeLeaves carries the prior text-op id (prev) and is buffered until it arrives, keeping out-of-order delivery convergent.

Dotted version frontier

The frontier summarizing “which ops do I hold” is a dot set — per peer, a contiguous prefix plus out-of-order holes — never a per-peer max counter:

frontier[peer] = { contiguous, sparse: {…} }

diff(their_frontier) returns the ops a replica holds that the partner’s frontier lacks (a true set difference over dots), ordered by dotted id. apply_update is idempotent (already-held ops are skipped) and buffers ops whose parent/target or prev has not arrived yet.

A per-peer max would record only the highest delivered dot and imply every lower dot is held; delivering dot 3 while dot 2 is missing would make the partner believe it holds 2, so its diff would omit an op it genuinely lacks and the replicas would never converge. The dot set keeps the hole representable and re-requestable. This is proven in LazilyFormal.LosslessTreeSync (frontier_no_skip, perPeerMax_skips) and exercised by the non_contiguous_anti_entropy fixture.

Offset policy

Wire and API text offsets are UTF-8 byte offsets, leaf-local. The embedded text CRDT is char-indexed, so the byte→char conversion happens only at the two byte-taking mutators (edit_leaf, split_leaf), against that leaf’s current text. Offsets must land on a UTF-8 char boundary and within the leaf, else the mutation is rejected. No binding may treat UTF-16 code units as wire offsets; the Kotlin/JS bindings (later milestones) must convert.

Conformance

The M1 fixtures live in conformance/lossless-tree/ as compute fixtures — each builds an initial tree on replica a, runs a schedule of ops / forks / anti-entropy syncs across named replicas, and asserts exact rendered text, live-node counts, and convergence across delivery orders:

  • exact_roundtrip — Token/Trivia/Raw/Error leaves incl. an invalid span and multi-byte text; render == source;
  • one_leaf_edit_delta — a one-leaf edit at a byte offset into multi-byte text, delivered by anti-entropy;
  • split_merge — split then merge, render preserved, live-node count grows then restores;
  • concurrent_insert_same_parent — two replicas insert into the same gap; both survive, deterministic order;
  • concurrent_reorder_and_leaf_edit — a concurrent move + text edit both apply;
  • non_contiguous_anti_entropy — a delivery hole is re-requested and converges;
  • token_trivia_preservation — a leaf edit leaves adjacent Token/Trivia leaves byte-for-byte unchanged;
  • invalid_source_roundtrip — unclosed fence / comment carried as Error leaves round-trips exactly, and editing an adjacent Raw leaf keeps the Error spans;
  • concurrent_conflict_preserves_text — incompatible concurrent shapes (element wrap vs bare leaf) both survive with no bytes dropped (text preservation wins over semantic shape; adapter-level raw/error degradation layers above the core);
  • apply_update_advances_counter — the counter advance above, observed: a replica reorders AFTER syncing in six remote reorders, and its new op must outrank the stamp it just ingested. The failure is symmetric (both replicas converge on the wrong text), so render_on is what sees it, not converged;
  • out_of_order_delivery_buffers — a three-op batch delivered in reversed order (deliver.order) must drain through the dependency buffer; a binding that drops what it cannot yet apply still records the dot, so the following full sync returns nothing and the loss is permanent.

See Conformance Fixtures for the fixture format and the binding replay contract.

Wire schemas

The op-delta wire form is pinned by two JSON Schemas, derived from the lazily-rs reference’s serde output (which is the normative form):

  • schemas/lossless-tree.json — the shared vocabulary $defs: OpId, SortKey (frac is a u8 array, never base64), LeafKind (PascalCase on the wire), NodeSeed, TextOp, the externally-tagged TreeOpKind / TreeOp, and the dotted TreeVersionFrontier / DotRange.
  • schemas/lossless-tree-delta.json — the TreeUpdate message: { "ops": [TreeOp, …] }, the output of diff and the input to apply_update.

The reference validates its own TreeUpdate and frontier serde output against these schemas (lazily-rs/tests/lossless_tree_schema.rs), and the M2 Kotlin/JS ports validate their emitted frames against the same files. The snapshot/checkpoint wire schema is deferred with snapshot/GC.

Capability

A binding that implements this chapter advertises the capability name lossless_tree_crdt_v1 (reserved here; wired into SessionHandshake capability negotiation in a later milestone). A peer that lacks it falls back to the flat text CRDT floor.

CrdtTree Contract (#lzcrdttree)

CrdtTree is the shared lossless document-CRDT seam used by snapshots, relay canonicals, and document-op replication. An implementation exposes:

  • version_vector() — a compact distributed frontier;
  • delta_since(frontier) — every operation the frontier has not observed;
  • apply_delta(delta) — an identity-preserving, idempotent fold;
  • text() / value() — the visible lossless projection; and
  • merge_from(other) — the state join.

The join and delta fold are commutative, associative, and idempotent. A full snapshot is exactly delta_since(empty_frontier); applying it to an empty replica must preserve operation identities, not reparse the visible text and mint a new lineage. Applying delta_since(version_vector()) is a no-op.

The compute fixture at conformance/crdt-tree/algebra.json pins merge order independence, idempotence, snapshot equivalence, and incremental round-trip behavior. The Lean model in LazilyFormal.Replication pins the three join laws.

Durable Outbox Stores (#lzdurableoutbox)

Reliable sync separates protocol from persistence:

OutboxStore (ordered byte CRUD) → Outbox<S> (ack/prune/replay protocol) → SyncDriver

OutboxStore owns only put, delete_through, scan_after, load_cursor, and save_cursor. Outbox<S> owns serialization and the invariants:

  1. append before send;
  2. retain every frame until the peer acknowledges its epoch;
  3. keep the acknowledgment cursor monotone, including when a stale storage handle writes after a newer handle;
  4. prune only epochs at or below that cursor; and
  5. replay epochs above the cursor in ascending order after restart.

Persistent save_cursor(epoch) implementations MUST serialize max(stored_cursor, epoch) atomically. A process-local maximum is insufficient: two handles can both open at cursor zero, then write 9 and 3 in that order. The serialized result must remain 9, and a subsequent protocol read must observe 9.

Bindings may provide platform stores without duplicating this protocol. Rust ships InMemoryStore and feature-gated SqliteStore; Kotlin ships a SQLite/Room-shaped store; browser JS ships IndexedDbStore; other native bindings ship append-only file journals. SQLite is never a default/WASM feature.

conformance/reliable-sync/outbox_store_protocol.json pins the storage-independent behavior. LazilyFormal.Replication proves cursor monotonicity and that replay never contains a pruned epoch.

An append-only journal is a versioned durable wire surface: a later process, and possibly a different build of the same binding, reads records written earlier. Readers MUST reject a complete record whose opcode they do not recognize. They MUST NOT skip it as though it were a cursor marker, because an ignored pruning operation can resurrect acknowledged frames and redeliver them while reporting a successful scan.

The sole recovery exception is a torn trailing record. A crash may interrupt the last append, so a reader MUST forgive one incomplete final record and preserve every complete record before it. The same malformed bytes in an interior record are corruption and MUST be rejected. The logical, encoding-neutral cases are pinned by conformance/reliable-sync/outbox_journal_decode.json; bindings encode them into their native JSON-lines or binary journal format before replay.

Durable Effect Sinks (#lzdurablesink)

Lazily fixes one direction of authority between live reactive state and durable storage:

cold durable state ──hydrate once──▶ live Lazily state
                                      │
                                      ▼
                          computed projection or ordered fact stream
                                      │
                                      ▼
                              Effect / AsyncEffect
                                      │
                                      ▼
                             write-only durable sink
                                      │
                                ack / failure
                                      └────────▶ live Lazily state

Invariant. While a Lazily runtime is live, transitions are decided from Lazily state. Durable storage receives a projection or an ordered fact as an effect. A sink MUST NOT reload storage and use it to arbitrate the transition it is currently persisting.

This makes durable storage an effect sink. It does not claim that every effect is durable, nor that durable storage is always eventually consistent. Lazily is not a database framework: applications continue to own serialization, transactions, schema migration, storage selection, and startup recovery. Lazily ships no SQLite, filesystem, or cloud-store adapter as part of this decision — the sink is an application-owned trait (see examples below).

Authority rule

  • Live coordination, compare-and-swap, ownership, deduplication, leases, and transition selection happen in Lazily state.
  • Durable I/O runs from an Effect / AsyncEffect, or from a TopicCell / DurableOutbox drain when history must be lossless.
  • A runtime sink is write-only with respect to transition authority. Loading and migration belong to a separate startup hydrator that runs once before the runtime is live, not on the decision seam.
  • Computed values and transition reducers stay pure — a reducer MUST NOT perform I/O or read storage to decide a transition.
  • Success advances a monotone acknowledgement such as durable_through(epoch).
  • Failure stays represented in live state as pending / retrying / backpressured. It MUST NOT trigger a database reload at the decision seam.
  • Values on the existing Ephemeral plane MUST NOT enter a durable sink. Reuse the existing Durable marker — do not invent a second marker hierarchy. (The Ephemeral-never-Durable separation is already pinned by Presence.ephemeral_never_durable.)

Projection vs history — pick the shape before the API

Ordinary Effect / AsyncEffect observe settled reactive state. Lazily already coalesces effect reruns across a batch, so a batch containing A → B → C may persist only C. That is correct for a durable projection (current/recoverable state). Durable history has a different contract: every accepted fact must survive and remain ordered. It uses the existing TopicCell / DurableOutbox family — stable cursor, replay, idempotency key, and monotone acknowledgement — not ordinary effects. Do not modify ordinary effects to retain intermediate values; that would weaken their coalescing semantics and duplicate the ordered-stream primitives.

Before choosing an API, answer two questions:

  1. Is this sink persisting the latest projection or an ordered history?
  2. Must the transition be durable before it is externally visible?

Those two answers fix the shape:

NeedLazily sourceSink contract
Latest recoverable stateComputed read by Effect / AsyncEffectIdempotent upsert of the latest epoch
Every accepted factTopicCell or existing DurableOutboxAppend / replay / ack with a stable cursor
Durable before visibleOrdered fact + application acknowledgement cellVisibility waits for monotone durable_through
Ephemeral statePresence / Ephemeral primitivesPersistence rejected

Application-owned sink trait

The sink is a narrow, write-only trait owned by the application. The reactive graph holds no reference to a query/hydration interface — those live in a separate module used only by the startup hydrator, never passed into the runtime effect. (Phase 3 adds architecture tests that fail if a hot-path actor module imports the persistence query/CAS or file-lock modules.)

The two examples below sketch the trait shape in Rust-flavoured pseudocode. The store API is deliberately minimal — upsert_latest for a projection, append_fact for history — and always carries the epoch so success can advance a monotone durable_through.

Example 1 — coalesced current-state projection

A Computed projects the actor’s current state. An Effect reads it and upserts only the settled value; intermediate batch values are coalesced away by Lazily’s existing effect-batch dedup, so the sink sees one write per batch. Acknowledgement advances durable_through; a sink failure flips the live actor to retrying without reloading storage.

#![allow(unused)]
fn main() {
// Application-owned, write-only. No read/CAS surface reaches the runtime effect.
trait ProjectionSink {
    fn upsert_latest(&mut self, epoch: u64, state: &ActorState) -> Result<(), SinkErr>;
}

// Live state. Pure reducer; no I/O.
let live_state: Source<ActorState> = ctx.source(initial);
let epoch:     Source<u64>         = ctx.source(0);
let projected: Computed<ActorState> = ctx.computed(|c| c.get(live_state).clone());

// Durable I/O runs from an Effect — never reads storage to decide a transition.
ctx.effect(|c| {
    let s = c.get(projected);
    let e = c.get(epoch);
    match sink.upsert_latest(e, s) {
        Ok(())  => c.set(durable_through, e),            // monotone ack
        Err(_)  => c.set(status, Status::Retrying),      // failure stays live
    }
});
}

Example 2 — lossless ordered fact sink

Every accepted fact must survive and stay ordered, so the source is a TopicCell (or the existing DurableOutbox), drained by an effect into the application store. Replay from a stable cursor covers every epoch after the durable frontier; duplicate delivery is idempotent by the event_id key. This is the DurableOutbox append-before-send / replay-from-cursor / ack_through contract, applied as a sink.

#![allow(unused)]
fn main() {
trait HistorySink {
    fn append_fact(&mut self, epoch: u64, fact: &Fact) -> Result<(), SinkErr>;
    fn ack_through(&mut self, epoch: u64);
}

let facts: TopicCell<Fact> = ctx.topic();
// Drain: every unacked fact is appended in order; ack advances the cursor.
ctx.effect(|c| {
    while let Some(fact) = facts.read_after(cursor) {
        sink.append_fact(fact.epoch, &fact)?;
        sink.ack_through(fact.epoch);          // cursor advances; GC-safe below it
        cursor = fact.epoch;
    }
});
}

Visibility policy (caller-chosen)

A caller picks an explicit policy per transition; Lazily mandates none:

  • eventual_projection — live state is immediately authoritative; the sink persists in the background and may lag.
  • durable_before_applied — external visibility waits for the durable_through(epoch) acknowledgement before the transition is observed as applied (an ordered fact plus an application acknowledgement cell).
  • ephemeral — no durable sink; the value lives on the Ephemeral plane.

Cold restart

Restart recovery is the hydrator’s job, run once at startup before the runtime is live: load the last acknowledged projection (or replay the ordered fact log up to the durable cursor) into live state, then resume. After hydration, authority is live-only — a sink failure during a live transition never rolls authority backward by rehydrating storage at the decision seam.

Formal backstop

lazily-formal/LazilyFormal/DurableSink.lean pins the load-bearing invariants: durable_through is monotone; a batched projection persists only the settled epoch (coalescing); an ordered history replays every epoch past the durable cursor; and a sink failure leaves live authority unchanged (no rehydrate-at- decision-seam). The Ephemeral-never-Durable separation is already proven in Presence.ephemeral_never_durable.

Transport-agnostic reactive egress (#lzegress)

Egress is the send-side mirror of reactive ingress. The pure EgressCore<T> owns delivery authority:

  • monotone sequence assignment;
  • the pending queue and bounded unacknowledged window;
  • the monotone cumulative acknowledgement watermark;
  • retry attempts, bounded exponential backoff, and exhaustion; and
  • the producer-generation fence.

The graph shells expose that state as Computeds. Transport I/O lives in exactly one Effect per attachment:

EgressCore<T>                 sequence, window, watermark, retry, fence
├── EgressCell<T>             Context shell
├── ThreadSafeEgressCell<T>   ThreadSafeContext shell
└── AsyncEgressCell<T>        AsyncContext shell
          │
          └── one attached Effect ──send──▶ transport

An Effect is not an egress family by itself: it has no retained delivery state. The Effect observes the pending and inflight projections, asks the core to claim eligible envelopes, and sends those envelopes. The core decides whether a claim, acknowledgement, failure, retry, or reconnect is admissible.

This is reactive projection, not a request/ACK protocol. Enqueue invalidates pending; claiming moves an envelope to inflight; an acknowledgement invalidates inflight and re-runs the same attachment when it reopens a bounded send window. The acknowledgement is domain delivery input to the core, not an acknowledgement of a projection request.

Envelope identity and ordering

Each enqueue assigns the next sequence in the current sequence space:

enqueue(payload) =
  pending.push({ generation, sequence: next_sequence, attempt: 0, payload })
  next_sequence += 1

(generation, sequence) is the transport idempotency identity. Pending records are claimed oldest-sequence first. A successful claim increments attempt and moves the record into the unacknowledged window. A claim is refused when that window reaches inflight_limit.

An acknowledgement is cumulative. ack(generation, through) advances acked_through only when through is greater than the current watermark, and removes all pending, in-flight, or retry-parked records at or below it.

Send-side generation fence

Every transport Effect captures the generation current when it attaches. Every send-side transition supplies that generation to the core. A mismatched generation is rejected without changing state or invalidating a reader.

reconnect(new_generation) must strictly advance the generation. It:

  1. fences the old Effect;
  2. preserves next_sequence and acked_through;
  3. moves every unacknowledged in-flight or retry-parked record back to pending in sequence order; and
  4. rewrites all pending envelopes to the new generation.

The old Effect remains attached but inert. The replacement transport receives one new Effect, which replays the retained pending projection under the new incarnation.

Retry

retry_budget counts retries after the first send attempt. A failed in-flight record is parked until the derived backoff elapses:

backoff(attempt) =
  min(retry_ceiling, retry_base * 2^(attempt - 1))

While budget remains, retry_now returns the parked record to pending in sequence order. Once exhausted, the record is terminal and cannot be scheduled again. Parking is explicit so a failed send cannot cause the attachment Effect to spin in an immediate reactive retry loop.

Reader kinds and invalidation

ReaderTypeInvalidated by
pendingVec<EgressEnvelope<T>>enqueue, claim, retry scheduling, replay, cumulative ACK pruning
inflightVec<EgressEnvelope<T>>claim, failure, replay, cumulative ACK pruning
acked_throughOption<u64>advancing ACK only
retryOption<EgressRetry>failure, retry claim/scheduling, replay, ACK pruning

No-op and stale-generation transitions invalidate nothing. The async shell uses synchronous Computeds for the delivery state; only the transport attachment is async-coloured.

Composition boundaries

Egress does not duplicate existing storage or flow-control primitives:

  • use RelayCell / Outbox ahead of enqueue for conflation and backpressure;
  • use SpillStore for overflow-to-storage;
  • use DurableOutbox for durable recovery; and
  • use transport framing outside the core.

Those adapters feed or persist the egress state machine. They do not move sequence assignment, acknowledgement authority, or the generation fence into an Effect.

Conformance and formal obligations

The canonical corpus in conformance/egress/ covers ordered cumulative ACK, window reopening, bounded retry, and reconnect fencing. Binding runners must assert both returned outcomes and the full projected state after every step.

The Lean model proves:

  • sequence assignment is monotone;
  • acknowledgement progress is monotone;
  • stale-generation send-side transitions preserve state;
  • reconnect strictly advances generation while preserving the ACK watermark; and
  • permitted retry attempts are bounded by the configured budget.

Cross-Language Feature Coverage

This is the canonical feature-coverage matrix for the lazily family. Each binding’s README reproduces it; this page is the source of truth. It is a status view (what each port ships today), distinct from the normative Binding Conformance Matrix, which fixes what every binding must eventually provide.

Legend: ✅ shipped · ~ partial · absent · not applicable (see notes).

The table below is generated from coverage.json by scripts/sync-coverage.mjs. Edit coverage.json and run make coverage-sync (or node scripts/sync-coverage.mjs) to update this table and every binding README in one shot; make coverage-check guards drift in CI.

Summary — family × language

FamilyRustPythonKotlinJSDartZigGoC++C#GDScript
Reactive graph~
Materialization
Family sync
Statecharts
Keyed collections~
Reactive queue
Broadcast topic
Work queue
CRDT data types~~~~~~~
Lossless tree
Egress~~~~~~~~
Ingress
Wire codec~
Transport & FFI~~~
Message passing~
Reliable sync~~~~~~~~~
Distributed plane
Causal receipts~
Security boundary
Membership
Coordination
Presence
Temporal
Rate shaping
Windowing
Resilience
Portable stdlib
Service plane
Instrumentation

Roll-up rule: a family cell is only when every required row in that family is ; ~ when the family is mixed (some shipped or partial); when no required row is shipped or partial; only when every required row in the family is not applicable. Rows the spec marks MAY (optional, shown as opt below) are excluded from the roll-up — declining an optional feature is not a gap.

Reactive graph

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Reactive graph 1~
Thread-safe context 2
Async reactive context 3
Merge algebra 4~

Materialization

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Keyed-map materialization 5
Thread-safe keyed map 6
Async keyed map 7

Family sync

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Keyed-map sync 8

Statecharts

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Flat state machine 9
Harel state charts 10

Keyed collections

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Keyed reactive maps 11
ReactiveMap core — single-threaded 12~
ReactiveMap core — thread-safe 13
ReactiveMap core — async 14
Exact-key dependency availability 15
Atomic ordered move 16
Memoized semantic tree 17
Stable-id alignment 18

Reactive queue

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Reactive queue core — single-threaded 19
Reactive queue core — thread-safe 20
Reactive queue core — async 21

Broadcast topic

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Broadcast topic core — single-threaded 22
Broadcast topic core — thread-safe 23
Broadcast topic core — async 24

Work queue

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Work queue core — single-threaded 25
Work queue core — thread-safe 26
Work queue core — async 27

CRDT data types

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Free-text character CRDT 28
TextCrdt delta sync 29
CrdtTree lossless document 30
Move-aware sequence CRDT 31
Registers (LWW/MV) + PnCounter 32~~~~~~~

Lossless tree

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Lossless tree CRDT core 33
Lossless tree — anti-entropy 34
Lossless tree — merge convergence 35

Egress

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Reactive egress 36
Egress — thread-safe 37
Egress — async 38
RelayCell 39

Ingress

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Reactive ingress 40
Ingress — thread-safe 41
Ingress — async 42
Boundary-ingress adapter 43

Wire codec

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
IPC wire — Snapshot/Delta/CrdtSync 44~
Frame codec — json 45
Frame codec — msgpack 46
Frame codec — postcard (opt) 47
NodeId/PeerId exact-representation 48
NodeKey null-leniency 49
Capability negotiation 50

Transport & FFI

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Shared-memory blob path 51~~~
Cross-process zero-copy transport 52
C-ABI FFI boundary 53

Message passing

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Message-passing + RPC command plane 54~

Reliable sync

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Reliable sync 55~~~~~~~~~
Storage-independent durable outbox 56
Reliable-sync transport seam 57

Distributed plane

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Distributed CRDT plane 58
Distributed plane — WebRTC 59
State projection / mirror 60

Causal receipts

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Causal receipts 61~

Security boundary

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Permission boundary 62

Membership

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Membership + failure detection 63

Coordination

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Distributed coordination 64

Presence

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Presence + ephemeral plane 65

Temporal

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Temporal sources 66

Rate shaping

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Rate-shaping operators 67

Windowing

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Stream windowing 68

Resilience

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Fault tolerance 69

Portable stdlib

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Portable stdlib Timer 70
Portable stdlib Timeout 71
Portable stdlib RevisionBarrier 72

Service plane

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Embedded-service plane 73

Instrumentation

FeatureRustPythonKotlinJSDartZigGoC++C#GDScript
Instrumentation / benchmarks 74

Convergence and the wire protocol are pinned by the shared conformance fixtures and JSON Schemas in this repo and the Lean models in lazily-formal.

Notes

  • ᵃ Python reactive graph: Cell / Slot / Signal / Effect (sync) and the top-level batch(run) boundary ship; the != PartialEq memo guard applies to cells, slots, and signals. The async counterpart (AsyncEffect) queues reruns at the batch boundary for asyncio reactors.
  • ᵇ Dart reactive graph: Context / Slot / Cell / Signal ship; there is no standalone Effect type (observers subscribe on cells) and batch is scoped to the async context.
  • ᶜ Zig reactive graph: Cell / Slot / Signal / Effect and the public batch(run) boundary ship (context.zig coalesces the eager-recompute drain at the outermost batch exit).
  • ᵈ Serialized context (JS / Dart) — decision: both are meaningful as serialized, realm-local execution flavors, not as shared-graph contexts. They are scored when their own surfaces replay the portable Core fixtures, and are excluded from cross-thread shared-graph stress/model-check tests. A duplicate wrapper is not sufficient by itself: unsupported Core features remain staged, and runner-local locks/dictionaries cannot stand in for them. Either binding may remove the flavor and declare it absent; while it advertises the flavor, it must join every feature peer group it actually supports. See protocol.md § Concurrency layers are required for the conditional layer requirement and reactive-graph.md § Declared context capabilities for the serialized / shared-graph distinction.
  • ᵉ Zig async context: Zig removed language async and has no suspendable executor, so the layer is a task-queue + settle() drain surface — the synchronous graph’s pending_recompute/drainPendingRecompute generalized with revision tracking and the 4-state slot machine (async_context.zig).
  • ᶠ Zig collections: SourceMap / ComputedMap with atomic move, SourceTree (per-level membership/order reactivity, atomic child move), and the LIS-move-minimized reconcile op-set all ship (collection.zig, cell_tree.zig, reconcile.zig).
  • ᵍ Shared-memory blob path (JS / Dart): carry ShmBlobRef wire references but no host-side ShmBlobArena — the I/O-channel fallback of the shared-memory carve-out (see protocol.md § Shared-memory payload path is required).
  • ʰ Dart distributed CRDT plane: the CrdtPlane engine (HLC / stamp frontier / stability watermark) ships, but is not yet wired to live merge: crdt root cells.
  • ⁱ C-ABI FFI (JS): platform carve-out ffi = none — browser/Worker JS has no shared in-process C ABI. The full state plane (including CrdtSync) still flows over IPC / WebSocket / WebRTC (see protocol.md § C-ABI FFI is required).
  • Distributed plane — WebRTC transport + signaling (Rust / Kotlin / JS / Zig): the portable stack (signaling protocol + client, the DataChannel seam, permission- filtering sink/source, in-memory loopback, and the CRDT plane runtime) ships and is conformance-tested; the concrete native WebRTC backend is a platform adapter (str0m in Rust; the browser RTCPeerConnection in JS; a consumer-provided seam in Kotlin and Zig), matching the reference design where the heavy transport is optional behind the seam.

  1. Reactive graph — two cell kinds (nodes SourceCell / ComputedCell; handles Source<T, M> / Computed<T>) + Effect sink + eager Computed (computed().eager()) / all cells guarded / batch

  2. Thread-safe context (lock-backed)

  3. Async reactive context

  4. Merge algebra + Source<T, M> — associative MergePolicy (KeepLatest/Sum/Max/SetUnion/RawFifo), Cell ≡ Source<KeepLatest>, read-any-cell/write-Source split (#relaycell)

  5. Keyed-map materialization (ComputedMap) — mint-on-access derived slots: transparency + deferral (#lzmatmode)

  6. Thread-safe keyed map (ThreadSafeComputedMap) — Send + Sync + materialization confluence (#lzmatmode)

  7. Async keyed map (AsyncComputedMap) — eventual transparency (#lzmatmode)

  8. Keyed-map sync — membership propagation + materialize-on-ingest + derived-aggregate transparency (#lzfamilysync)

  9. Flat state machine

  10. Harel state charts

  11. Keyed reactive maps (ReactiveMap: SourceMap / ComputedMap) + SourceTree + reconcile

  12. ReactiveMap Core surface — single-threaded flavor (cell-model.md § Core surface vs. binding extensions)

  13. ReactiveMap Core surface — thread-safe flavor (ordering + membership reactivity)

  14. ReactiveMap Core surface — async flavor (ordering + membership reactivity)

  15. Exact-key dependency availability (DependencyMap: observe before publish, unrelated-key isolation, stable identity; #lzdependencyavailability)

  16. Atomic ordered move replayed against all three flavors (cellmap_atomic_move + cellmap_independence)

  17. Memoized semantic tree (SemTree)

  18. Stable-id alignment (manufactured identity)

  19. Reactive queue (QueueCell SPSC/MPSC + QueueStorage adapter) Core surface — single-threaded flavor

  20. Reactive queue (QueueCell SPSC/MPSC + QueueStorage adapter) Core surface — thread-safe flavor (reader kinds + closure lifecycle)

  21. Reactive queue (QueueCell SPSC/MPSC + QueueStorage adapter) Core surface — async flavor (reader kinds + eventual transparency)

  22. Broadcast topic (TopicCell) Core surface — single-threaded flavor — independent cursors + durable replay + safe GC (#lztopiccell)

  23. Broadcast topic (TopicCell) Core surface — thread-safe flavor (reader kinds + closure lifecycle)

  24. Broadcast topic (TopicCell) Core surface — async flavor (reader kinds + eventual transparency)

  25. Competing-consumer work queue (WorkQueueCell) Core surface — single-threaded flavor — exclusive leases + ack/nack + redelivery + DLQ (#lzworkqueue)

  26. Competing-consumer work queue (WorkQueueCell) Core surface — thread-safe flavor (reader kinds + closure lifecycle)

  27. Competing-consumer work queue (WorkQueueCell) Core surface — async flavor (reader kinds + eventual transparency)

  28. Free-text character CRDT (TextCrdt)

  29. TextCrdt delta sync (version_vector / delta_since / apply_delta)

  30. CrdtTree lossless document contract (#lzcrdttree)

  31. Move-aware sequence CRDT (SeqCrdt)

  32. Registers (LWW / MV) + PnCounter + CellCrdt

  33. Lossless tree CRDT core (LosslessTreeCrdt, M1)

  34. Lossless tree — dotted-frontier anti-entropy

  35. Lossless tree — concurrent merge convergence

  36. Transport-agnostic reactive egress (EgressCore) — monotone sequence assignment, bounded unacknowledged window, cumulative ACK watermark, bounded retry/backoff/exhaustion, producer-generation fence (#lzegress)

  37. Egress family — Send + Sync flavor (ThreadSafeEgressCell): delivery authority stays in the shared core, one attached transport Effect per incarnation (#lzegress)

  38. Egress family — async flavor (AsyncEgressCell): the delivery-state readers stay synchronous Computeds; only the transport attachment is async-coloured (#lzegress)

  39. RelayCell — conflating relay + BackpressurePolicy + SpillStore + Transport + Inbox/Outbox + Rate/Window/Expiry/Priority/keyed policies (#relaycell)

  40. Transport-agnostic reactive ingress (IngressCell) — keyed lifecycle scopes, generation/sequence/freshness envelopes, reorder buffer, accepted/dropped/error receipt readers (#designimplementtransport)

  41. Ingress family — Send + Sync flavor (ThreadSafeIngressCell): one frontier walk per admission (#designimplementtransport)

  42. Ingress family — async flavor (AsyncIngressCell): admission is not async-coloured (#designimplementtransport)

  43. Boundary-ingress adapter (BoundaryIngressAdapter) — the non-reactive/reactive seam ahead of IngressCell: one monotone channel cursor per producer generation, snapshot-plus-event bootstrap applied as one batch, derived ReplayRequired(from) on a cursor gap, generation fence + hot resubscribe (#designimplementtransport)

  44. IPC wire — Snapshot + Delta + CrdtSync

  45. Frame codec — json reference codec: dependency-free interop floor, FFI baseline form, byte-canonical (MUST) — executable round-trip obligation (conformance/codec/frame_roundtrip_json.json, #lzmsgpackparity)

  46. Frame codec — msgpack cross-language binary default: externally-tagged frame over named-field maps, semantic (not byte-identical) round-trip (MUST) — executable round-trip obligation (conformance/codec/frame_roundtrip_msgpack.json, #lzmsgpackparity). Shipping a MessagePack codec does not earn this mark: lazily-cpp read ~ here while its private internally-tagged framing wore the token, and only flipped once it shipped the spec wire (#lzcppmsgpackwire)

  47. Frame codec — postcard positional same-schema fast path: smallest + byte-canonical, not cross-language (MAY)

  48. NodeId / PeerId exact-representation bound (MUST) — a decoder that cannot represent a received identifier exactly rejects the frame rather than rounding it (conformance/codec/nodeid_exact_range.json, #lzspecdecoderbound). A binding’s exact range MAY be narrower than the u64 wire type; ✅ means it refuses outside that range instead of substituting a neighbouring id, not that it carries the full u64. Exact ranges: full u64 in Rust / Zig / C#, unbounded in Python, [0, 2^63) in Kotlin / Go / C++, [0, 2^53) in JS, and platform-split in Dart (63-bit on the VM, 53-bit on web). protocol.md stated only the PRODUCER half until this audit, and two C++ decoders were substituting rather than refusing.

  49. NodeKey null-leniency on decode (MUST) — omit-when-absent binds the ENCODER; a decoder reads both an omitted key and an explicit key: null as absent, refusing neither and constructing a key from neither (conformance/codec/nodekey_null_leniency.json, #lzkeynullstrict). Replayed on BOTH optional-key sites (NodeSnapshot, the NodeAdd delta op) in both codecs, and the fixture pins the RE-ENCODED field set as well: reading null as absent and writing it back out is a correct decode with a non-conforming encoder. Before the audit lazily-py and lazily-zig refused the null form, and lazily-kt decoded it into a real key named null — all three had the same field right on CrdtOp, in the same file.

  50. Capability negotiation (SessionHandshake)

  51. Shared-memory blob path (ShmBlobArena)

  52. Cross-process zero-copy transport (BlobBackend / shm / arrow)

  53. C-ABI FFI boundary

  54. Message-passing + RPC command plane (command-plane-v1)

  55. Reliable sync — resync coordinator + at-least-once durable outbox + OR-set/LWW liveness (#lzsync)

  56. Storage-independent durable outbox (OutboxStore + shared outbox protocol; SQLite/Room/IndexedDB/file adapters)

  57. Reliable-sync transport seam + full-duplex SyncDriver loop (IpcSink/IpcSource, #sync-driver)

  58. Distributed CRDT plane (CrdtPlaneRuntime / anti-entropy)

  59. Distributed plane — WebRTC transport + signaling

  60. State projection / mirror

  61. Causal receipts (CausalReceipts outcome projection)

  62. Permission boundary (PeerPermissions / RemoteOp)

  63. Membership + failure detection — MembershipCell (SWIM + Phi-accrual) / PeerSet / PeerChangeEvent (#lzmemb)

  64. Distributed coordination — LeaseCell / LeaderCell / LockCell / SemaphoreCell / BarrierCell+QuorumCell (#lzcoord)

  65. Presence + ephemeral plane — PresenceCell / AwarenessCell / EphemeralCell + Ephemeral/Durable markers (#lzpresence)

  66. Temporal sources — TimerCell / IntervalCell / CronCell / DeadlineCell over a logical clock (#lztime)

  67. Rate-shaping operators — DebounceCell / ThrottleCell / SampleCell / ProbabilisticSampleCell (#lzrateshape)

  68. Stream windowing — TumblingWindow / SlidingWindow / SessionWindow over the merge algebra (#lzwindow)

  69. Fault tolerance — CircuitBreakerCell / RetryPolicyCell / BulkheadCell / TimeoutCell (#lzresilience)

  70. Portable stdlib Timer (stdlib_timer_v1) — canonical fixture + mutation-gate verified

  71. Portable stdlib caller-driven Timeout<T> (stdlib_timeout_v1) — distinct from reactive TimeoutCell

  72. Portable stdlib RevisionBarrier (stdlib_revision_barrier_v1) — register/recheck lost-wakeup guard

  73. Embedded-service plane — HealthCell / ReadinessCell / DiscoveryCell / ServiceRegistry (#lzservice)

  74. Instrumentation / benchmarks

lazily Wire Protocol Specification

Normative source of truth for all lazily language bindings.

Shared Types

NodeId / PeerId

NodeId = u64
PeerId = u64

Wire-stable identifiers decoupled from internal SlotId. Serialized as bare JSON numbers.

Producer bound. A peer whose runtime represents integers as IEEE-754 doubles MUST keep emitted NodeId/PeerId values at or below 2^53 − 1 (Number.MAX_SAFE_INTEGER). The constraint is the runtime’s numeric representation, not the source language: it binds every JavaScript target — which includes Dart, Kotlin, and TypeScript compiled to JavaScript — and does not bind a Dart or Kotlin peer running on the VM/JVM. Earlier wording named “JavaScript/TypeScript peers”, which read as a source-language rule and left compiled-to-JS peers apparently exempt.

Decoder obligation — refuse, never round (#lzspecdecoderbound). A decoder that cannot represent a received NodeId/PeerId exactly MUST reject the frame. It MUST NOT round, truncate, saturate, wrap, or otherwise substitute a nearby representable value. Rounding is the worst available behaviour: the frame decodes cleanly, the identifier now addresses a different node, and nothing downstream can detect the substitution — whereas a rejected frame is a visible protocol error a peer recovers from by resync (§ Resync / gap handling). The producer bound keeps a conforming producer from ever provoking this; the decoder clause is what makes a non-conforming producer, or a corrupted frame, fail loudly instead of silently. The bound stated only the producer half until #lzspecdecoderbound, and the receiving half is exactly where bindings diverged.

A binding’s exact range MAY be narrower than u64. The wire type stays u64; a binding whose identity type is narrower is interoperable over the sub-range it represents and satisfies the clause by failing the decode outside it — it is not required to widen. Audited ranges:

BindingIdentity typeExact rangeOutside the range
lazily-rsu64full u64serde decode error
lazily-zigu64full u64error.Overflowstd.json yields .number_string, parseInt(u64) refuses
lazily-csulongfull u64FormatException from JsonElement.GetUInt64
lazily-pyintunboundedn/a — a Python int has nothing to round to
lazily-goint64[0, 2^63)json: cannot unmarshal number … of type int64
lazily-ktLong[0, 2^63)NumberFormatException from JsonPrimitive.long
lazily-cppint64_t[0, 2^63)std::runtime_error from the json parser / msgpack reader
lazily-jsnumber[0, 2^53)TypeError — a Number.isSafeInteger guard in both codecs
lazily-dartint[0, 2^63) on the VM, [0, 2^53) on webUnsupportedError (#lzdartintwidth)

Only lazily-js and lazily-dart carried the rule deliberately; the other seven inherited it from a parser that happens to fail closed, which is a property no test held in place — and two of them did not hold it. lazily-cpp’s msgpack reader cast a uint 64 straight to int64_t, so u64::MAX decoded as -1: a well-formed identifier for a different node, with no error anywhere. Its json parser refused the same value, but by letting std::stoll throw std::out_of_range, which is a std::logic_error and therefore escaped the std::runtime_error every caller guards a decode with. Both are fixed; the fixture is what found them. conformance/codec/nodeid_exact_range.json makes the clause executable in both json and msgpack form. Its wire frames are carried as raw text (json) and hex bytes (msgpack), and the expected identifier as a decimal string, because a runner that parsed the fixture as JSON on a double runtime would round the expected value before it ever compared anything. The 2^53 − 1 scenarios are exact — every binding MUST decode them to that value, which is what stops a runner that rejects everything from passing vacuously; 2^53 + 1 and u64::MAX are exact_or_reject. A decoder that yields a different number fails.

NodeKey

NodeKey = string   // a "/"-joined path, e.g. "scores/alice", "outer/k1/inner/k2"

An optional, wire-stable keyed address for a collection entry (a SourceMap / ComputedMap entry). Unlike NodeId — the volatile internal handle a producer may re-mint after a resync or remove-then-readd — a NodeKey is producer-defined and stable across NodeId churn, so a peer can subscribe to “entry scores/alice” without an out-of-band key→NodeId map. A multi-segment path addresses nested collections (an entry of a SourceMap inside a SourceMap entry) with no extra machinery.

NodeKey is additive — it never changes NodeId semantics. It appears only as the optional key field on NodeSnapshot and the NodeAdd delta op.

Bounds (reject on construction and on the wire): path ≤ 1024 bytes; ≤ 32 /-separated segments; empty path and empty segments (leading/trailing/double /) are rejected.

Serialization is format-aware. Self-describing codecs (JSON, MessagePack) omit the key field when absent, so pre-key encoders/decoders and existing conformance fixtures round-trip unchanged; positional Postcard always carries the optional discriminant for binary schema stability. Cross-language implementations (lazily-py, lazily-zig, lazily-js, lazily-kt, lazily-go) add the optional nullable key field; they need not emit it when no key is set. Multi-producer key uniqueness (last-writer rule) is owned by the distributed CRDT plane, not this protocol.

Omit-when-absent binds the ENCODER; the decoder is lenient (#lzkeynullstrict). A conforming encoder omits key when there is no key. A decoder MUST accept both the omitted field and an explicit key: null, and read both as absent — it MUST NOT refuse the null form, and MUST NOT construct a key from it. This was previously stated only as “a decoder that sees no key field treats it as absent”, which settled the omitted form and left the null form undefined; three bindings then diverged. The null form is not hypothetical: a serde-based peer that simply did not apply skip_serializing_if emits it, rmp_serde’s own decoder reads it back as absent, and a decoder that refuses is stricter than the reference implementation on a frame the reference implementation produces.

Note the asymmetry that makes this easy to get wrong. CrdtOp.key is always written, null when unset, because an anti-entropy op’s addressing is part of its merge identity (§ Anti-entropy wire format) — so every decoder is already obliged to read key: null as absent one field over. Every binding that got NodeSnapshot/NodeAdd wrong had CrdtOp right, in the same file.

Audited (#lzkeynullstrict) — the field appears on NodeSnapshot and on the NodeAdd delta op:

BindingBefore the audit
lazily-rs, lazily-go, lazily-js, lazily-dart, lazily-cs, lazily-cppalready lenient
lazily-pyrefused"key" in d was true for the null form, so None reached NodeKey.from_wire and raised
lazily-zigrefusederror.ExpectedString on the JSON null
lazily-ktsilently wrongJsonNull is a JsonPrimitive and its content is the string "null", so the node decoded with a real wire-stable key literally named null, which then re-encoded as "key": "null"

All three are fixed. lazily-kt’s is the reason the clause says MUST NOT construct a key from it rather than only MUST NOT refuse: a decoder that invents an entry address is worse than one that fails loudly, and a refusal-only rule would have called it conforming.

conformance/codec/nodekey_null_leniency.json replays the rule in both codecs and on both fields. It pins the decoded key as absent and pins the re-encoded frame: a decoder may read the null form, but an encoder must still emit only the omitted one, so a binding cannot satisfy the clause by round-tripping null straight through.

IpcValue (payload)

A DeltaOp cell payload is carried as an externally-tagged IpcValue:

IpcValue = { "Inline": [u8] }           // inline byte array (JSON array of 0..255)
          | { "SharedBlob": ShmBlobRef } // descriptor into a shared-memory arena

NodeState

A NodeSnapshot / NodeAdd node body is carried as an externally-tagged NodeState:

NodeState = { "Payload": [u8] }          // concrete serialized value bytes
           | { "SharedBlob": ShmBlobRef } // concrete value in shared memory
           | "Opaque"                      // visible node whose value cannot be serialized

Opaque serialized value bytes are owned by the producing language; type-aware decoding is fixed by the stable type_tag carried on the node. Over the JSON codec, bytes are transmitted as JSON arrays of integers in 0..255 (not base64). Under the negotiated json-base64 feature (#lzspecbase64, see § Capability Negotiation), a peer MAY instead encode Inline/Payload byte arrays as a base64 string, cutting ~4× wire bloat and ~3× parse cost; json-u8 stays the canonical fixture form.

type_tag

Each serializable node carries a type_tag: &'static str — a stable cross-process key that maps to a language-local deserialization constructor. The type-tag registry is per-implementation; tags must not collide across nodes.

Batch string-intern table (#lzspecintern)

A Snapshot, Delta, or CrdtSync batch that repeats the same type_tag string or the same NodeKey prefix across many nodes pays a per-node string cost that dominates wire size at scale. A batch MAY carry an optional sidecar intern table that assigns small integer ids to repeated strings; nodes then reference the id instead of inlining the string.

InternTable = { strings: [string] }   # index → string; position is the id
  • An empty or absent intern field is the default and means “all strings are inlined” — existing decoders are unaffected (additive, backward-compatible).
  • A sender populates intern.strings with the deduplicated type_tag values (and, opt-in, repeated NodeKey namespace prefixes) appearing in the batch, then writes the integer id in each node’s tagged field. The reference is scoped to the batch and does not persist across frames.
  • A receiver resolves ids against intern.strings; an id with no entry is a decode error (fail closed).
  • The intern table is a pure wire optimization: the decoded IpcMessage is identical whether or not interning was used. Conformance fixtures exercise both the interned and inlined forms.

IPC: Snapshot + Incremental Update Protocol

lazily-IPC transmits a reactive graph’s state to a remote observer and keeps it in sync as the graph mutates.

Two message kinds

  • Snapshot — full graph state. Sent on connect and on resync.
  • Delta — incremental change set. Sent once per outermost batch-flush invalidation pass.

Epoch / versioning

A context-level monotonic ipc_epoch: u64 advances once per outermost batch flush, not per write.

  • Snapshot carries epoch.
  • Each Delta carries { base_epoch, epoch } with epoch >= base_epoch + 1. The common single-flush case is epoch == base_epoch + 1; a multi-epoch-span delta (epoch > base_epoch + 1) coalesces several accepted-event epochs into one op batch (see § Multi-epoch-span delta). The span epoch - base_epoch is the count of accepted (deduped) events this batch advances past — a re-emit that dedups to a no-op adds no span. epoch < base_epoch + 1 (empty or backward) is never valid.
  • Deltas are contiguous by base, not strictly unit-stepped. A receiver detects gaps, reorders, or sender restarts by checking base_epoch == last_epoch: a delta whose base_epoch != last_epoch is a gap and triggers resync (§ Reliable Sync) regardless of its span.

Snapshot

{
  "Snapshot": {
    "epoch": 1,
    "nodes": [
      { "node": 1, "type_tag": "i32", "state": { "Payload": [1, 2, 3, 4] } }
    ],
    "edges": [
      { "dependent": 2, "dependency": 1 }
    ],
    "roots": [1]
  }
}
FieldTypeDescription
epochu64Current IPC epoch
nodesNodeSnapshot[]All serialized nodes
edgesEdgeSnapshot[]Dependency edges (dependent → dependency)
rootsNodeId[]Cell and source slot ids

NodeSnapshot

{ "node": 1, "type_tag": "i32", "state": { "Payload": [1, 2, 3, 4] } }
FieldTypeDescription
nodeNodeId (u64)Wire-stable node identifier
type_tagstringStable cross-process type key for decoding state
stateNodeState{"Payload":[u8]} | {"SharedBlob":ShmBlobRef} | "Opaque"
keyNodeKey?Optional wire-stable keyed address; omitted in JSON/MessagePack when absent

EdgeSnapshot

{ "dependent": 1, "dependency": 0 }

Delta

{
  "Delta": {
    "base_epoch": 40,
    "epoch": 41,
    "ops": [
      { "CellSet":    { "node": 1, "payload": { "Inline": [10] } } },
      { "SlotValue":  { "node": 2, "payload": { "Inline": [20] } } },
      { "Invalidate": { "node": 3 } },
      { "NodeAdd":    { "node": 4, "type_tag": "u64", "state": { "Payload": [64] } } },
      { "NodeRemove": { "node": 5 } },
      { "EdgeAdd":    { "dependent": 2, "dependency": 1 } },
      { "EdgeRemove": { "dependent": 3, "dependency": 1 } }
    ]
  }
}
FieldTypeDescription
base_epochu64Epoch this delta applies to (must equal the receiver’s last_epoch)
epochu64New epoch, >= base_epoch + 1; epoch - base_epoch is the accepted-event span (usually 1, > 1 for a multi-epoch-span delta)
opsDeltaOp[]Ordered operations; applied as an ordered fold, atomically advancing last_epoch from base_epoch to epoch

DeltaOp variants

All DeltaOp, IpcValue, NodeState, and IpcMessage variants are externally tagged: a single-key JSON object whose key is the PascalCase variant name and whose value is the body (or a bare "Opaque" / unit string).

OpBody fieldsDescription
CellSetnode, payload: IpcValueChanged-value cell write (PartialEq-guarded)
SlotValuenode, payload: IpcValueA recompute published a new value
InvalidatenodeDirtied, not yet recomputed (lazy)
NodeAddnode, type_tag, state: NodeState, key: NodeKey?New node (optional wire-stable key, omitted in JSON/MessagePack when absent)
NodeRemovenodeRemoved node (free-list reuse: Remove then Add)
EdgeAdddependent, dependencyNew dependency edge
EdgeRemovedependent, dependencyRemoved dependency edge
QueuePushnode, payload: IpcValueOp-log: append to a QueueCell tail
QueuePopnodeOp-log: remove a QueueCell head (no value; determined by replay)
QueueClosenodeOp-log: close a QueueCell (idempotent, terminal)
QueueCell op-log delta form (#queue-oplog)

A QueueCell reconciles by two complementary wire forms, chosen by plane:

  • Snapshot plane → storage-snapshot form. The full queue state is the storage backend’s snapshot (the reference VecDequeStorage serializes as a FIFO-ordered JSON array — cell-model.md § Wire and snapshot shape). This is the state form.
  • Delta plane → op-log form. Incremental change is the ordered op-log of QueuePush / QueuePop / QueueClose shell ops above, carried in a Delta exactly like any other DeltaOp (same base_epoch/epoch span, gap rule, and batch = fold semantics). This is the op-log form.

Why a queue needs the op-log form and cannot use state-supersede coalescing: a queue’s order and multiplicity are its value, and the consumer’s pop position is receiver-side state a sender snapshot cannot see — so collapsing the unacked suffix to a state snapshot would break exactly-once FIFO consumption. Instead the op-log fuses a maximal run of same-direction QueuePush ops into one multi-epoch batch Delta (order-preserving, lossless; a QueuePop/QueueClose is a fusion boundary). Fusion bounds frame count, not memory — see § Backpressure & outbox coalescing.

SPSC crossing rule. In the default producer→consumer sync the producer’s op-log carries QueuePush/QueueClose; the remote consumer owns consumption, so its QueuePop is local and does not cross. Only a mirror sync (an authoritatively-replicated consumer) also carries QueuePop so mirrors converge on the consumption position. A re-delivered queue op (base_epoch < last_epoch) is Ignored by the ResyncCoordinator, so at-least-once replay neither drops nor double-applies a push or pop.

Consistency invariants

  • PartialEq cell guard: An equal set emits no CellSet and no downstream ops.
  • Memo equality suppression: A dirty memo() that recomputes to an equal value emits no SlotValue and no downstream Invalidate.
  • Coalesced frontier: A dependent reached through many changed cells in one batch appears at most once per delta.
  • Eager Signal values are concrete: A changed eager Signal emits a concrete SlotValue for its backing slot, not a bare Invalidate.

The companion Lean model in formal/lean encodes these IPC transition rules and checks them with lake build.

Eager Signal nodes

A Signal is the eager derived value in the Slot -> Cell -> Signal family. It is not a separate wire type. A Signal is represented by the ordinary backing slot node that stores its materialized value:

  • Snapshot: the backing slot appears as a NodeSnapshot with a concrete payload/shared-blob payload like any other readable slot.
  • Delta: a value change appears as SlotValue for the backing slot’s NodeId. Because the value is recomputed during the invalidation flush, eager Signals do not emit bare Invalidate ops for their own changed value.
  • Memo guard: an eager recompute that yields an equal value suppresses SlotValue and downstream invalidation exactly like a lazy memoized slot.
  • Local puller: the producer-side effect that keeps the Signal eager is local execution state and is not serialized as a graph node.

Consumers therefore need no protocol extension to read eager Signals from a producer. They observe the same permission-filtered Snapshot/Delta state plane and see Signals as slots whose changed values are reliably materialized.

Lazy reconciliation

  • Value-mirror (default): At flush, the sender resolves each invalidated allowlisted slot so the delta carries concrete SlotValues. The receiver holds no compute closures.
  • Mirror-lazy: The sender emits bare Invalidate; the receiver keeps a stale marker. Requires compute closure replication. Deferred to lazily-distributed.

Wire shape. The value-mirror default means an allowlisted dirty slot appears in a flush Delta as a concrete SlotValue, never a bare Invalidate (the latter is the mirror-lazy form). This invariant — and the eager-Signal rule that a changed Signal publishes a SlotValue for its backing slot, not an Invalidate — is pinned by the IPC fixtures delta_sequential.json and delta_shared_blob.json, both of which carry SlotValue ops for resolved slots.

Resync / gap handling

On a Delta whose base_epoch != last_epoch:

  1. Receiver discards the delta.
  2. Receiver requests a fresh Snapshot.
  3. Sender replies with Snapshot { epoch }.
  4. Deltas resume from the new epoch.

This narrative is the informal shape; the normative decision function (inbound frame → Apply / RequestSnapshot / Ignore) is the ResyncCoordinator state machine specified in § Reliable Sync, which also fixes the durable-outbox replay and sync-driver loop that make gap recovery, reconnect backfill, and at-least-once delivery a protocol rather than an each-consumer hand-roll.

Messages are length-prefixed and tagged Snapshot / Delta. The protocol is transport-agnostic (unix socket, pipe, WebSocket, shared memory).

Multi-epoch-span delta

A Delta MAY advance more than one epoch in a single frame: epoch > base_epoch + 1. This models a producer whose epoch is a cumulative count of accepted (deduped) events and who coalesces several such events into one flush (agent-doc’s WireDelta is exactly this: a delta “may span multiple epochs”, epoch = per-document accepted-event count). The op list is still the ordered change set; epoch - base_epoch records how many accepted events the batch folds.

Normative apply semantics:

  • Batch = fold. Applying one Delta { base_epoch, epoch, ops } MUST equal applying the same ops in order as a run of unit deltas that advances last_epoch from base_epoch to epoch. The receiver observes only the endpoints (base_epoch, epoch); intermediate epochs are not separately materialized. Proven equivalent in lazily-formal (ReliableSync.multi_epoch_apply_eq_fold).
  • Atomic advance. The receiver advances last_epoch to epoch only after the whole op list applies; a partial application never leaves last_epoch at an intermediate value.
  • Gap rule unchanged. Acceptance still requires base_epoch == last_epoch; the span does not relax gap detection. A delta with base_epoch != last_epoch is a gap at any span.
  • Idempotent re-emit adds no span. A re-emitted delta that dedups to no accepted change carries epoch == base_epoch worth of new effect and is either omitted or applied as a no-op; it never advances last_epoch.

Wire-compat: the unit-step case (epoch == base_epoch + 1) is the span-1 special case, so every existing Delta fixture remains valid. Conformance: conformance/reliable-sync/multi_epoch_delta.json.

Shared-memory IPC

ShmBlobArena provides the shared-memory payload path (a required layer wherever the platform supports it — see § Shared-memory payload path is required):

  • Arena writes a fixed header before each payload: { generation, epoch, length, checksum }.
  • Readers validate the header before accepting a descriptor.
  • IpcMessage control frames carry ShmBlobRef descriptors instead of embedding large bytes inline.

Shared memory carries large blob payloads; ordinary control transport carries framed IpcMessages. Each process keeps its own local Context / ThreadSafeContext and reconciles via snapshots and deltas. A binding whose platform cannot host a shared-memory arena carries every large payload Inline over the control transport (IPC / WebSocket / WebRTC) — the I/O channel accesses the memory directly, so the peer still receives the bytes without a shared-memory descriptor.

Capability Negotiation

Each non-local session starts with a compatibility handshake:

{
  "protocol_id": "lazily-ipc",
  "protocol_major_version": 1,
  "codec": "json",
  "max_frame_size": 1048576,
  "fragmentation_supported": false,
  "ordered_reliable": true,
  "peer_id": 1,
  "session_id": "abc-123",
  "features": ["shared-blob", "signaling-relay"]
}
FieldDescription
protocol_idMust be "lazily-ipc"
protocol_major_versionBreaking change indicator
codec"json", "msgpack" (cross-language binary default), or "postcard" (Rust/same-schema fast path) — see § Frame codecs
max_frame_sizeMaximum unfragmented frame this endpoint can receive, in bytes
fragmentation_supportedWhether this endpoint can send and reassemble fragmented frames
ordered_reliableDelivery guarantee requirement
peer_idPeerId for this session
session_idShared non-empty session/graph identifier
featuresSupported feature flags

max_frame_size is a positive receive ceiling, not an equality constraint. Peers with different ceilings remain compatible and negotiate min(local.max_frame_size, remote.max_frame_size). Both directions use that common ceiling; a binding MUST retain the negotiated value rather than continuing with its local advertisement. A zero ceiling is invalid and fails closed.

Fragmentation is available to the session only when both peers advertise fragmentation_supported = true. A true/false pair remains compatible but negotiates false; treating one peer’s support as sufficient would ask the other peer to reassemble a frame it said it cannot accept.

session_id names the shared graph/session, while peer_id names one endpoint. The two handshakes MUST carry the same non-empty session_id; their peer_id values may and normally do differ.

If peers disagree on protocol_major_version, codec, ordered_reliable, session_id, or required features, or if either max_frame_size is zero, they fail closed before applying any Snapshot or Delta.

Feature flags. The features array advertises optional capabilities both peers must offer to use. Defined flags:

FlagEffect when both peers advertise it
shared-blobLarge payloads travel as SharedBlob descriptors into a shared-memory arena (§ zero-copy transport).
signaling-relayA signaling relay may mediate peer discovery (§ signaling).
command-plane-v1The command/RPC message plane is active (§ message-passing).
json-base64 (#lzspecbase64)Over the json codec, Inline/Payload byte arrays MAY be encoded as a base64 string instead of a JSON array of 0..255 integers (~4× smaller, ~3× faster to parse). The array form (json-u8) remains the canonical form for conformance fixtures; a decoder that advertises json-base64 MUST accept both forms.
protobuf-graph-boundary-v1Peers may exchange the generated, versioned graph-boundary algebra described in Protobuf graph-boundary interoperability.

protobuf-graph-boundary-v1 is an additive typed boundary family, not another encoding of IpcMessage and not a second graph runtime. Its ProtocolEnvelope carries stable identity, causality, bounded mutations, projections, effect intents, and receipts. Canonical JSON remains the fixture and diagnostic authority, and logical identity hashes canonical values rather than Protobuf bytes.

Frame codecs

Every serialized IpcMessage frame (Snapshot, Delta, CrdtSync) carries the same logical wire schema; the codec handshake field selects only how those fields are bytes-encoded. Three codecs are defined; a binding advertises the set it can encode/decode and the peers negotiate one.

Two senses of “canonical” — read this first. This spec keeps two terms distinct:

  • Reference codec = a role: the required, dependency-free, human-inspectable encoding that every binding MUST speak, that the FFI baseline re-encodes to, and that conformance fixtures / logs / receipts are written in. json is the reference codec. It is chosen for this role because it is universal (a parser in every stdlib), inspectable, and deterministic — not because it is efficient (it is the least efficient codec).
  • Canonical bytes = a property of one codec+message: the single deterministic byte string a given codec produces for a given IpcMessage. json and positional postcard are byte-canonical (one byte form per message per codec). msgpack named-field maps are not byte-canonical across encoders (map key order is encoder-defined), so msgpack is the reference-compatible efficient transport, never the reference codec.

These are orthogonal: “reference codec” answers which encoding is the required interop floor; “canonical bytes” answers does this codec produce one deterministic byte form.

Codec tokenSelf-describingRoleRequired of a binding
jsonyesThe reference codec: the required, dependency-free, human-inspectable interop floor — what the FFI baseline re-encodes to (§ FFI Boundary) and what fixtures/logs/receipts are written in. Byte-canonical. Least efficient (blob bytes travel as arrays of integers 0..255).MUST
msgpackyesThe negotiated cross-language binary default on any binary boundary (IPC / WebSocket / WebRTC) between differing languages. Compact binary, self-describing, evolution-safe — the portable efficient transport every binding can host. Reference-JSON-compatible field names, but not byte-canonical across encoders.MUST
postcardnoA compact, positional Rust/same-schema fast path for two peers that share the exact Rust struct layout. Smallest and fastest on the wire, byte-canonical, but not cross-language.MAY

Default selection. A local, same-process, same-language pairing MAY default to postcard. A boundary that crosses languages (the plugin⇄controller boundary, browser peers) negotiates msgpack; json is the interoperable fallback and the required reference codec (also the FFI baseline form). Peers never apply a frame under a codec they did not negotiate.

MessagePack encoding is named-field. msgpack frames encode each struct as a MessagePack map keyed by the JSON field name (Rust: rmp_serde::to_vec_named), not as a positional array. This keeps the field names identical to the json schema, so the omit-when-absent rule for optional fields (the nullable key on NodeSnapshot/NodeAdd/CrdtOp, § NodeKey) holds uniformly across json and msgpack, and a decoder that predates a later optional field ignores it. Positional postcard instead always carries the optional discriminant for binary schema stability.

Conformance is semantic round-trip, not byte-identical, for the map codecs. Because a MessagePack map’s key order is encoder-defined, two conforming bindings MAY emit byte-different msgpack frames for the same IpcMessage; conformance requires only that decode(encode(m)) == m and that decoding a peer’s frame yields the equal IpcMessage. Only postcard (positional) is byte-canonical across encoders. Cross-language msgpack fixtures therefore pin the decoded value, never a golden byte string.

The codec requirement is executable, not prose (#lzmsgpackparity). Stating “every frame MUST round-trip through both json and msgpack” in this document proved nothing: the conformance ladder verifies fixture content replay (was the file opened, were its keys consumed, were they asserted, were all its scenarios replayed), and content replay never exercises a codec. A binding could therefore carve out a MUST-level codec and stay green everywhere. Two canonical fixtures now carry the obligation:

FixtureCodecObligation
conformance/codec/frame_roundtrip_json.jsonjsonMUST — the reference codec; every binding replays it
conformance/codec/frame_roundtrip_msgpack.jsonmsgpackMUST — a binding that has not implemented it declares the gap in its conformance-coverage ledger

Both carry the same three wire values (one per IpcMessage variant) so one runner shape serves both codecs. A runner MUST decode wire, re-encode the decoded message, decode it again, and evaluate every expect key against that second decode — asserting against the fixture literal proves nothing, because the literal never passed through the codec. The msgpack fixture additionally pins encoded_envelope_key and the sorted encoded_body_field_names, which is the executable form of the named-field rule above: a positional encoder round-trips a value correctly and fails those keys. postcard carries no cross-language fixture — it is MAY, and by construction only two peers sharing a struct layout can speak it.

A binding that does not implement msgpack is not silently exempt. Its scripts/check-conformance-coverage.sh sees the canonical fixture and fails until the binding either replays it or names it in KNOWN_UNCOVERED with a reason — the same ledger every other declared gap lives in — and coverage.json carries one row per codec flavor so the matrix shows the carve-out beside every other parity fact.

Shipping a MessagePack codec is not implementing msgpack. The codec token names one wire, not a serialization technology: the externally-tagged frame ({"Snapshot": {…}}) over named-field maps whose keys are the json field names, with the same omit-when-absent rule for optional fields. A codec that packs the same data as an internally-tagged envelope ({"type": 0, "value": …}), gives NodeState/IpcValue integer kind discriminators instead of the Payload/Inline external tags, or uses positional arrays, is a private codec that happens to use MessagePack framing — a peer that negotiated msgpack with it would not decode its frames. That distinction is invisible to a file-presence audit and is exactly what the round-trip fixture makes checkable, which is why the matrix distinguishes from ~ on this row.

Causal Receipts

Some integrations send a command or publish an effectful request and need a durable, queryable outcome for that causation id. lazily supplies a generic causal receipt primitive for that use case; it is not a transport ACK and does not make delivery success authoritative.

{
  "CausalReceipts": {
    "receipts": [
      {
        "receipt_id": "receipt-1",
        "causation_id": "patch-123",
        "observer": "editor",
        "generation": 7,
        "outcome": "applied",
        "reason": null,
        "payload_hash": "sha256:2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"
      }
    ]
  }
}
FieldDescription
receipt_idIdempotency key for this receipt event. Duplicate receipt_ids are no-ops.
causation_idStable id of the command, event, or effect request the receipt observes.
observerPeer, process, or subsystem that produced the receipt.
generationMonotonic producer/editor generation. Consumers discard receipts whose generation does not match the current authority generation for the causation id.
outcome"observed", "accepted", "applied", or "rejected".
reasonOptional human/debug rejection reason; null when absent.
payload_hashOptional hash of the state/payload the receipt observed; null when absent.

Receipt projection rules:

  • observed and accepted are non-terminal. They may model ACK-like transport or queue admission observations, but a domain MUST NOT treat them as proof that an effect happened.
  • applied and rejected are terminal. They are the generic outcome vocabulary that domain-specific facts refine (for example an editor may publish EditorPatchApplied / EditorPatchRejected facts keyed by the same causation_id).
  • A stale-generation receipt is ignored by the current projection and may be retained only as audit/debug data.
  • A second terminal receipt for the same causation_id and generation with a different terminal outcome is a terminal conflict; consumers fail closed instead of selecting a winner.
  • The primitive is state/projection data. Transports may still provide their own delivery acknowledgements internally, but lazily does not expose delivery ACKs as authority.

Command / RPC Message Plane

Editor and runtime integrations issue commandsRun Agent Doc, sync, focus, save, session operations — and need one reusable admission, dedupe, cancellation, generation-guard, progress, and reconnect story instead of a per-caller ad hoc request/response contract. lazily supplies an evented command message plane for that use case. It is an additive sibling family to Snapshot / Delta / CrdtSync; it does not add new state-plane variants.

The plane is feature-gated. Peers advertise command-plane-v1 in the Capability Negotiation features array. A peer that lacks command-plane-v1 fails closed before accepting command traffic; a command that requires the plane is not silently downgraded.

Four externally-tagged frames make up the family:

CommandSubmit

{
  "CommandSubmit": {
    "command_id": "cmd-run-1",
    "causation_id": "cmd-run-1",
    "source": "vscode-plugin",
    "target": "project-controller",
    "namespace": "agent-doc",
    "name": "editor_route",
    "authority_generation": 42,
    "idempotency_key": "project-root:plan.md:run",
    "deadline_ms": 120000,
    "policy": { "dedupe": "same_idempotency_key", "supersede": false, "cancel_on_preempt": true },
    "payload_type": "agent-doc.editor_route.v1",
    "payload_hash": "sha256:…",
    "payload": { "Inline": [123, 34, 102, 34, 58, 34, 46, 34, 125] },
    "required_features": ["causal-receipts", "command-events"]
  }
}

lazily owns the envelope (command_id, correlation, idempotency, generation, policy, payload framing). The namespace owns the payload: lazily never interprets the payload body, which is a normal IpcValue (inline bytes or a shared-memory blob). payload_type and payload_hash identify and pin the domain body.

CommandCancel

CommandCancel preempts a still-non-terminal command by command_id at a given authority_generation, with an optional reason. A stale-generation cancel is ignored. A cancel after a terminal outcome never rewrites it.

CommandEvents

CommandEvents batches progress/detail events keyed by command_id. Event kinds are observed, accepted, started, progress, cancelled, superseded, timed_out. Events are UX and diagnostics only — queue position, retry advice, copied CLI output. They are never terminal proof. Even cancelled / superseded / timed_out events are surfaced for UX; their terminal authority is a matching rejected causal receipt.

CommandProjection

CommandProjection is the folded, queryable image of known command state: per-command status, an explicit terminal flag, generation, terminal reason, and the terminal_receipt_id / last_event_id that produced it. It is also the reconnect resync frame: after a controller handoff/recycle a plugin folds a fresh CommandProjection and recovers in-flight and terminal state without replaying the underlying events.

Projection rules

  • Terminal authority is the causal receipt, not the event or the transport. A command becomes terminal (applied / rejected / cancelled / superseded / timed_out) only when a terminal CausalReceipt for its command_id folds in. observed / accepted / started / queued admission are non-terminal progress. A network ACK is never terminal.
  • Generation guards. Events and receipts whose generation does not match the command’s current authority generation are ignored by the projection and retained only as audit data.
  • Idempotency. A replayed CommandSubmit, event, or receipt (same command_id / event_id / receipt_id) is a no-op; the projection is unchanged.
  • Cancel before terminal only. A cancel terminally rejects a non-terminal command; a cancel after applied is ignored.
  • Terminal conflict fails closed. Two terminal receipts at the same generation with different outcomes is a conflict; consumers fail closed rather than pick a winner (the same rule as Causal Receipts).
  • Reconnect equivalence. Folding a CommandProjection image is equivalent to folding the events and receipts it summarizes.

RPC facade

Bindings expose an RPC-style API (call / submit / cancel / observe / projection) implemented entirely over these frames:

  • call builds and sends a CommandSubmit, observes events and receipts, and resolves only when the command projection reaches a terminal causal receipt. A transport ACK, controller admission, or accepted / queued event never resolves a unary call.
  • submit returns the command_id immediately for callers that manage events and projection themselves.
  • cancel sends CommandCancel and returns the resulting projection.
  • observe / stream exposes CommandEvents and projection updates for UI progress.
  • Reconnect uses CommandProjection; callers replay a call only when the idempotency policy says replay is safe.
  • The terminal-result error shape exposes the terminal command projection (status, reason), not a collapsed boolean.

Unary RPC completion means command effect completion, proven by a terminal causal receipt — not network delivery. The schema is schemas/message-passing.json; fixtures live in conformance/message-passing/.

Cross-language Channel Compatibility

All channels carry the same IpcMessage state plane:

ChannelStrategy
FFIC ABI: opaque context/session handles + owned byte buffers
IPCUnix sockets, pipes, local TCP: length-prefixed serialized IpcMessage
WebSocketOne WebSocket frame = one serialized IpcMessage
WebRTC dataReliable ordered data channels carry serialized IpcMessage

Cross-language rules

  • Compute closures are language-local. Cross-language sync shares the cell state plane.
  • Permission filtering happens before serialization on every channel.
  • Back-pressure: if frames gap, reorder, or truncate, the receiver requests a fresh Snapshot.

Binding Conformance Matrix

A lazily binding is a language port that intends to interoperate with the wider lazily ecosystem (lazily-rs, lazily-py, lazily-zig, lazily-js, lazily-kt, lazily-dart, lazily-go, …). The layers below are required of every binding; none is an optional lazily-rs extension. A binding that omits a MUST row is non-conforming and MUST advertise its missing surface via Capability Negotiation rather than failing silently.

LayerRequiredSpecConformance
Reactive core (Cell / Slot / Effect / Signal)MUSTReactive Graph, Cell Model
Keyed cell collections (SourceMap, SourceTree, keyed reconciliation)MUSTCell Model § Keyed cell collectionsconformance/collections/
Flat state machineMUSTState Machine
Harel state chartsMUSTState Chartsconformance/statechart/
Thread-safe reactive contextMUST²Reactive Graph § Context layers
Async reactive contextMUST²Async Reactive Context
IPC (Snapshot + Delta)MUST§ IPCconformance/ IPC fixtures
Frame codecs (json reference + msgpack cross-language binary; postcard optional)MUST§ Frame codecsconformance/codec/frame_roundtrip_json.json and conformance/codec/frame_roundtrip_msgpack.json — each carries one scenario per IpcMessage variant (Snapshot/Delta/CrdtSync), replayed through the codec rather than read as data
Shared-memory payload path (ShmBlobArena / ShmBlobRef)MUST³§ Shared-memory IPCconformance/ shared-blob fixtures (snapshot_shared_blob, delta_shared_blob)
C-ABI FFI boundary (LazilyFfiBytes, LazilyFfiStatus, LazilyFfiMessageKind)MUST¹§ FFI Boundary, ffi.jsonevery binding decodes the FFI frame to IpcMessage and re-encodes canonical JSON bytes
Distributed CRDT plane (CrdtSync / WireStamp)MUST§ Distributed: CRDT Cell Plane, distributed.jsonconformance/ CrdtSync round-trip
Causal receipts (CausalReceipt, terminal outcome projection)MUST§ Causal Receipts, receipts.jsonconformance/receipts/causal_receipts.json
Permission boundary (RemoteOp / PeerPermissions)MUST§ Permission Boundary
Capability negotiationMUST§ Capability Negotiation
Signaling (WebSocket)MAY§ Signalingonly for bindings that bridge browser/runtime peers
WebRTC data transportMAY§ Cross-language channelsonly for bindings that peer over WebRTC

¹ C-ABI FFI has a platform carve-out — see § C-ABI FFI is required. ² The thread-safe and async context layers have a platform carve-out — see § Concurrency layers are required. ³ The shared-memory payload path has a platform carve-out (I/O-channel fallback) — see § Shared-memory payload path is required. CRDT and the keyed cell collections have no carve-out: they are wire/logic properties implementable on any runtime that speaks the wire.

C-ABI FFI is required

Every binding whose platform can host a native in-process boundary MUST expose and consume the C-ABI FFI boundary: the LazilyFfiBytes / LazilyFfiStatus / LazilyFfiMessageKind contract with explicit allocation ownership, panics caught before crossing the C ABI, and a channel that decodes each accepted frame as IpcMessage and re-encodes canonical JSON bytes. The FFI message kind discriminant MUST include CrdtSync = 3 and the reliable-sync control frames ResyncRequest = 4 / OutboxAck = 5 (§ Reliable Sync). This is the lingua franca that lets any binding embed any other without a language-specific bridge.

Platform carve-out. A binding whose runtime structurally cannot host a native C ABI declares the ffi capability as none; otherwise it declares host. none is reserved for platforms with no shared in-process address space (for example browser/Worker JS, or a fully-sandboxed runtime) — a binding MAY NOT declare none merely because FFI is inconvenient or unimplemented. This is a binding-level conformance declaration (advertised in the binding’s conformance statement and discoverable at build/link time), not a per-session wire flag, since in-process embedding is not a runtime session.

A ffi = none binding:

  • conforms to the interop contract but not the in-process embedding contract, and MUST NOT advertise itself as embeddable;
  • MUST still expose the full state plane — including CrdtSync — over IPC/WebSocket/WebRTC, so it interoperate without in-process embedding; and
  • MUST be treated by peers/host tooling as unable to be loaded in-process (fail closed on any attempt to do so, rather than silently degrading).

This reuses the existing fail-closed principle: the limitation is explicit and advertised, never silent. There is no equivalent carve-out for CRDT or the cell/collections model — those are implementable on any Turing-complete runtime that speaks the wire.

CRDT is required

Every binding MUST implement the CrdtSync plane — the merge: crdt mechanism, the WireStamp version-vector frontier, and the causal-stability watermark / GC contract — and MUST round-trip a CrdtSync IpcMessage byte-identically. Multi-write convergence is a property of the cell model, not an optional distributed extra; a binding that ships only the single-producer Snapshot/Delta mirror conforms only to the single-writer subset and MUST downgrade its advertised capability accordingly.

Concurrency layers are required

The reactive core ships as three context layers — single-threaded (base), thread-safe (lock-backed), and async (future-returning) — defined in Reactive Graph § Context layers and Async Reactive Context. The single-threaded base context is unconditionally required of every binding (it is the reactive-core row above). The thread-safe and async layers are required conditionally: a binding whose platform structurally supports a layer MUST implement that layer.

Thread-safe context. A binding whose platform exposes preemptive multi-threading or shared-memory concurrency (native threads, OS goroutines, JVM threads, Kotlin coroutines over a shared heap, etc.) MUST ship the lock-backed context whose handles are clonable and whose transition function and state are Send + Sync, so observers fire synchronously within the invalidating send/batch preserving glitch-free pull-based ordering. “Synchronously within” mandates glitch-free ordering, not literal in-lock dispatch — a threaded binding MAY defer observer dispatch out of the graph lock provided the ordering invariant holds (#lzspecobserverclarify). A platform with no shared-memory threading model — a strictly single-threaded runtime, or a process/actor-isolation model (e.g. a Dart isolate, a browser Worker) where peers do not share an address space — declares the thread_safe capability as none.

Async context. A binding whose platform exposes an async/future runtime (async/await, promises, coroutines, an executor that suspends and resumes across .await points) MUST ship the async context with its full slot state machine (Empty / Computing / Resolved / Error), revision tracking, five-point cancellation contract, and get_async re-resolve loop. A platform with no notion of suspendable async computation declares the async capability as none.

A none declaration, for either layer:

  • is a binding-level conformance declaration (advertised in the binding’s conformance statement and discoverable at build time), not a per-session wire flag, because in-process concurrency structure is not a runtime session;
  • MUST be reserved for platforms that structurally lack the primitive — a binding MAY NOT declare none merely because the layer is inconvenient or unimplemented; and
  • MUST be advertised rather than fail silently, reusing the existing fail-closed principle.

There is no carve-out for the keyed cell collections, the state machine, the state charts, the reactive core, or CRDT — those are implementable on any Turing-complete runtime that speaks the wire, single-threaded or not.

Shared-memory payload path is required

The shared-memory payload path (ShmBlobArena + ShmBlobRef descriptors) is the zero-copy large-payload transport for IpcValue and NodeState. A binding whose platform exposes a shared-memory primitive (POSIX shm / memory-mapped files / OS shared memory / a peer-reachable arena) MUST implement it: payloads above the inline threshold are written into the arena, the control frame carries the ShmBlobRef descriptor, and readers validate the { generation, epoch, length, checksum } header before accepting it.

Platform carve-out — I/O-channel fallback. A binding whose runtime structurally cannot host a shared-memory arena (browser/Worker JS, a sandboxed runtime, WASM without shared memory) declares the shared_memory capability as none. Such a binding MUST fall back to I/O channels accessing the memory: every payload that would have been a SharedBlob descriptor is instead carried Inline over the ordinary IPC / WebSocket / WebRTC transport, so the same IpcMessage state plane reaches the peer without a shared-memory descriptor. The wire format already supports both paths (IpcValue and NodeState are externally-tagged Inline | SharedBlob); the fallback simply never emits the SharedBlob variant on that binding and treats an absent descriptor as Inline on read.

Shared memory is negotiated per session via the shared-blob entry in the Capability Negotiation features array: two peers that both advertise shared-blob MAY exchange ShmBlobRef descriptors; if either peer omits it, both sides carry payloads Inline over the control transport for that session. (The thread_safe, async, and ffi capabilities, by contrast, are binding-level declarations — in-process properties — and are not per-session wire flags.)

This reuses the existing fail-closed principle: the limitation is explicit and advertised, never silent. The shared-blob conformance fixtures (snapshot_shared_blob.json / delta_shared_blob.json) fix the descriptor shape for bindings that ship the arena; an Inline-only binding round-trips the same fixtures with the blob bytes inlined.

The backend discriminator: absence is lenient, an unknown value is not (#lzblobbackendstrict)

ShmBlobRef.backend selects which pluggable backend resolves a descriptor (shm | arrow | in_process, optional, default shm — see docs/zero-copy-transport.md). The two ways a decoder can fail to recognise it are not the same fact and MUST get opposite answers:

  • An OMITTED backend MUST decode as shm. This is the forward-compatibility channel, and the only one. Every descriptor minted before the field existed has this shape, which is exactly why the field is optional. A conforming encoder MUST also omit it when the backend is shm, so a pre-field descriptor round-trips byte-identically — the same encoder/decoder split § NodeKey makes for key.
  • A PRESENT backend outside the enum MUST be rejected, with the offending token named in the error. A decoder MUST NOT normalize it to shm, to any other backend, or to a sentinel.
  • An explicit backend: null is the ABSENT form, not a present-unknown one, and MUST decode as shm. This follows § NodeKey rather than the bullet above, and for the same reason: a serde-style peer that did not apply skip_serializing_if to an optional field emits null where a conforming encoder omits, so refusing it is stricter than the reference implementation on a frame the reference implementation produces. Four bindings raised this edge independently while implementing the clause, and they had already split three ways on it — accept-as-shm, error.ExpectedString naming nothing, and a refusal naming the token ''.
  • shm is the permanent default. A future revision MUST NOT redefine which backend an omitted backend denotes. The omitted form is the only shape a pre-field descriptor can have, so changing its meaning would silently reinterpret every descriptor ever written rather than break them visibly — the one genuinely undecidable case on this wire, and cheaper to pin as a sentence than to discover as an incident.

Two obligations on the refusal itself.

It must be catchable through the codec’s documented decode-error type. A refusal raised as a type outside the family every caller already guards a decode with is invisible: the frame still fails, but it fails past the handler. In C++ that means std::runtime_error and specifically not std::invalid_argument, which derives from std::logic_error — the identical hierarchy trap that let a std::out_of_range from std::stoll escape every decode guard until the NodeId bound was audited. The general rule: a decoder’s refusals belong in one catchable family, and this clause adds no exception to it.

Naming the token is bounded. A binding MAY truncate the offending token in its error (64 bytes is sufficient), and a truncated name satisfies the obligation. The refusal path is the path a hostile or corrupt producer controls, so an unbounded “echo the token back” requirement is an allocation and log-flood vector; a truncated name still identifies the producer, where an absent one does not.

The cost this accepts, stated plainly. A refusal fails the whole frame, not just the descriptor. So a fourth backend added later is a hard break for every older peer rather than a degraded read, and at the decoder a corrupt frame and a future-version frame are indistinguishable — nothing on the frame carries a protocol version, only the session-level shared-blob capability flag. That is the deliberate trade: a visible break a peer recovers from by resync, in exchange for never resolving a descriptor against a backend its producer did not name. The alternative buys graceful degradation with a guarantee that holds only until a checksum collides.

The asymmetry is not stylistic. A new backend enters the protocol by adding an enum value — a spec change with a fixture, not a wire event (§ Pluggable backends in the zero-copy transport doc) — so an unknown token is not a newer peer talking to an older one. It is a corrupt frame or a non-conforming producer, and both are conditions a peer recovers from by resync (§ Resync / gap handling).

Normalizing it is worse than it looks, and the reason is a theorem the model already proves. resolve_wrong_backend states that a descriptor of one kind never resolves against a different backend’s table — receivers route by kind. Reading an unknown kind as shm is routing a non-shm descriptor into the shm table; the {generation, epoch, len, checksum} verification then usually rejects it, so the observable outcome is right most of the time. That is the trap: the guarantee is supposed to be structural, discharged by routing, and normalization silently downgrades it to a probabilistic one discharged by a 64-bit checksum, against a backend this build genuinely resolves. It is the same substitution the decoder bound already outlaws for NodeId in § NodeId / PeerId — the frame decodes cleanly, addresses something other than what the producer named, and nothing downstream can tell.

Audited (#lzblobbackendstrict) across the library decode surface of all nine bindings, which had split 5–2 — and every one of the five documented its choice as deliberate wire forward-compat:

BindingBefore the audit
lazily-rs, lazily-jsrejected — the conforming behaviour, reached independently
lazily-go, lazily-py, lazily-kt, lazily-zig, lazily-cppnormalized to shm, each with a written forward-compat rationale
lazily-cs, lazily-dartno dispatch site — the descriptor decodes through a field-typed path

The lazily-cs row is wrong and is left standing as a record of the error. Replaying fixture v2 found a real dispatch site there — IpcWire.ReadBlob on decode, IpcWire.WriteBlob on encode, and BlobTransport.EffectiveBackend at every routing site. “No dispatch site” was a review conclusion, and it is the kind of conclusion a replay overturns: an audit that reads for a switch finds nothing when the discriminator is read through a field-typed path that dispatches anyway.

The five agreed with each other on the defence, too: the checksum catches it, so the descriptor resolves to nothing rather than to another backend’s bytes. That argument is what the theorem exists to make unnecessary. An undocumented default and a deliberate one are indistinguishable from the outside — and, it turns out, so are two deliberate ones pointing in opposite directions. conformance/codec/blob_backend_discriminator.json replays both halves in both codecs. It carries an arrow scenario so the leniency cannot be implemented by ignoring the discriminator outright, and pins the re-encoded frame so a binding cannot satisfy the clause by round-tripping whatever it received. The encoder half is a separate obligation and is not implied by the decoder half: lazily-rs has carried skip_serializing_if since the commit that introduced the field, while a binding can implement strict rejection and still emit backend: "shm" — which is why the re-encode assertions exist rather than being inferred.

Fixture v2 (#lzblobbackendstrict) closes four holes v1 left. Every one of them was found by a binding replaying v1, not by reviewing it — which is the argument for replaying a fixture in nine places before trusting it. The fixture is now 14 scenarios, seven wire shapes in both codecs:

  • in_process is carried, not merely declared. v1 listed three backends in assertions.backends and shipped scenarios for two, so a binding that knew only {shm, arrow} rejected in_process — naming the token, conformingly, by the letter of this clause — and passed all eight scenarios while implementing a smaller enum than the clause declares. Found independently by three bindings. The arrow control proves the discriminator is read; in_process proves the vocabulary is complete, and no scenario count substitutes for it: the missing fact was a set difference, so the corpus-side guard is now a set difference (test_backend_fixture_carries_every_backend_it_declares).
  • The null bullet above is executed, not just written. Four bindings raised it independently while implementing v1 and had already split three ways. It is an accept scenario that is deliberately schema-invalid: the string-typed enum binds the ENCODER, and the decoder’s leniency is the separate fact under test. The re-encode assertion still applies, so the null does not survive a round trip.
  • A non-string backend is a scenario. The clause is written entirely about tokens, so a runtime whose reader coerces rather than throws on a number in a string position normalizes silently there — the same failure this clause names, arriving through a door it does not describe. One binding’s only real defect under v1 was exactly this. Both reject forms now assert expect.rejection_is_decode_error, which is the It must be catchable obligation above made executable; expect.rejection_kind distinguishes them, and only unknown_token carries error_names_token, because a type error has no token to name and requiring the field name would pin a message format no codec’s native type error produces.
  • expect.epoch is gone, replaced by frame_epoch (9) and blob_epoch (5). v1 carried 9 in both the Delta frame and the ShmBlobRef descriptor, so a runner reading the frame’s epoch and one reading the descriptor’s both passed a single assertion. Found by two bindings. The key was removed rather than redefined so a runner still reading it fails loudly instead of silently reading the other one.

Two codecs are still not two implementations. Several bindings bridge MessagePack into the same DOM the JSON decoder produces, or share one serde impl, so the msgpack half of a scenario pair can yield one discriminator verdict rather than an independent second one. It still covers the bridge and the encoder; a fully green run must not be read as two implementations agreeing, and a binding whose two codecs share a decode path should record that in its own ledger rather than infer independence from the scenario count. This one is not fixable in the corpus — it is a property of the bindings — so it stays stated rather than pinned.

What v2 found when the nine bindings replayed it. The four holes above were predictions about what an unexecuted clause hides; this is what was actually behind them:

  • Four of nine bindings REFUSED the explicit null — lazily-rs (#[serde(default)] supplies the default when the key is missing, so a present null still reached BlobBackendKind::Deserialize and failed as a type error in both codecs), lazily-cs (a ValueKind error), lazily-zig (error.ExpectedString), lazily-kt (JsonNull is a JsonPrimitive whose isString is false, so it fell into the non-string arm). Each returned a decode error for a frame the reference implementation emits. The bullet stating the rule had been in this document since the clause landed; it was prose, and four bindings read it and still got it wrong — including lazily-rs, which the row above records as reaching the conforming answer on the unknown token independently. Being right about the hard half of a clause is not evidence of being right about the easy half. That ratio is the argument for making an adjudication executable rather than writing it down.

    The lazily-rs fix carries a detail the other bindings do not face: serde_json’s deserialize_str constructs the invalid_type error itself on a null and never calls the visitor, so reading the null requires deserialize_option — and that forced a branch on is_human_readable(), since postcard writes no option tag for this field and asking it for one would misalign the frame. The strict deserialize_str path stays for non-self-describing codecs.

  • Two bindings had no decode-error family at all. lazily-zig’s decode errors were an inferred error set nobody had written down, and lazily-kt raised IllegalStateException from one refusal and IllegalArgumentException from the other — neither a subtype of the other, so a caller catching the documented one had the other fail past the handler. Both now name the family (ipc.BlobDescriptorDecodeError; sealed class IpcDecodeException). The It must be catchable obligation above was unassertable until rejection_is_decode_error existed, which is why a refusal being wrong in this specific way survived a nine-binding audit.

  • rejected and rejection_is_decode_error are genuinely two facts. lazily-cpp demonstrated it on demand: throwing std::invalid_argument for the non-string turns rejection_is_decode_error red while rejected stays green. A bare is-error assertion passes the hierarchy trap.

  • A fifth false-green shape, distinct from the four already catalogued. rejection_kind must be asserted against what the LIBRARY raised — a distinct error type, or the message the refusal actually produced. A runner that derives it from the scenario’s own backend_form compares the fixture to itself and passes while proving nothing. Raised by lazily-kt, which made its two refusals distinct types precisely so the assertion has something of the library’s to compare against. The four already on record are: a stale build artifact, a filter matching nothing, a probe aimed at a value the corpus never carries, and a filtered build run reporting N/N passed while selecting the target test not at all.

Two bindings’ green is worth less than it looks, and both say so in their own runners rather than letting the scenario count imply otherwise: lazily-cpp bridges MessagePack into the same JsonValue DOM decode_json produces, and lazily-py bridges into the same DOM as well, so each scenario pair there yields one discriminator verdict. The msgpack half still covers the bridge and the msgpack encoder, which are distinct failures.

FFI Boundary

Types

typedef struct {
    uint8_t* ptr;
    size_t   len;
} LazilyFfiBytes;

typedef enum {
    LazilyFfiStatus_Ok            = 0,
    LazilyFfiStatus_Empty         = 1,
    LazilyFfiStatus_NullPointer   = 2,
    LazilyFfiStatus_InvalidMessage = 3,
    LazilyFfiStatus_EncodeFailed  = 4,
    LazilyFfiStatus_Panic         = 5,
} LazilyFfiStatus;

typedef enum {
    LazilyFfiMessageKind_Unknown       = 0,
    LazilyFfiMessageKind_Snapshot      = 1,
    LazilyFfiMessageKind_Delta         = 2,
    LazilyFfiMessageKind_CrdtSync      = 3,
    LazilyFfiMessageKind_ResyncRequest = 4,  /* #lzsync control frame */
    LazilyFfiMessageKind_OutboxAck     = 5,  /* #lzsync control frame */
} LazilyFfiMessageKind;

Contract

  • All allocation ownership is explicit: caller owns input bytes; Rust owns output buffers until the paired free function is called.
  • Errors return LazilyFfiStatus; panics are caught before crossing the C ABI.
  • The channel decodes each accepted frame as IpcMessage, then re-encodes canonical JSON bytes.

Signaling Protocol (WebSocket)

Client → Server

TypeFieldsDescription
joinpeer, capabilities?Register with session
offerto, sdpWebRTC SDP offer
answerto, sdpWebRTC SDP answer
iceto, candidateICE candidate
relayto, payloadRelay opaque payload
leaveDisconnect

Server → Client

TypeFieldsDescription
welcomepeer, peersRoster on join
peer-joinedpeerNew peer in session
peer-leftpeerPeer disconnected
offerfrom, sdpForwarded offer
answerfrom, sdpForwarded answer
icefrom, candidateForwarded ICE
relayfrom, payloadForwarded payload
errorcode, messageError response

Anti-spoofing

The from field on every forwarded frame is the sender connection’s registered peer id, never client-supplied.

Example frames

{ "type": "join", "peer": 1 }
{ "type": "welcome", "peer": 1, "peers": [] }
{ "type": "offer", "to": 2, "sdp": "v=0\r\n..." }
{ "type": "answer", "from": 2, "sdp": "v=0\r\n..." }
{ "type": "ice", "from": 2, "candidate": "candidate:..." }
{ "type": "relay", "to": 2, "payload": { "any": "json" } }
{ "type": "peer-joined", "peer": 2 }
{ "type": "leave" }

Permission modes

ModeDescription
openAny peer may join and signal any other joined peer
allowlistDefault-deny: peers require explicit grants; directed frames only to allowed targets

Distributed: CRDT Cell Plane

This plane specifies merge: crdt — the first multi-write merge mechanism of the Cell Model. CRDT is one mechanism among several the cell model reserves (lww, ot, lease, custom); it is the first defined because it converges without coordination. Everything here applies to a multi-write cell that declares merge: crdt; the cell-kind classification, ingress-on-roots-only boundary, and cell-as-merge-unit granularity are defined once, mechanism-independently, in the Cell Model.

Cell register types

These are the CRDT-mechanism register types (the value shapes available within merge: crdt); they are distinct from the cell-model’s MergeMechanism axis.

TypeMergeDescription
LWW-registerLast-write-wins (HLC timestamp)Default; “current value” semantics
MV-registerMulti-valueSurfaces concurrent writes as a set
PN-counterAdditivePositive-negative counter

CRDT properties

  • Each replicated cell is keyed by a hybrid logical clock (HLC): wall-clock for human-meaningful ordering, logical counter for causal tiebreak.
  • Local PartialEq invalidation guard applies after merge: equal values invalidate nothing.
  • Memo equality suppression holds post-merge.
  • lazily-ipc’s Delta generalizes to per-peer causal stamps: each peer keeps its own sequence; cross-peer order comes from HLC/dot metadata.

Anti-entropy wire format (CrdtSync)

The plane rides the same lazily-ipc transport as Snapshot/Delta. Alongside the single-producer mirror, a third IpcMessage variant carries multi-writer plane traffic:

# Snapshot/Delta/CrdtSync are the forward (state) plane; ResyncRequest/OutboxAck
# are the reverse-channel reliable-sync control frames (§ Reliable Sync, #lzsync).
IpcMessage = Snapshot(Snapshot) | Delta(Delta) | CrdtSync(CrdtSync)
           | ResyncRequest(ResyncRequest) | OutboxAck(OutboxAck)

WireStamp = { wall_time: u64, logical: u64, peer: u64 }   # total order (wall, logical, peer)

CrdtOp = {
  node:  NodeId,           # volatile target id
  key:   NodeKey?,         # optional wire-stable address (survives NodeId churn, #lzwirekey)
  stamp: WireStamp,        # the HLC stamp that produced this state
  state: IpcValue,         # the converged CRDT state to merge (state-based / CvRDT)
}

CrdtSync = {
  frontier: [(peer: u64, WireStamp)]?,   # the sender's per-peer stamp frontier
  ops:      [CrdtOp],                    # the op batch this frame ships
}

WireStamp is the wire mirror of the runtime HLC stamp (all plain integers), so the wire format is codec-stable whether or not a peer compiles the CRDT runtime in. It round-trips across all three codecs (JSON, MessagePack, postcard) and is classified by the FFI message kind (CrdtSync = 3).

Frontier suppression (#lzspecfrontiersuppress). frontier is optional. A frame that omits frontier (JSON/msgpack) or carries the sentinel frontier: [] means “the sender’s stamp frontier is unchanged since the last frame the receiver accepted” — the receiver reuses its last-merged frontier to compute the watermark. This is a pure wire savings on chatty peers (a frame shipping only ops need not repeat an unchanged frontier). The self-describing codecs (JSON, msgpack) omit the field when unchanged; positional postcard carries a 1-bit discriminant. A sender MUST include frontier whenever its advertised stamp has advanced; a receiver that receives a frontier-less frame with no prior frontier (cold start) MUST request a full CrdtSync (fail closed). Conformance risk is medium: paired fixtures exercise both the suppressed and the full forms.

State-based, idempotent. Each CrdtOp ships the converged register/sequence/text state for a node. The receiver merges state into its local replica; because every cell CRDT merge is commutative, associative, and idempotent (proven in formal/lean/LazilyFormal/CRDT.lean, stampJoin_{comm,assoc,idem}), out-of-order, duplicated, or batched delivery all converge. Re-sending a frame the receiver already has is a no-op.

Stamp-frontier exchange. CrdtSync.frontier advertises the highest WireStamp the sender has observed from each peer. The receiver merges it into its own frontier (per-peer max); the causal-stability watermark is the min over membership of that frontier — the causal point every replica has provably passed.

Watermark / GC contract. A tombstone whose delete stamp is the stability watermark is collectable on every replica, so dropping it cannot lose an edit. This safety property is formally proven (LazilyFormal.CRDT.collectable_implies_observed_everywhere: a collectable stamp is every member’s observation) and drives the runtime SeqCrdt::gc / TextCrdt::gc_with. A single replica’s local clock is explicitly not a sound watermark; only the version-vector minimum is.

GC scheduling (#lzspecgcdefer). The contract above fixes when a tombstone is collectable; it does not mandate when collection runs. A binding MAY defer GC to an idle window, batch it under memory pressure, or run it incrementally (sweep a bounded number of tombstones per anti-entropy round), provided (a) no tombstone below the watermark is re-examined after reclaim and (b) unbounded accrual is reported via the instrumentation surface. Deferring GC changes only memory footprint, never observable values.

Permission filtering. CrdtSync.filter_readable(peer) drops ops for non-readable nodes entirely (omission, not redaction — like Delta). The frontier advertisement is retained in full: it names peers and stamps, not node content, and the receiver needs the whole frontier to compute a sound watermark.

Status. The wire format, codec round-trips, permission filtering, and point-to-point IpcSink/IpcSource delivery are implemented (#lzcrdtplane5a). Wiring the plane to live merge: crdt root cells (local edits → CrdtOps; remote CrdtOps → ReplicatedCell ingress merge) and BridgeHub fan-out of CrdtSync is the runtime-integration slice (#lzcrdtplane5b).

Delta-CRDT sync (#lzspecdeltacrdt). Today each CrdtOp ships the full converged register state per anti-entropy round (state-based / CvRDT). For LWW/MV registers and PN-counters whose state is small this is acceptable, but for OR-set membership and large register values the full-state ship is wasteful when little changed. The watermark + join algebra already exist (formal/lean/LazilyFormal/CRDT.lean, stampJoin_{comm,assoc,idem}); the #lztextsync pattern (cell-model.md § Delta sync) already proves the version_vector / delta_since(their_vv) / apply_delta triad for text.

This section lifts the same delta pattern to the cell-register plane as an additive, optional control frame:

DeltaSinceRequest = { node: NodeId, their_vv: [(peer, counter)] }
  • A receiver of a DeltaSinceRequest responds with a CrdtSync whose ops carry only the states whose stamp is past their_vv (the delta), instead of the full converged state. An empty delta (nothing past their_vv) is a valid response.
  • The join is the same semilattice: apply_deltamerge — commutative, associative, idempotent — so a delta is safe to resend and applies in any order.
  • A binding that does not implement delta-CRDT sync continues to ship full state; the DeltaSinceRequest frame is opt-in and the receiver falls back to full state when it does not recognize the request. Conformance fixtures exercise both the full-state and the delta-sync paths and assert they converge to the same state (#lzspecdeltacrdt).

Reactive keyed-map sync (#lzfamilysync)

A keyed reactive map (ReactiveMap — its SourceMap / ComputedMap specializations; cell-model.md § “Keyed cell collections”) is a local keyed reactive collection. This section fixes its distributed contract: what a peer does with a keyed CrdtOp (NodeKey = namespace/suffix) for a map entry it has not registered locally.

The base plane, given such an op, resolves the node by NodeKey; if no cell is registered under that key the op is dropped. For a keyed map that is wrong: a key added on one replica would never appear on another, and any derived aggregate over the map (a count of entries) would diverge.

Map-granularity sync closes the gap with materialize-on-ingest: a replica registers a keyed map under a namespace; an inbound keyed op whose first NodeKey segment matches a registered map materializes a fresh entry (a new local NodeId, indexed by the wire NodeKey) seeded from the op’s converged register, then merges. Because the materialized entry is seeded from the op state, materialize-on-ingest is exactly the pointwise CRDT merge, so it inherits the full semilattice convergence.

Contract (proven in lazily-formal FamilySync.lean):

  • Materialize, never drop. A keyed op for an absent family entry makes that key present and adopts the op’s value (applyOp_present, applyOp_absent_adopts).
  • Membership propagation. After sync a key is present iff it was present on either replica — the union (present_merge). The present set only grows (deferral-not-dealloc); a removed entry is a value-level tombstone, not a dropped key.
  • Convergence + idempotence. Materialize-on-ingest equals merging the op’s single-entry state (applyOp_eq_merge), so op delivery is order-independent (applyOp_comm) and re-delivery is a no-op (applyOp_idem).
  • Derived-aggregate transparency. Once two replicas converge, any derived count over the keyed map agrees regardless of sync direction/batching (aggregate_converges, aggregate_batch_invariant) — e.g. a live-editor / open-document count converges across editors. A NodeId-churn-stable membership signal drives the recompute so a remote-materialized key is picked up by the derived aggregate.

Conformance: conformance/familysync/materialize_on_ingest.json.

Single-writer effect authority

CRDT convergence covers state. For irreversible external actions (send email, charge card, fire webhook), gate the effect behind a single-writer authority — a designated peer (or small Raft group) decides when the effect fires, at-most-once.

Durable effect sinks (#lzdurablesink)

Durable storage is an effect sink, not a transition authority. While a Lazily runtime is live, transitions are decided from Lazily state; durable storage receives a projection or an ordered fact as an effect, and a sink MUST NOT reload storage to arbitrate the transition it is currently persisting. A projection (latest recoverable state) is an idempotent upsert of the settled epoch from an Effect / AsyncEffect; lossless history uses the existing TopicCell / DurableOutbox drain (append / replay / ack with a stable cursor). Success advances a monotone durable_through(epoch); failure stays represented in live state as pending / retrying / backpressured. Cold loading and migration belong to a separate startup hydrator, never the decision seam. Ephemeral-plane values MUST NOT enter a durable sink — reuse the existing Durable marker. Full authority rule, the projection-vs-history shape table, and the two reference examples (coalesced projection; lossless ordered fact sink) live in docs/durable-sinks.md; the formal backstop is lazily-formal/LazilyFormal/DurableSink.lean.

Reliable Sync (#lzsync)

The Snapshot/Delta and CrdtSync planes above define what is on the wire and how state converges once delivered. They do not define delivery reliability: what a receiver does with a gap, what a sender does with a send that failed, or how a reconnected peer catches up. Today each consumer hand-rolls that loop (and Delta::apply_status → ResyncRequired is a signal with no production handler). This section fixes the reliable-sync protocol so gap recovery, reconnect backfill, and at-least-once delivery are specified and cross-language-conformant, not re-invented per integration.

Layering — mechanism here, policy injected. The three components below are pure protocol: identical logic in every binding, no I/O, no clock, no storage engine. The environment-specific choices — which byte transport, which persistence backend, retry cadence, threading — are supplied by the host application behind the named seams (SnapshotProvider, DurableOutbox store, Clock/scheduler, the IpcSink/IpcSource transport). A binding ships the protocol and a default in-memory backend; the host plugs durable storage and a real transport.

Control frames are IpcMessage variants, not a side channel. ResyncRequest and OutboxAck are two new externally-tagged IpcMessage variants (FFI message kinds 4 / 5), riding the same framed, codec-negotiated, bidirectional message plane as Snapshot/Delta/CrdtSync — the reverse (receiver → sender) direction of that plane. This is deliberate over a separate control type: they share one encode/decode path, one demux point, one FFI kind discriminant, and — because they interleave in the same ordered stream as the deltas — a well-defined in-band position (an OutboxAck { through_epoch: N } is meaningful relative to the deltas already sent on that channel). The liveness ops are CrdtOps on the existing CrdtSync variant. A binding MUST add both variants to its IpcMessage enum and its FFI message-kind mapping.

Codec. Every reliable-sync frame (the two control variants and the liveness CrdtOps) is an ordinary framed message on the negotiated codec. Per § Frame codecs the cross-language boundary negotiates msgpack (self-describing, evolution-safe, named-field), with json as the required reference codec; postcard stays the Rust-only fast path. Conformance requires every new frame to round-trip through both json and msgpack (semantic round-trip, not byte-identical, for the map codecs), the same discipline the three prior IpcMessage variants already hold.

ResyncCoordinator (#resync-coord)

A pure receiver-side decision function over the inbound frame stream. It holds one piece of state per source — last_epoch (the highest epoch this receiver has fully applied) — and classifies each inbound Snapshot/Delta:

enum ResyncAction { Apply, RequestSnapshot { from: u64 }, Ignore }

// pure; no I/O. `ingest` inspects a frame and returns the action; the caller performs it.
fn ingest(&mut self, msg: &IpcMessage) -> ResyncAction

Decision table (given receiver last_epoch = L):

InboundConditionActionEffect on L
Snapshot { epoch: e }alwaysApplyL := e (adopt snapshot state)
Delta { base_epoch: b, epoch: e }b == L and e >= b + 1ApplyL := e after fold
Delta { base_epoch: b }b < LIgnore (already applied / re-delivery)unchanged
Delta { base_epoch: b }b > LRequestSnapshot { from: L }unchanged until snapshot
Delta { epoch: e, base_epoch: b }e < b + 1Ignore (malformed/empty)unchanged
  • RequestSnapshot { from } is emitted at most once per detected gap; a coordinator that has already requested and not yet applied a covering Snapshot suppresses duplicate requests for the same gap (it stays in a resyncing sub-state, Ignore-ing further deltas until the snapshot lands). This bounds request storms under a burst of ahead-of-cursor deltas.
  • Convergence guarantee. A receiver that drops an arbitrary suffix of deltas, then applies the resync Snapshot, reaches the same graph state as one that saw every delta — gap recovery is state-equivalent, not lossy (proven ReliableSync.resync_convergence). This holds because a Snapshot is a full-state frame, not an incremental one.
  • Idempotent re-delivery. A Delta with base_epoch < L (a frame the receiver already folded, re-sent by the outbox) is Ignored, so at-least-once delivery yields exactly-once effect.

The application supplies snapshots for the sender side of a resync via:

trait SnapshotProvider { fn snapshot(&self, from_epoch: u64) -> IpcMessage; }  // returns Snapshot { epoch >= from_epoch }

DurableOutbox (#durable-outbox)

The sender-side contract that makes delivery at-least-once across a crash/reconnect. Today a failed send leaves a permanent gap (the out-epoch is bumped before the send; a reconnect is a fresh peer at epoch 0 with no backfill). The outbox closes that: every frame is durably recorded before it is sent, retained until the peer proves receipt, and replayed from the peer’s cursor on reconnect.

trait DurableOutbox {
    fn append(&mut self, epoch: u64, msg: &IpcMessage);          // MUST persist before the send is attempted
    fn ack_through(&mut self, epoch: u64);                        // peer proved receipt through `epoch`; retained frames <= epoch may be pruned
    fn replay_from(&self, cursor: u64) -> impl Iterator<Item = (u64, IpcMessage)>;  // frames with epoch > cursor, in epoch order
    fn retained_depth(&self) -> usize;                           // unacked queue depth — the backpressure fill level (Progress.retained)
    fn coalesce_to_snapshot(&mut self, epoch: u64, snapshot: &IpcMessage) -> bool { false }  // state cells: replace the unacked suffix with one snapshot; op-log frames decline (false)
}

Normative contract:

  • Append-before-send. A frame MUST be durably appended before it is handed to the transport. If the process dies between append and a confirmed send, the frame is still in the outbox and is replayed on reconnect. (The pre-send epoch bump that caused the permanent-gap bug is replaced by: append at epoch, send, and only ack_through retires it.)
  • Replay-from-cursor. On (re)connect the peer advertises the highest epoch it has applied (its ResyncCoordinator.last_epoch, carried in the reconnect handshake or an OutboxAck); the sender replay_from(cursor) re-sends every retained frame with epoch > cursor in order. A frame the peer already applied (base_epoch < last_epoch) is Ignored by the coordinator — replay is safe.
  • At-least-once ⇒ exactly-once effect. Replay delivers every op at least once; idempotent apply (the coordinator’s re-delivery Ignore + the CRDT/PartialEq guards) makes the net effect exactly-once — no lost op, no doubled op (proven ReliableSync.outbox_at_least_once_exactly_once_effect).
  • Ack semantics. ack_through(e) is a retention signal (safe to prune <= e), never a delivery-success authority for a domain effect (that is a causal receipt). An outbox MAY prune lazily; correctness does not depend on prompt pruning, only on not pruning un-acked frames.
  • OutboxAck frame. The receiver periodically (or on request) sends OutboxAck { through_epoch: u64 } — a new framed IpcMessage — so the sender can advance retention and so a reconnect handshake can carry the resume cursor.
  • Coalescing (optional, element-algebra-gated). An outbox MAY bound its retained depth without dropping an op, per § Backpressure & outbox coalescing. A state outbox (LWW/OR-set/counter/graph frames) collapses the unacked suffix to one Snapshot via coalesce_to_snapshot (memory-bounded — the coalesce is the cell’s own join, e.g. LWW → last value, proven ReliableSync.coalesce_by_join_sound / coalesce_to_snapshot_state_equiv). An op-log outbox (QueueCell frames) declines snapshot-coalesce (returns false) and instead fuses a run of same-direction ops into one batch frame (framing-bounded, order-preserving, ReliableSync.batch_fusion_state); it relies on source-side QueueCell.is_full for memory bounding. Coalescing is a retention optimization only — it never changes which effects the receiver observes.

Bindings ship an InMemoryOutbox (default; correct within a process lifetime) and a reference file-backed impl for tests/conformance (proves the crash-replay path deterministically). The host plugs its own durable store (agent-doc: SQLite) behind the same trait; the cursor math is protocol, the storage is not.

SyncDriver (#sync-driver)

The loop shape that wires an outbound producer to a transport through the outbox, and drives resync on reconnect. It owns no clock and no runtime: the host calls tick() from its own scheduler, so the driver stays a pure state machine with injected seams.

struct SyncDriver<S: IpcSink, R: IpcSource, O: DurableOutbox, C: Clock> { /* transport, outbox, coordinator, clock */ }

impl SyncDriver {
    // One scheduler-driven step. Returns Progress (sent N, applied M, resynced) or a DriverError.
    fn tick(&mut self) -> Result<Progress, DriverError>;
}

tick() performs, in order:

  1. Drain inbound. Pull available frames from IpcSource; feed each to ResyncCoordinator.ingest and perform the returned action (Apply into the local graph, emit a ResyncRequest / OutboxAck, or drop). Advance last_epoch on applied frames.
  2. Send outbound. For each new local flush, outbox.append(epoch, frame) then IpcSink.send(frame). A send error does not unwind the driver and does not lose the frame: the frame stays in the outbox, the driver records the transport as degraded, and the send is retried from the outbox on a later tick (cadence/backoff is the injected Clock’s policy).
  3. Resync on reconnect. When the transport reports a fresh/reopened peer, exchange cursors (peer’s OutboxAck.through_epoch) and replay_from(cursor); if the local receiver is behind, emit its ResyncRequest.
  4. Retention. On an inbound OutboxAck, outbox.ack_through(through_epoch).

Contract:

  • No frame lost on send failure. The append-before-send + retain-on-error rule means a frame is delivered on a subsequent tick once the transport recovers (the exact bug the current ?-propagating poll has, where a failed send bumps the epoch and unwinds).
  • Bounded work per tick. tick() does a bounded amount of drain/send work and returns; the host controls cadence. No internal blocking, no async runtime baked in.
  • Injected clock/transport. Retry cadence, backoff, and “is the peer fresh” come from the injected Clock and transport signals — policy stays in the host, mechanism in the driver.
  • Backpressure is host policy, not driver mechanism. The driver does not bound its outbound staging or block the producer: enqueue is unbounded and the DurableOutbox retains every unacked frame until the peer’s OutboxAck (this is the at-least-once durability guarantee — a frame is not dropped to relieve pressure). Instead the driver exposes the signals a host uses to apply backpressure itself: the stall state (is_stalled / stalled_for) and the retained-outbox depth (Progress.retained). A host bounds memory by (a) rate-limiting or coalescing enqueues off those signals — a re-emit at an already-accepted epoch is an idempotent no-op delta, so coalescing is the natural limiter — and (b) choosing a DurableOutbox that spills to durable storage rather than RAM (e.g. a SQLite-backed outbox), so a long-stalled peer grows disk, not heap. A bounded outbox with an is_full / high-watermark cap, and the per-item coalesce modes that keep it bounded without shedding an op, are specified in § Backpressure & outbox coalescing.

IpcSink/IpcSource are the existing abstract transport traits (feature ipc); the byte carrier is any DataChannel. Un-gating: the driver and the BridgeHub fan-out it can wrap depend only on these abstract ipc traits, so they MUST be usable without the webrtc feature (a caller supplying a Unix-domain-socket DataChannel gets reliable sync with no WebRTC dependency). The per-document channel decision (one driver/transport per document vs one hub with document_hash+NodeKey namespacing) is a host concern; the pull-only consumer path may not need BridgeHub fan-out at all.

Backpressure & outbox coalescing (#lzsync-backpressure)

The unacked outbox is a bounded cursor-queue, and that framing makes backpressure a queue property rather than a bolt-on: append is a push to the tail, OutboxAck.through_epoch is the consumer cursor, replay_from is a FIFO peek of the unacked suffix, and the retained depth (Progress.retained) is the fill level. A peer that stops acking is a consumer that stops advancing its cursor, so the queue fills — and a bounded outbox surfaces that as an is_full / high-watermark signal a host uses to throttle the producer. Recovery is the reactive dual: an OutboxAck advances the cursor, the queue drains below the watermark, and the fill signal clears so the producer resumes — the same reactive backpressure the QueueCell model pins (queuecell_bounded_backpressure), applied to the outbox. Non-ack is backpressure; the liveness cells disambiguate a slow consumer (queue fills — wait) from a dead one (queue drains — reconnect + replay). Formalized as ReliableSync.outbox_is_bounded_queue (enqueue grows depth by one, ack dequeues the acked front FIFO, a coalesced suffix is one frame).

A bounded queue must not grow unbounded under a persistently-slow peer, and the outbox stays bounded by coalescing its unacked suffix — never by dropping an op. How it coalesces is dispatched by the element’s merge algebra, not by a single global switch. Every cell type supplies its own coalesce, and each falls into one of two families by whether the type has an idempotent join:

ElementCoalesceBoundsBasis
LWW registerkeep the last (max-stamp) valuememoryWireLwwRegister::join folded over the suffix
OR-setunion of adds/removesmemoryOrSet::join folded
Counter / PN-countersum the deltasmemoryadditive monoid
Sequence CRDT / graph projectionmerged SnapshotmemorySnapshotProvider
QueueCell (op-log)batch-fuse a run of same-direction opsframes onlyop concatenation

The unifying law: coalesce = the element’s semilattice join folded over the unacked suffix.

  • Join / snapshot coalesce (idempotent cells: LWW, OR-set, counter, graph). The suffix collapses to one merged value — order-, regroup-, and retry-independent (ReliableSync.coalesce_by_join_sound; for the whole-graph frame, adopting the snapshot subsumes the dropped deltas — ReliableSync.coalesce_to_snapshot_state_equiv). This bounds memory: a peer arbitrarily far behind costs one frame, not the whole backlog. It is the CRDT answer to “don’t block the producer” — the producer keeps appending; the queue keeps collapsing. LWW-to-last-value is exactly the join being max-stamp.
  • Batch-fusion coalesce (op-log cells: QueueCell). A queue has no idempotent join — order and multiplicity are the value — so it cannot collapse; it only fuses a run of same-direction ops (QueuePush(a); QueuePush(b); QueuePush(c) → one atomic batch Delta, the op-log delta form of § QueueCell op-log delta form). Lossless and order-preserving (ReliableSync.batch_fusion_state; a fused multi-epoch delta equals its expanded unit run, multi_epoch_apply_eq_fold). This bounds frame count and per-frame overhead (one header, one codec envelope, one reactive invalidation on apply), not memory — a queue cannot drop elements and stay a queue. An op-log sync therefore still needs source-side try_push → Full (QueueCell.is_full) to bound memory; batch-fusion complements it, it does not replace it. Fuse only runs of same-direction ops; do not fuse across a pop in a bidirectional log.

Rule of thumb: a state sync (LWW/OR-set/counter/graph) absorbs pressure by collapsing (memory-bounded, never blocks the producer); an op-log sync (QueueCell) absorbs it by rejecting at the source (is_full, blocks the producer) and may fuse frames to cut overhead. Both keep every accepted op’s effect; neither sheds load silently.

Bounding producer memory (op-log). is_full / try_push → Full bounds the queue, not the producer: if the producer answers Full by stashing the rejected item in an unbounded side-buffer, the queue is bounded but memory is not — the backlog just moved up one hop. A QueueCell sync bounds producer memory only when the Full signal terminates at a stoppable or sheddable source, and no hop in between holds an unbounded buffer:

  • Suspend (the lazily-native path). The producer is a reactive effect that reads is_full; while full it does not run, and the pop that clears full (true→false) invalidates the reader and re-fires it. Producer memory is bounded because a not-running effect holds no backlog — it generates on demand, gated by the queue. This is the queuecell_bounded_backpressure loop used as a producer gate rather than a mere status read.
  • Propagate upstream. An imperative producer fed by its own bounded input returns Full/pending to its source, recursively, until the chain reaches something that can actually stop (a pull-based reader; a socket whose reads pause, so TCP advertises a zero window and the remote sender stops). Every hop is bounded-and-backpressuring; the stop propagates to the origin.
  • Shed. When the source cannot stop (a real-time feed, an external peer that will not slow), the only way to bound memory is an overflow drop policy — drop-oldest / drop-newest, the non-reject QueueStorage backends the QueueCell model names — or a lossy summary. Dropping is a deliberate, observable choice, not silent truncation.

The invariant: backpressure bounds memory only if it ends at a stoppable or sheddable source; an unbounded buffer at any hop defeats it. The queue supplies the signal and the bound at the queue; the producer must convert that signal into actually-not-generating (suspend/propagate) or actually-discarding (shed) — never into unbounded local absorption. (A state cell escapes this because its coalesce is an unbounded-absorption-that-stays-bounded: churn collapses into the join. A queue has no such join, which is exactly why its producer must pause or drop.)

Two seam refinements this makes explicit, both host-invisible in the current traits:

  • Ack-as-credit. For non-ack to mean consumer backpressure (not merely non-receipt), a receiver SHOULD gate its OutboxAck.through_epoch on its consumer’s drain cursor, not on mere fold-into-projection — otherwise a backed-up consumer keeps acking and the credit signal never forms. The unacked depth then behaves as a sliding credit window; withholding the ack pushes back end-to-end onto the sender’s outbox.
  • Sink would-block. IpcSink::send returning only Ok/Err cannot express a transport that is up but momentarily full (kernel socket buffer, DataChannel.bufferedAmount); that fullness is a distinct layer from consumer-slowness and is invisible to the ack. A binding MAY widen the sink outcome to distinguish Full (re-stage the frame + back off — not a reconnect) from a hard failure. This is the one backpressure axis the ack cursor structurally cannot cover.

Transport seam (IpcSink / IpcSource, #lzsync-transport-seam)

The SyncDriver is generic over exactly two host-supplied seams. Every binding that ports the driver MUST provide the same two-method contract so the loop above is identical across languages; the seams are the injected boundary the design assigns to the host (“which socket” is a deployment choice), so they carry no wire form of their own — what crosses the wire is the codec-encoded IpcMessage frame (msgpack is the cross-language default; see § Frame codecs). A binding names them idiomatically (Rust traits, Kotlin/TS interfaces); the semantics are normative:

// Outbound: deliver exactly one already-encoded protocol frame.
trait IpcSink   { fn send(&mut self, msg: &IpcMessage) -> Result<(), SinkError>; }
// Inbound: poll for the next frame without blocking.
trait IpcSource { fn recv(&mut self) -> Result<Option<IpcMessage>, SourceError>; }
  • send MAY fail and MAY be lossy. A send error means the frame was not durably handed to the peer. At-least-once is a driver property, not a sink property: on a send error the driver keeps the frame in the DurableOutbox and replays it after the next reconnect. A sink is therefore free to be a plain best-effort write (one connect-send-receipt on a Unix socket, one DataChannel frame, …) — it never has to buffer or retry, because the outbox already does.
  • recv is poll, not block. Ok(None) means the source is currently exhausted or closed; the driver treats it as “no inbound progress this tick” and returns — it never parks a thread on the source (cadence is the host’s Clock/scheduler policy). Ok(Some(frame)) yields one frame.
  • A recv Err is the reconnect signal. A source read failure surfaces from tick() as DriverError::Source; the host re-establishes the byte carrier and calls on_reconnect(), after which the next tick() replays the unacked outbox suffix from the peer ack cursor and re-advertises the receiver cursor. (A sink failure, by contrast, is retain-and-stall, not a DriverError — it is reported through Progress/stall signals so the host can back off.)

Because the seam has no wire representation, it adds no conformance fixture: the reliable-sync fixtures already pin the driver’s observable behavior (gap→resync convergence, outbox replay, idempotent redelivery) at the message-sequence level, which is the correct abstraction — the seam sits deliberately below it. This is also why the seam is not formalized in ReliableSync.lean: “send/receive a frame” has no algebraic content to prove; the invariants that matter (resync_convergence, outbox_at_least_once_exactly_once_effect, the liveness lattice joins) are proven over frame sequences, above the transport.

Liveness cells: OR-set and LWW (#lzsync-liveness)

Cross-process liveness — “editor pid X has doc Y open”, “pid X holds the owner lease” — is carried as CRDT cells on the CrdtSync plane, not as a bespoke frame, so it inherits that plane’s idempotent, frontier-resumable, re-delivery-safe convergence. Two register shapes cover the liveness needs:

  • OR-set membership — the open-set of (doc, pid) presence. An observed-remove set: a (doc, pid) is present iff some add-tag for it is not shadowed by a remove-tag that observed that add. This gives the exact “add wins over a concurrent stale remove” bias liveness needs (a re-open concurrent with a lagging close keeps the doc open), and re-delivery of an add/remove is a lattice join → idempotent. Whole-editor death removes every (doc, pid) for that pid.
  • LWW liveness flag — a per-pid alive: bool and the owner lease as HLC-stamped last-writer-wins registers (the CRDT plane’s default register, § Cell register types). The OS process-exit event writes alive[pid] = false; the highest-stamp write wins, and a stale re-assert is dominated.

Normative semantics:

  • Frontier-resumable. A liveness op that fails to send while the peer is down is re-sent on reconnect from the CrdtSync.frontier (the plane is already “safe to resend”); no liveness state is lost across a disconnect. This is what lets the open-set/lease survive a controller recycle without the editor re-announcing.
  • Derived authority is reactive. “Is doc Y live” = any present (doc, pid) in the OR-set whose alive[pid] is true — a derived aggregate over the liveness keyed map (the #lzfamilysync materialize-on-ingest + derived-count contract). One alive[pid] = false write fans out to every doc that pid held (whole-editor death cascade), reactively.
  • Idempotent + convergent. OR-set join and LWW join are semilattice joins, so out-of-order, duplicated, or batched delivery all converge and re-delivery is a no-op (proven ReliableSync.crdt_liveness_convergence_under_retry, building on the existing CRDT lattice proofs).
  • Per-doc isolation. Liveness keys are namespaced (document_hash, …) like every other keyed op, so a stale overlay for doc B cannot flip doc A’s authority.

Partition & eviction (#lzsync-partition-eviction)

A slow peer, a partitioned peer, and a dead peer are three different states, and dropping a peer is the last rung of an escalation ladder — never the first response to a missed ack. This section fixes when a peer is dropped, how the network is isolated from one bad peer, and how a dropped peer rejoins.

Per-peer outbox isolation (normative). In a multi-peer fan-out each peer MUST have its own DurableOutbox and ack cursor. A single shared outbox would make the slowest peer the network’s pace-setter (head-of-line blocking) and let one partition stall every peer. With a per-peer outbox, a slow or partitioned peer fills only its own cursor-queue; fast peers are untouched, and the drop decision is per-peer, not network-wide. This is what lets backpressure (§ Backpressure & outbox coalescing) be contained rather than contagious.

Escalation ladder. A degraded peer is handled in this order; each rung is exhausted before the next:

RungTriggerActionDrops data?
1. Backpressureconsumer behind, outbox fillingis_full throttles the producer on that peer’s channelno
2. Coalesce / suspend / shedoutbox at watermarkstate cell → collapse suffix to one frame; op-log queue → suspend the producer, or shed (drop-oldest/drop-newest) if the source can’t stopstate: no; queue: only on explicit shed
3. Retain + replaysend/recv error (partition)keep the unacked suffix; replay from the peer cursor on on_reconnectno
4. Evictliveness lease expiry, or bounded outbox exceeded by an un-coalescible op-log with an un-stoppable sourceremove the peer’s OR-set presence, reclaim its outboxreclaims retained frames; peer full-resyncs on return

Eviction is gated on the liveness lease, not on missed acks. A peer is dropped only when its OR-set / LWW liveness lease expires (the partition/death signal) — or, for a persistently-slow op-log peer, when its own bounded outbox is exceeded and neither coalescing (a queue cannot state-supersede coalesce) nor producer-suspension (the source cannot stop) can bound it. Eviction MUST be observable (it removes the peer’s (doc, pid) presence, a visible OR-set remove), never silent. A state peer effectively never reaches rung 4 on slowness alone: coalescing collapses its backlog to one frame, so a slow state peer costs one snapshot, not an eviction.

A returning peer rejoins as fresh. An evicted peer that reconnects is a fresh receiver (state empty, last_epoch = 0): it MUST full-resync by adopting a Snapshot, not replay from a stale cursor (its retained frames were reclaimed on eviction). Convergence is lossless — the fresh receiver reaches the sender’s full state (ReliableSync.evicted_peer_resyncs_fresh, a corollary of resync_convergence). The OR-set add-wins-over-stale-remove bias (ReliableSync.orset_add_wins_over_stale_remove) protects the race where a peer returns just as its lease expires: the rejoin add is not shadowed by the lagging eviction remove, so a peer that is coming back is not wrongly dropped.

Partition rule differs for queues (CAP). Because a queue’s total order is authoritative (a single sequencer in SPSC) and has no idempotent join, divergent queue order across a partition cannot losslessly converge — a queue cannot be AP-and-lossless. So a distributed (consensus-backed) queue favors CP under partition: the minority side (no quorum) blocks writes rather than dropping the peer — it refuses writes it could not later reconcile, and rejoins the majority order on heal. You do not “drop the partition”; you decline minority writes and evict individual peers only on lease expiry. State cells (LWW/OR-set) are the opposite: their join converges, so they MAY stay AP — accept writes on both sides of the partition and merge on heal. This is why distribution of a QueueCell is a consensus storage-backend property (the distributed-queue PRD / RaftQueueStorage), while liveness/register cells distribute on the plain CrdtSync plane. See § QueueCell op-log delta form.

Conformance

New fixtures under conformance/reliable-sync/ pin the protocol cross-language (rs/js/kt):

FixturePins
resync_gap_converge.jsondrop a delta suffix → RequestSnapshot → apply Snapshot → same graph as the no-drop receiver (ResyncCoordinator decision table + convergence)
outbox_replay_after_crash.jsonappend-before-send, replay-from-cursor after a simulated crash, ack_through retention, exactly-once effect under replay
idempotent_redelivery.jsona re-delivered (base_epoch < last_epoch) delta is Ignored; net state unchanged
multi_epoch_delta.jsona Delta with epoch > base_epoch + 1 applies equal to the unit-delta fold; atomic last_epoch advance
liveness_orset_lww.jsonOR-set open-set membership + LWW alive/lease; whole-editor-death cascade; derived live-doc aggregate converges under retry/re-delivery
coalesce_bounds_outbox.jsonstate outbox collapses an unacked delta suffix to one Snapshot (retained depth → 1) with the receiver reaching the same graph as the full-run receiver; op-log outbox declines snapshot-coalesce and fuses a run into one batch (order-preserving); non-ack fill / ack drain of the cursor-queue (#lzsync-backpressure)
liveness_lease_eviction.jsonescalation ladder (backpressure → coalesce/suspend/shed → retain+replay → evict); per-peer outbox isolation; eviction gated on liveness-lease expiry not missed acks; evicted peer rejoins fresh + full-resync (evicted_peer_resyncs_fresh, add-wins race); distributed-queue CP minority-blocks-writes vs state-cell AP (#lzsync-partition-eviction)

Every fixture’s frames MUST round-trip through both json and msgpack. The ResyncCoordinator, DurableOutbox, and liveness models are the shared cross-language pins; lazily-formal ReliableSync.lean is the correctness backstop the implementations must match.

Permission Boundary (RemoteOp)

Only nodes on the per-peer allowlist are serialized into a snapshot or delta. Non-allowlisted nodes are omitted entirely (not even as Opaque).

RemoteOp

RemoteOp = { kind: OpKind, node: NodeId }
OpKind = "read" | "write" | "trigger_effect"
  • Three kinds are gated independently: a read grant never implies write or effect-trigger.
  • PeerPermissions is default-deny per-peer allowlist.
  • filter_readable(peer, nodes) drops non-readable nodes from results before serialization.

Protobuf graph-boundary interoperability

Protobuf is an optional, capability-negotiated encoding of Lazily’s graph boundary algebra. It does not own a second graph runtime. The canonical semantics remain in this specification and its executable conformance traces; canonical JSON remains the diagnostic and fixture representation, and msgpack remains part of the existing interop matrix.

The canonical schema is proto/lazily/graph_boundary/v1/graph_boundary.proto. Its reviewed semantic classification is proto/field-ledger.json.

Ordinary text mutation uses GraphInput.cell_text_splice, bounded to one stable cell with a local UTF-8 offset and expected revision. bootstrap_snapshot is a different oneof variant and is legal only for bootstrap, explicit recovery, or checkpoint compaction. A cache or native-library reload is therefore never promoted into operator mutation authority.

Peers advertise the protobuf codec, the protobuf-graph-boundary-v1 feature, and a compatible protocol range before exchanging these envelopes. The feature is disabled unless both sides advertise it. Unsupported versions, unknown semantic enum values, stale generations or epochs, sequence gaps, and invalid snapshot purposes fail closed at admission. Duplicate sequences are idempotent.

Logical hashes are computed from the existing canonical logical representation, never raw Protobuf bytes. Protobuf map ordering, unknown-field preservation, and implementation-specific serialization make byte equality unsuitable as logical identity.

The six canonical traces in conformance/protobuf/graph_boundary_traces.json pin partial typing, cross-cell bounds, cache fencing, native reload behavior, duplicate/reordered delivery, and the snapshot/mutation variant boundary. The binding parity ledger in proto/field-ledger.json records Rust, Kotlin, and TypeScript as the reproducible-generation pilot. Python, Go, C++, Dart, Zig, and C# remain capability-gated known-uncovered findings until their native generators and reducers replay the same logical traces.

Assertion-block schemas

Canonical conformance claims use three historical object names: assertions, expect, and expected. They are executable contract data, not free-form fixture metadata.

schemas/assertion-blocks.json gives every such object a fail-closed Draft 2020-12 schema. Each route is identified by its fixture-relative path and a JSON pointer whose array indexes are normalized to *. Route schemas:

  • reject unknown keys with additionalProperties: false;
  • require every key present in all examples at that route;
  • validate scalar, array, object, and nested value shapes;
  • cover every fixture area, including families without a whole-fixture schema;
  • are checked in both directions, so an unvalidated block and a stale unused schema route both fail.

The file is generated from reviewed corpus state, but it is intentionally checked in. A fixture edit therefore has to carry a visible schema diff:

python3 scripts/gen_assertion_block_schema.py

make check runs the generator in --check mode and mutation-checks unknown keys, missing required keys, wrong value types, and unknown fixture routes. This guard complements binding-side read/assert ledgers: JSON Schema constrains what a fixture may say, while the runtime ledgers prove a binding actually consumed and asserted what the fixture said.

Causal Receipts

CausalReceipt is lazily’s generic outcome projection for commands and effect requests keyed by a stable causation_id.

It is intentionally not a transport ACK. observed and accepted are non-terminal receipt outcomes; they can record that a peer saw or queued work. applied and rejected are terminal outcomes. Domain-specific facts may refine those terminal outcomes, but they should not invent a delivery-ACK authority.

{
  "description": "Causal receipts fold by causation id and generation. observed/accepted are non-terminal; applied/rejected are terminal; stale generations are ignored by the current projection.",
  "protocol_version": 1,
  "kind": "Receipt",
  "model": "CausalReceipt",
  "assertions": {
    "receipt_count": 4,
    "current_generation": 7,
    "causation_id": "patch-123",
    "terminal_outcome": "applied",
    "stale_receipt_ids": ["receipt-stale"],
    "nonterminal_outcomes": ["observed", "accepted"]
  },
  "wire": {
    "CausalReceipts": {
      "receipts": [
        {
          "receipt_id": "receipt-observed",
          "causation_id": "patch-123",
          "observer": "editor",
          "generation": 7,
          "outcome": "observed",
          "reason": null,
          "payload_hash": null
        },
        {
          "receipt_id": "receipt-accepted",
          "causation_id": "patch-123",
          "observer": "editor",
          "generation": 7,
          "outcome": "accepted",
          "reason": null,
          "payload_hash": null
        },
        {
          "receipt_id": "receipt-applied",
          "causation_id": "patch-123",
          "observer": "editor",
          "generation": 7,
          "outcome": "applied",
          "reason": null,
          "payload_hash": "sha256:2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"
        },
        {
          "receipt_id": "receipt-stale",
          "causation_id": "patch-123",
          "observer": "editor",
          "generation": 6,
          "outcome": "rejected",
          "reason": "stale generation",
          "payload_hash": null
        }
      ]
    }
  }
}

The normative field list and projection rules live in protocol.md § Causal Receipts. The schema is schemas/receipts.json.

Command / RPC Message Plane

Editor and runtime integrations need to issue commandsRun Agent Doc, sync, focus, save, session operations — with a single reusable admission, dedupe, cancellation, generation-guard, progress, and reconnect story. Without one, every caller reinvents in-flight/dedupe/supersede/retry/timeout logic, and accepted / queued feedback gets mistaken for terminal success.

lazily’s command plane (command-plane-v1) is that shared substrate. It is an additive sibling to Snapshot / Delta / CrdtSync, not a replacement: command frames can ride the same transports and reflect into the normal state graph, but they carry command traffic, not cell state.

The four frames

FrameRole
CommandSubmitSubmit a command: envelope + domain payload (an IpcValue)
CommandCancelPreempt a still-non-terminal command by command_id
CommandEventsProgress/detail events (UX + diagnostics only, never proof)
CommandProjectionFolded, queryable command state; also the reconnect resync image

lazily owns the envelope; the namespace owns the payload. lazily never decodes payload. Agent-doc publishes its own payload schemas (agent-doc.editor_route.v1, agent-doc.sync_tmux_layout.v1, …) and only references lazily’s envelope.

Progress is not proof

The single hard rule: terminal authority is the causal receipt. A command is terminal only when a terminal CausalReceipt for its command_id folds in (applied, or rejected — including the cancelled / superseded / timed_out reasons). observed / accepted / started / queued-admission events are non-terminal progress. A transport ACK is never terminal.

This keeps command events from becoming a second proof system. Events may carry queue position, retry advice, or copied CLI output; the effect still folds through receipts and domain facts.

RPC is a facade

call / submit / cancel / observe / projection are implemented entirely over the four frames:

await client.call("agent-doc.editor_route", payload, {
  commandId,
  idempotencyKey: "project-root:plan.md:run",
  authorityGeneration: 42,
  deadlineMs: 120000,
  policy: { dedupe: "same_idempotency_key", supersede: false, cancelOnPreempt: true }
});

call resolves only on a terminal causal-receipt projection. A network ACK, controller admission, or accepted / queued event never resolves a unary call. submit returns the command_id for callers that manage events and projection themselves. Reconnect uses CommandProjection; a call replays only when the idempotency policy says replay is safe.

Rules

  • Generation guards — events/receipts outside the command’s current authority generation are ignored (kept only as audit data).
  • Idempotency — replaying a submit/event/receipt with a known id is a no-op.
  • Cancel before terminal only — a cancel after applied is ignored.
  • Terminal conflict fails closedapplied vs rejected at the same generation is not resolved by winner selection.
  • Reconnect equivalence — folding a CommandProjection equals folding the events and receipts it summarizes.

The normative field list and rules live in protocol.md § Command / RPC Message Plane. The schema is schemas/message-passing.json. Conformance fixtures live in conformance/message-passing/; each binding replays them through its CommandProjection reducer and RPC facade.

{
  "protocol_version": 1,
  "kind": "Command",
  "model": "CommandProjection",
  "description": "observed/accepted events are progress only; the command becomes terminal ONLY when the applied CausalReceipt folds in.",
  "frames": [
    {
      "schema": "message-passing",
      "wire": {
        "CommandSubmit": {
          "command_id": "cmd-run-1",
          "causation_id": "cmd-run-1",
          "source": "vscode-plugin",
          "target": "project-controller",
          "namespace": "agent-doc",
          "name": "editor_route",
          "authority_generation": 42,
          "idempotency_key": "project-root:plan.md:run",
          "deadline_ms": 120000,
          "policy": {
            "dedupe": "same_idempotency_key",
            "supersede": false,
            "cancel_on_preempt": true
          },
          "payload_type": "agent-doc.editor_route.v1",
          "payload_hash": "sha256:f753816ec570dcaa98e80dafcf975b020a5ccc9db551c69e121567a48ee16346",
          "payload": {
            "Inline": [
              123,
              34,
              102,
              105,
              108,
              101,
              34,
              58,
              34,
              47,
              104,
              111,
              109,
              101,
              47,
              117,
              115,
              101,
              114,
              47,
              112,
              114,
              111,
              106,
              101,
              99,
              116,
              47,
              112,
              108,
              97,
              110,
              46,
              109,
              100,
              34,
              44,
              34,
              114,
              101,
              108,
              97,
              116,
              105,
              118,
              101,
              95,
              112,
              97,
              116,
              104,
              34,
              58,
              34,
              112,
              108,
              97,
              110,
              46,
              109,
              100,
              34,
              44,
              34,
              100,
              105,
              115,
              112,
              97,
              116,
              99,
              104,
              95,
              111,
              110,
              108,
              121,
              34,
              58,
              102,
              97,
              108,
              115,
              101,
              44,
              34,
              112,
              108,
              97,
              105,
              110,
              95,
              116,
              114,
              105,
              103,
              103,
              101,
              114,
              34,
              58,
              102,
              97,
              108,
              115,
              101,
              44,
              34,
              119,
              97,
              105,
              116,
              95,
              98,
              117,
              100,
              103,
              101,
              116,
              95,
              109,
              115,
              34,
              58,
              49,
              50,
              48,
              48,
              48,
              48,
              44,
              34,
              108,
              97,
              121,
              111,
              117,
              116,
              95,
              97,
              114,
              103,
              115,
              34,
              58,
              123,
              34,
              99,
              111,
              108,
              117,
              109,
              110,
              115,
              34,
              58,
              50,
              44,
              34,
              102,
              111,
              99,
              117,
              115,
              34,
              58,
              116,
              114,
              117,
              101,
              125,
              44,
              34,
              114,
              111,
              117,
              116,
              101,
              95,
              107,
              101,
              121,
              34,
              58,
              34,
              112,
              114,
              111,
              106,
              101,
              99,
              116,
              45,
              114,
              111,
              111,
              116,
              58,
              112,
              108,
              97,
              110,
              46,
              109,
              100,
              58,
              114,
              117,
              110,
              34,
              44,
              34,
              101,
              100,
              105,
              116,
              111,
              114,
              95,
              97,
              116,
              116,
              101,
              109,
              112,
              116,
              95,
              105,
              100,
              34,
              58,
              34,
              97,
              116,
              116,
              101,
              109,
              112,
              116,
              45,
              55,
              34,
              125
            ]
          },
          "required_features": [
            "causal-receipts",
            "command-events"
          ]
        }
      }
    },
    {
      "schema": "message-passing",
      "wire": {
        "CommandEvents": {
          "events": [
            {
              "event_id": "ev-1",
              "command_id": "cmd-run-1",
              "kind": "observed",
              "generation": 42,
              "detail": null
            },
            {
              "event_id": "ev-2",
              "command_id": "cmd-run-1",
              "kind": "accepted",
              "generation": 42,
              "detail": "queued at position 1"
            },
            {
              "event_id": "ev-3",
              "command_id": "cmd-run-1",
              "kind": "started",
              "generation": 42,
              "detail": null
            }
          ]
        }
      }
    },
    {
      "schema": "receipts",
      "wire": {
        "CausalReceipts": {
          "receipts": [
            {
              "receipt_id": "rcpt-1",
              "causation_id": "cmd-run-1",
              "observer": "project-controller",
              "generation": 42,
              "outcome": "applied",
              "reason": null,
              "payload_hash": "sha256:f753816ec570dcaa98e80dafcf975b020a5ccc9db551c69e121567a48ee16346"
            }
          ]
        }
      }
    }
  ],
  "expect": {
    "projection": {
      "generation": 42,
      "commands": [
        {
          "command_id": "cmd-run-1",
          "status": "applied",
          "terminal": true,
          "generation": 42,
          "reason": null,
          "terminal_receipt_id": "rcpt-1",
          "last_event_id": "ev-3"
        }
      ]
    },
    "terminal_after_frame_index": 2
  }
}

Cross-Process Zero-Copy Transport

#lzzcpy — large payloads cross the IPC plane as descriptors, not copies.

A Snapshot / Delta / CrdtSync message may carry large cell/slot payloads (an Arrow record-batch, an image, a serialized sub-document). Copying those bytes through the wire codec on every hop is the dominant cost of a distributed lazily deployment. The zero-copy transport instead spills a large payload to a blob backend and ships a small Descriptor; the receiver resolves the descriptor against the same backend and reads the bytes in place — no copy, no checksum recompute.

This chapter defines the transport model, the pluggable backend (adapter) contract, and the wire Descriptor. The invariants are proven in lazily-formal/LazilyFormal/ZeroCopyTransport.lean (spill-then-resolve identity, backend isolation, ABA/generation safety, checksum integrity) and pinned by conformance/delta_zero_copy_arrow.json.

Model

 producer                                            receiver
 ────────                                            ────────
 bytes ──spill──▶ backend.write ──▶ Descriptor        Descriptor ──resolve──▶ backend.read_view ──▶ bytes (view)
                     (mint id+gen+csum)    │                              (kind routes to the right backend)
                                           └──────── wire (msgpack) ──────▶
  1. Spill (producer). For an IpcValue / NodeState payload above a session-defined threshold, the producer calls backend.write(bytes) and gets a Descriptor ({kind, offset, len, generation, epoch, checksum}). It puts the descriptor in the message as SharedBlob instead of Inline bytes.
  2. Wire. Only the descriptor crosses the codec — the message stays small.
  3. Resolve (receiver). The receiver reads descriptor.kind, routes to the matching backend, and calls backend.read_view(descriptor) → a const view of the backend’s own bytes. No copy, no checksum recompute (the checksum was computed once at write and is validated against the cached value).

Threshold policy: payloads below the threshold stay Inline (copied through the codec — cheaper than a backend round-trip for tiny values). The threshold is a session/deployment knob, not a protocol constant.

The wire Descriptor

The descriptor is ShmBlobRef (schemas/defs.json) extended with an optional backend discriminator:

fieldtypemeaning
offset, lenu64byte range within the backend’s resolved buffer
generationu64ABA guard — a slot reused at a later generation is not misread
epochu64validity epoch (advanced on backend compaction/restart)
checksumu64FNV-1a-64 over the bytes (computed once at write, validated at read)
backendenum shm | arrow | in_process (optional, default shm)which pluggable backend resolves this descriptor

backend is optional and defaults to shm, so every legacy descriptor validates unchanged — the transport is a strict superset of the pre-existing shared-memory blob path. A receiver routes resolution by kind: a shm descriptor never resolves in an Arrow table and vice versa (the resolve_wrong_backend theorem).

Pluggable backends (adapters)

A backend is anything that satisfies the blob-backend contract:

operationcontract
write(bytes) → Descriptormint a fresh id at the current generation/epoch, store the bytes immutably, return a descriptor whose checksum is the bytes’ FNV-1a-64
read_view(Descriptor) → const bytes*return the stored bytes iff kind + id + generation + epoch + checksum all match; nullptr otherwise. No copy, no recompute
lifecycleadvance epoch on compaction/restart; never mutate a stored buffer in place (entries are immutable + stable-addressed for the lifetime a descriptor may reference them)

Because the contract is stated only over a backend’s issued-blob table + the read_view lookup, the transport theorems hold uniformly for every backend that maintains the contract — this is the universal guarantee no single-adapter fixture can establish. Three backends ship / are anticipated:

backendwhat holds the bytescross-process?typical use
shmPOSIX shared-memory region (shm_open + mmap)yes (same host)the default — host ↔ binding / peer ↔ peer on one host
arrowApache Arrow IPC stream / Flight-resolved bufferyes (Arrow’s zero-copy columnar IPC)analytics / columnar payloads — the bytes are an Arrow IPC stream the receiver imports as an Array/RecordBatch with no copy
in_processan in-process arena (single address space)nothe FFI host ↔ a binding loaded in the same process (editor plugin)

An Apache Arrow adapter implements the contract by holding spilled payloads as Arrow buffers and resolving a descriptor to the buffer’s raw bytes (or, for columnar consumers, directly to the Arrow Array — the descriptor’s bytes are an Arrow IPC stream). Because Arrow’s IPC format is itself zero-copy across a shared buffer, shm and arrow compose: an Arrow batch can live in a shm region and be resolved by either backend. New backends (e.g. a RDMA/verbs adapter, a Cuda IPC adapter) plug in by implementing the contract and adding a backend enum value — no transport or codec change.

Backend-agnostic invariants (proven)

The formal model parameterises over an abstract backend and proves, for any backend satisfying the contract (ZeroCopyTransport.lean):

  • resolve_write / transport_roundtrip — resolving the descriptor a backend minted via write returns exactly the bytes written. The consumer reads the backend’s own bytes, not a copy: the end-to-end zero-copy guarantee.
  • resolve_wrong_backend — a descriptor of one kind never resolves against a different backend’s table → receivers route by kind.
  • resolve_stale_generation — a slot reused at a later generation (or a stale ref to a freed slot) does not resolve against the new occupant (ABA safety via generation).
  • resolve_corrupt_checksum — a descriptor corrupted in transit is rejected rather than resolving to the wrong bytes.

Conformance

  • conformance/delta_zero_copy_arrow.json — a Delta whose SlotValue payload is a SharedBlob with backend: "arrow", validating the optional discriminator against schemas/delta.json.
  • conformance/delta_shared_blob.json — the legacy backend-absent (= shm) form, unchanged → backward compatibility.
  • conformance/codec/blob_backend_discriminator.json — the decoder half of the discriminator (#lzblobbackendstrict, protocol.md § Shared-memory payload path). The two fixtures above are both conforming frames, so between them they establish that a known backend round-trips and an absent one defaults — neither says what happens to a token this build does not know. That gap is where five of nine bindings independently decided to normalize an unknown backend to shm, which routes a non-shm descriptor into the shm table and leaves resolve_wrong_backend to be discharged by the checksum instead of by routing. The fixture makes the refusal executable and pins arrow alongside it, so a binding cannot pass by ignoring the discriminator.

Relationship to the wire codec

The transport is codec-agnostic but pairs with the msgpack codec: spill replaces Inline bytes (which the codec would copy) with a small SharedBlob descriptor (which the codec serializes as a handful of integers). The codec already distinguishes Inline vs SharedBlob; the transport adds the policy (when to spill) and the backend contract (how SharedBlob is resolved without a copy). See Wire Protocol.

Conformance Fixtures

The conformance/ directory contains canonical test fixtures that all IPC-capable bindings must validate against. Each binding’s CI should deserialize the wire field, run the assertions, and re-serialize to confirm round-trip fidelity.

Fixture schema

{
  "description": "Human-readable summary",
  "protocol_version": 1,
  "kind": "Snapshot" | "Delta" | "Receipt",
  "assertions": {
    "prose": ["…keys below that are PARAGRAPHS, not comparable values…"],
    "…language-agnostic field checks…": "…"
  },
  "wire": { "…canonical protocol JSON…" }
}

assertions.prose declares which sibling keys state an obligation in English rather than carrying a comparable value. A runner discharges those by naming the executable keys that prove them — see Prose assertion keys.

Fixture discriminability (#lzfixturediscrim)

Opening a fixture, consuming every key, and comparing every executable value still does not prove that the fixture data can distinguish the behavior it names. For example, every ordering predicate returns true for an empty or singleton roster. A runner can faithfully assert roster_sorted_ascending: true while a library that reverses its roster order remains green.

scripts/check-fixture-discriminability.mjs audits this fourth rung:

  1. Non-boolean routes are exact-value oracles. A changed observation differs from the canonical value.
  2. Boolean routes carrying both true and false somewhere at the same normalized fixture pointer have an in-corpus control that rejects either constant mutant.
  3. Every remaining single-valued boolean route must appear in audits/fixture-discriminability.json. A claim is either mutation-killed, with the binding, library-source mutation, command, observed failure, and a fixture witness, or untested with a reason. Missing and stale ledger entries fail CI.

untested is deliberately not a weaker spelling of pass. It records that the corpus claim has no registered library mutation proof, so coverage reports and follow-up audits cannot silently treat an asserted predicate as behavior-discriminating.

The signaling transcript is the first recorded proof. Three peers join; the third welcome must carry the two-element roster [1, 2]. Reversing SignalingRoom.roster in lazily-js now fails the canonical replay with actual [2, 1], while removing self-filtering and stamping from with the target rather than the registered sender fail the other two signaling assertions.

To initialize ledger entries after intentionally adding claims:

node scripts/check-fixture-discriminability.mjs --write-initial-ledger

The initializer preserves existing evidence and creates new entries as explicit untested claims. Replace that status only after running the cited library-source mutant and confirming that a canonical fixture—not merely an independent unit test—reddens.

Custom assertion callback consumption (#lzassertwithseam)

Every binding exposes a custom assertion helper for relations that plain equality cannot express: tolerances, containment, decoding, and derived projections. The helper passes the fixture value to a callback. Calling the callback is not itself proof that the callback used that value: an ignored parameter previously marked the key asserted while comparing nothing.

A binding MUST therefore:

  1. require the callback to expose and syntactically read its fixture-value parameter;
  2. record the key as asserted only after the callback completes successfully; and
  3. keep extraction explicit by using a projection helper (for example Python’s assert_key_into) when the caller performs the comparison after the helper returns.

scripts/check-assert-with-consumption.py enforces rule 1 for every maintained binding. scripts/check-assertion-ordering.py runs it as part of the existing binding conformance gate, so comments and string literals do not count as reads and unsupported callback forms fail closed. Its self-test includes an accepted and an ignored callback for every supported language. The runtime helper ordering enforces rule 2: a callback that throws, rejects, or returns a failing verdict cannot leave the assertion ledger green.

Current fixtures

FixtureKindDescription
snapshot_minimal.jsonSnapshotOne payload node, no edges
snapshot_multi_node.jsonSnapshotMultiple nodes and edges
snapshot_shared_blob.jsonSnapshotSharedBlob node state
delta_sequential.jsonDeltaAll 7 DeltaOp variants, sequential
delta_non_sequential.jsonDeltaNon-sequential delta with gap
delta_shared_blob.jsonDeltaCellSet/SlotValue with SharedBlob
receipts/causal_receipts.jsonReceiptCausal receipt projection with non-terminal and terminal outcomes

Publishing a corpus change (#lzspecpushbeforebindings)

Push the corpus change to lazily-spec FIRST, then verify and push the bindings. Never the other way round, however green the bindings look locally.

The asymmetry is the whole reason. Local verification resolves the corpus through the sibling working tree (../lazily-spec/conformance/...), so it sees your change the moment you save it. Every binding’s CI instead clones published main — for example lazily-cs/.github/workflows/ci.yml runs git clone --depth 1 https://github.com/lazily-hub/lazily-spec.git ../lazily-spec. So a corpus change that is green in nine local checkouts is invisible to all nine CI runs until it is pushed, and any binding that pins a scenario or fixture count fails on the mismatch.

That is not hypothetical. Landing the SeqCrdt fork-clock scenarios (#lzspecforkclockfixture) in the wrong order turned lazily-cs red with Expected: 8 / Actual: 6 — its runner census correctly counted the eight scenarios in the working tree while CI replayed the published six — and forced lazily-cpp to pin MIN_SCENARIOS to the published 144 rather than the local 146, then raise it again in a second commit once the push landed.

The order that works:

  1. Fix the bindings’ behaviour first if the new fixture would otherwise redden them, but do not push a binding whose census (scenario counts, coverage floors) encodes the new corpus.
  2. Commit and push the lazily-spec change, including any vendored mirrors re-synced by scripts/sync-conformance-fixtures.mjs --sync.
  3. Only then push binding changes that assume the new corpus, and re-run any CI that failed against the old one.

A floor or count in a binding must always describe what CI’s clone guarantees, never what your working tree happens to hold. make check here warns when the local corpus is ahead of origin/main, which is the moment this rule applies — see scripts/check-corpus-published.mjs.

Adding a new binding

Copy the fixture-loading pattern from lazily-rs/tests/conformance.rs. Each test should:

  1. Load the fixture.
  2. Parse the wire field into the binding’s native IpcMessage type.
  3. Assert the assertions fields.
  4. Re-serialize and compare for byte-for-byte round-trip fidelity, subject to the equivalence exemptions below.

Round-trip equivalence exemptions

Byte-for-byte comparison is the default, but it is not the contract for a field whose schema declares two encodings equivalent. Where the schema says a field may be omitted or sent in some canonical empty form, a binding MUST NOT be required to reproduce the sender’s choice: both encodings decode to the same value, so a binding is free to emit either one.

For such fields the round-trip comparison is semantic: normalize the fixture’s wire and the binding’s re-serialized output to the same canonical form (fill in the declared default for an absent field) before comparing. All other fields remain byte-for-byte.

Exempt fields:

FieldEquivalent encodingsDeclared by
CrdtSync.frontieromitted ≡ [] — “unchanged since the last accepted frame” (#lzspecfrontiersuppress)schemas/distributed.json (required is ["ops"] only)

A binding MUST accept an omitted frontier on decode and treat it as empty. Rejecting the absent form is a conformance failure; re-emitting it as [] is not.

Prose assertion keys (#lzprosekeyconvention)

An assertions block mixes two kinds of key. Most carry a value a runner can compare against observed behaviour — a list, a count, a vocabulary. A few carry an English paragraph that states an obligation and nothing comparable: clause, anti_vacuity, null_form, theorem, note. This section says what a runner MUST do with the second kind, because nothing did, and the nine bindings each decided.

The failure this closes

Replaying blob_backend_discriminator.json v2 — which added four new paragraphs (backend_form_vocabulary, null_form, non_string_form, epoch_disambiguation) — produced four different treatments of the same four keys:

BindingTreatment
lazily-jsexcused all four, its own assert-key.js warning against comparing an English paragraph to a literal
lazily-py, lazily-dart, lazily-go, lazily-kt, lazily-cs, lazily-zigexcused them with individually-worded reasons naming the assertion that discharges each — falsifiable in principle, checked by nothing
lazily-rsmarked them Expect::prose, a third tracker state exempt from every check, which requires a reason and then discards it
lazily-cppasserted all four against tallies computed from the run

Every one is defensible alone. That is the point: this is the same shape as the 5-2 split the blob-backend clause itself came from — an undocumented default and a deliberate choice are indistinguishable from the outside, and so are four deliberate ones.

Definition

A prose key is a key of a fixture’s top-level assertions block whose value is a natural-language paragraph: it states an obligation and carries no value a runner can compare against observed behaviour.

The corpus declares which keys those are, in assertions.prose — an array of sibling key names. A binding MUST NOT decide for itself. The declaration is itself a key of the block, so the existing consumption guards see it: a runner that ignores it fails with an unconsumed key, which is what makes the rollout self-enforcing.

Prose nested inside a data key is not a prose key. assertions.outcomes maps a vocabulary to English glosses; the assertion is the key set, and the parent key’s own assertion discharges it.

The rule

A prose key is discharged, never asserted and never excused. To discharge it a runner names the executable assertion keys that carry its obligation, and its assertion-key tracker verifies the naming. A binding’s tracker MUST fail the run when:

  1. a key listed in assertions.prose is asserted — comparing a paragraph, or a tally derived from one, to an English string pins wording, not behaviour. A copy-edit reddens the run and a library regression does not. reject_obligation says exactly this about error message formats; it applies no less to the paragraph stating it;

  2. a key listed in assertions.prose is excused with free text — an unfalsifiable reason (“prose”, “explains why the wire is text/hex”) is indistinguishable from the undocumented default this clause exists to remove;

  3. a key not listed in assertions.prose is discharged;

  4. the set of discharged keys differs from assertions.prose — this is the comparison that consumes prose itself, and it is what makes a forgotten key fail rather than vanish;

  5. a discharge names no keys;

  6. a discharge names a key that the same fixture’s run did not assert;

  7. a discharge names a key that is itself prose, or names prose. The second half is not redundant: prose never lists itself, so without it rule 7 misses dischargedBy: ["prose"] — and rule 4’s own comparison marks prose asserted, so rule 6 would wave it through. A paragraph discharged by the declaration that it is a paragraph proves nothing. Seed the prose-name set with prose itself.

  8. an opened fixture whose block declares prose never reaches verification. Rules 1-7 are all satisfied over an empty population, so a fixture that is opened and then never replayed passes every one of them while proving nothing — the vacuity the corpus’s own anti_vacuity keys exist to name, reappearing in the guard meant to enforce them. Derive the required verifications from the corpus, never from a hand-kept count.

Evaluate rule 7 before rule 6. A paragraph can never be asserted — rule 1 forbids it — so a discharge naming another paragraph always also violates rule 6. Check 6 first and rule 7 becomes dead code that never reports, and the run fails with “names a key this run never asserted” when the real defect is “names a paragraph”. The numbering is not the evaluation order; this one pair is.

Rule 6 means ASSERTED, not merely satisfied: an excused key does not discharge anything, because an excuse is precisely the absence of a comparison. A discharge naming a key the fixture does not carry at all is a distinct failure — the discharge has rotted, exactly as a stale excuse has. One executable key may discharge several paragraphs; decoded_backend carries five in blob_backend_discriminator.json, and that is expected rather than a collision.

Rule 6 is the whole convention: the excuse becomes falsifiable, because the tracker can check it. “epoch_disambiguation is discharged by frame_epoch and blob_epoch” is a claim about the run; “epoch_disambiguation is prose” is not.

A discharge may name a key that carries the obligation only by PROXY. theorem names a Lean theorem in another repository; the run can only prove its consequence. Naming a proxy is conforming. Naming a key that has nothing to do with the obligation is not, and no tracker can tell the two apart — that judgement stays with review, which is why the discharge is written at the call site where review sees it. wire_encoding is no longer a proxy: every declaring scenario carries expect.wire_input_fnv1a64, and the runner hashes the exact UTF-8 JSON bytes or decoded MessagePack bytes it passes to the library decoder.

Scope of “the same fixture’s run”

The discharge ledger is fixture-scoped, not block-scoped. An obligation stated in assertions is routinely carried by a per-scenario expect key: epoch_disambiguation is discharged by expect.frame_epoch and expect.blob_epoch, which are asserted long after the assertions block itself is finished. A named key is therefore matched by key name, in any block of that fixture, and verification happens when the fixture’s replay is finished. A runner that never verifies MUST fail — an unverified discharge claim is reported by the ledger’s own teardown, exactly as an unconsumed key is.

A block declaring prose MUST also carry at least one non-prose key. A block that is entirely prose has nothing that could discharge it.

A block is identified by its path in the fixture, not by a human-written label. Rules 3 and 4 are block-local, so two blocks whose tracker labels happen to collide would merge silently and each would satisfy the other’s declaration. Key them on assertions, scenarios[3].expect, steps[7].expect — something the fixture determines.

Which rules are block-local and which are fixture-wide. The declaration is block-local: each block owns its own prose array, and rules 3 and 4 compare a block’s discharged set against that block’s array. Only the NAME MATCHING of rules 6 and 7 is fixture-wide, because that is the half that has to reach a per-scenario expect key.

A “run” is one test, not one process. Where a fixture is replayed by several tests, the ledger is scoped to each and cleared at its verification. Unioning asserted keys across tests would let a discharge in one test be satisfied by an assertion in another, which is the same accident-of-collocation the fixture-scoped ledger exists to bound.

Arming the verification net is a LIFO hazard. Every cleanup mechanism the nine use — Drop, t.Cleanup, addTearDown, IDisposable, a destructor — runs in reverse registration order. A net armed by the first prose_key call therefore fires BEFORE a verification the runner registered earlier in its body, and reports a false failure. Arm the net from the same seam that already owns the block’s consumption check, which is structurally guaranteed to run last.

Reserved annotation names

note, description and reason inside a per-step or per-scenario block are annotations, exempt by name in every binding.

A corpus declaration overrides the by-name exemption, and a tracker MUST apply it first. Both frame_roundtrip_json.json and frame_roundtrip_msgpack.json declare note prose. A tracker that subtracts its reserved-name set before consulting assertions.prose makes that declaration invisible to its consumption guards: the key is exempt from the unread guard and exempt from the unasserted guard. Three of the nine hit this independently while implementing the clause. Evaluate prose on the raw block, before any name-based exemption; the exemption applies only to keys the block did not declare.

How bad the inversion is depends on where rule 4 reads from, and this is worth stating precisely because one binding measured it rather than assuming. Where rule 4’s comparison reads the RAW block — the declared set kept separately, never subtracted from — the inversion degrades to a worse error message: the forgotten note is still caught, just by the set comparison rather than by the guard that would have named it. Where rule 4 reads through the exempted view, nothing is left: both fixtures skip the convention entirely and the binding still reports conforming. Keep the declared set independent of the exemption and the inversion cannot be fatal — but evaluate in the stated order anyway, because the guard that names the key is the one a reader acts on.

Inside a block that declares prose, the name exemption is off entirely: the corpus wins, so a note sitting in a declaring block but absent from its array needs an assertion or an excuse like any other key. Everywhere else the exemption stands as-is.

Naming a discharge that discharges nothing

The tracker checks that a named key was asserted. It cannot check that the assertion proves the paragraph, and three bindings found the gap the same way: a key that compares the fixture to itself is asserted, satisfies rule 6, and discharges nothing. Two shapes recur.

scenario_count asserted against len(fixture["scenarios"]), and codecs / key_forms compared to hand-written literals, are green over a runner that decodes nothing — which is the exact vacuity anti_vacuity exists to name. Compare them against what the run really replayed before naming them.

Worse, nodekey_null_leniency.json’s wire_encoding obligation — that the ABSENT and explicit-null wire forms stay distinguishable into the runner — was dischargeable by nothing at all in at least one binding: key ?? null collapses the two the instant the value is decoded, and every key in that fixture’s expect blocks is identical for the omitted and null families. The four null scenarios were the four omitted ones wearing a different id. The fix is a control that reads the raw wire slot BEFORE the decoder runs, which is what the sibling blob-backend runner already does for backend. Adding the missing control is conforming; naming a key that merely happens to be asserted is not. That exemption is only safe while they annotate; an annotation MUST NOT state an obligation, because a reserved name is a place no runner can be made to discharge anything. scripts/check-prose-keys.mjs enforces this and carries a both-directions allowlist of the instances that already do — five reactive-graph step notes, each a real normative rule (teardown is idempotent, a stale cell handle whose id has been recycled MUST be a no-op) that no binding checks today. New instances redden.

Tracker API

The nine trackers differ in mechanism — a Drop guard, a global recorder plus a manifest script, a t.Cleanup, an IDisposable — but the SPELLING is fixed here, because a convention whose name drifts per binding is how four treatments of one rule went unnoticed in the first place.

BindingDischargeFixture-end verification
lazily-rsexp.prose_key("clause", &["backends", "scenario_count"])expect::verify_prose(fixture), armed by a ProseLedger guard
lazily-pyprose_key(block, "clause", discharged_by=["backends"])verify_prose(fixture)
lazily-jsproseKey(block, "clause", ["backends"])verifyProse(fixture)
lazily-goproseKey(t, block, "clause", "backends")verifyProse(t, fixture), registered with t.Cleanup
lazily-dartproseKey(block, 'clause', dischargedBy: ['backends'])verifyProse(fixture), registered with addTearDown
lazily-ktproseKey("clause", listOf("backends"))verifyProse(fixture)
lazily-csProseKey("clause", "backends")VerifyProse(fixture)
lazily-cppblock.prose_key("clause", {"backends"})verify_prose(fixture)
lazily-zigproseKey("clause", &.{"backends"})verifyProse(fixture)

prose_key replaces whatever the binding did before — lazily-rs’s third prose() state goes away rather than gaining a sibling, and the free-text excuse_key reasons written for these keys are deleted, not kept alongside. Two paths to satisfy one key is the ambiguity this clause removes.

What this does not check, and what is still open

A discharge is checked for truth, not for sufficiency. Rule 6 proves the named key was asserted. Nothing proves it is relevant: discharging all nine of blob_backend_discriminator.json’s paragraphs with scenario_count alone would satisfy every rule. The excuse is now falsifiable, which it was not before, but it is not yet load-bearing — that judgement stays with review, which is why the discharge is written at the call site rather than in a table. Do not read a green run as “every paragraph is proven”; read it as “no paragraph is discharged by a claim the run contradicts”.

generator is provenance metadata, not an assertion. A generated fixture that records its source script places the path in the top-level generator field. It MUST NOT place that field under assertions: a replay cannot observe which script emitted its input, so an assertion tracker would have nothing executable to compare. Binding runners do not consume top-level provenance metadata.

The former wire_encoding gap is executable (#lzwireencodingrunner). check-prose-keys.mjs requires every wire_json to be raw parseable text, every wire_msgpack_hex to be even-length lowercase hex, both codecs to be represented, and every scenario’s expect.wire_input_fnv1a64 to match those exact bytes. Each binding then computes the same digest over the buffer it passes to its library decoder and asserts that key before decoding. A re-serialized value therefore changes the decoder-input witness instead of silently satisfying a proxy discharge.

What is checked where

HalfWhereWhat
Corpusscripts/check-prose-keys.mjs (make prose-keys-check)every paragraph declared, no stale or comparable entries, prose not self-listing, a declaring block carrying at least one non-prose key, no obligation hiding in a reserved annotation name
Bindingthe binding’s own assertion-key tracker, at runtimerules 1-7 above
Bindingthe binding’s coverage / ledger guard, beside the fixture-open and scenario-replay rungsrule 8

Rule 8’s row is separate on purpose. It cannot live in the test host: a test that never runs reports nothing, so the very run rule 8 exists to catch is the run that would have to report itself. It belongs where the other “did the suite actually do this?” rungs already live — the guard that reads the runtime manifest after the suite finishes.

The split is not arbitrary. Only the run knows which keys it asserted, so rule 6 cannot be checked from this repo; and only the corpus can settle which keys are prose, so leaving that to nine trackers is what produced four answers.

Object-valued assertion keys (#lzsubblockkeyset)

An assertion key whose value is a JSON object carries two obligations, not one: the value of each sub-field, and the sub-field key set. Bindings were discharging the first and not the second, which makes the object the null form one level down — a field added to it upstream is compared by nothing, and the fixture reports clean over the very change it exists to catch.

The failure this closes

arena_blob.json’s assertions.descriptor carries five sub-fields. Every binding that replayed it compared those five by name and stopped. Planting a sixth key inside the object left lazily-zig’s suite green while every scalar sibling in the same block reddened — found only by the corpus perturbation pass, because no rung above it can see inside a key it considers consumed. The consumption ledger saw descriptor read and asserted; the read-but-not-asserted rung saw an assertion; the bind ledger saw a bound block. All three were satisfied by a check that could not fail.

The rule

A binding’s assertion-key tracker MUST fail the run when an assertion key whose value is an object is consumed without its key set being checked. The tracker owns this, not the call site. Two ways to discharge it, and the tracker MUST recognise both:

  1. Descend. The tracker hands the caller a CHILD tracker bound to the object. The child owns the same unconsumed-key teardown the parent has, so a sub-field nothing reads fails exactly as an unconsumed top-level key does. This is the form to prefer: the obligation moves down rather than being restated.
  2. Compare the key set. The tracker compares the object’s key set against the set the run actually produced, in both directions — a fixture token nothing replayed and a replayed token the fixture omits are each failures. This is the form for a key whose sub-fields are a vocabulary rather than data: nodeid_exact_range.json’s assertions.outcomes maps outcome tokens to English glosses, and the assertion is which tokens exist, not what the glosses say.

Anything else — a plain value comparison, a hand-written field-by-field check, a count of sub-fields — leaves the run reporting nothing about a field the corpus grows later. A per-call-site field count is not conformance. It relies on the next author remembering, which is the property this rung exists to remove; it is at best a stopgap and MUST be replaced by 1 or 2.

A key the corpus declares in assertions.prose is out of scope here: a paragraph is a string, and prose nested inside a data key is not a prose key (see § Prose assertion keys).

The rule is not scoped to top-level assertions

It applies to every block a runner binds to its assertion-key trackerexpect, expected, per-step, per-scenario, per-frame — because the defect is identical wherever an object value is compared field by field. This matters because it is the one thing about this class that is consistently underestimated: a scan of top-level assertions blocks finds exactly two object-valued keys in the whole corpus, and the guard, once landed, found between 14 and 26 distinct key shapes per binding. Do not scope the audit from a corpus scan; land the guard and let it name the sites.

The recurring shapes, for orientation rather than as a checklist: invalidates, scopes, receipts, reads, subscriptions, values, handle_stable, observe, states, present, discovery, projection, frame, state_after, converged_nodes, text_on, version_vector_on, order_on, get / get_on, final_state, after_publish, authority, retry, dependents_of, readable, read. The ingress fixtures nest four levels deep (invalidates.scopes.<key>.<reader kind>), and every level is a key set.

What is checked where

HalfWhereWhat
Corpusthe fixture itselfwhich assertion keys have object values — the corpus decides, a binding never assumes
Bindingthe binding’s assertion-key tracker, at runtimean object-valued key consumed by neither 1 nor 2 fails the run

Neither half is provable from the other. The corpus cannot know whether a runner descended, and a runner cannot be trusted to notice that a value it compared happened to be an object.

Validating a tracker that claims to enforce this

The guard is only as good as its own falsifiability, and this whole rung exists because a check that cannot fail reads identically to one that passes. Plant an extra key inside each object-valued assertion value in a scratch copy of the corpus — never lazily-spec in place, where a probe reddens every other binding concurrently — and confirm the suite goes RED. A tracker that reports clean over a planted sub-field has not implemented this section, whatever its code says.

Assertion observation ordering (#lzassertordering)

An executable assertion has to remain reachable when the behavior it names is wrong. Two ordering rules follow:

  1. Fully evaluate and finish a fixture-owned assertion block, including its prose obligations, before applying runner-side coverage floors. A redundant floor may still guard the runner, but it must not preempt the fixture assertion that owns the falsifying corpus mutation.
  2. Evaluate an assertion about a run only after performing that run, and compare the declared rule with an observed runtime outcome. Checking a label or literal before ingest, decode, replay, or dispatch is not an assertion of the behavior the label names.

The semantic priority is therefore fixture assertion first, runner floor second. This matters for diagnostics as well as coverage: a mutated scenario_count must fail as scenario_count, not disappear behind an earlier hard-coded count.

scripts/check-assertion-ordering.py is the cross-binding static guard. Every binding invokes its own configured pass from make check; missing, duplicated, or renamed anchors fail closed. The guard checks order and attachment, not the sufficiency of the runtime comparison. Mutation probes remain required for the runtime assertion itself.

For distributed/anti_entropy_converge.json, resolution: max_stamp is tied to the runtime state selected after ingest. Its conflict witness delivers the greatest-stamp operation before a lower-stamp tail, so arrival-order resolution and max-stamp resolution produce different observations.

Keyed cell collections conformance

The conformance/collections/ directory contains canonical fixtures for the keyed cell collections layer, which is required of every binding (see the Binding Conformance Matrix). These are compute fixtures, not wire fixtures: a binding loads the initial state, replays each step’s op, and asserts the expected effects (resulting order, values, membership, and which reader classes — value / membership / order — invalidate). The reconciliation fixture is declarative: diff priortarget and assert the emitted minimal op set.

FixtureCovers
collections/cellmap_independence.jsonvalue / set-membership / order reactivity independence
collections/cellmap_atomic_move.jsonatomic ordered move keeps handle, bumps order once
collections/dependency_reactive_availability.jsonexact-key observation before publication, unrelated-key isolation, stable unavailable/available identity
collections/keyed_reconciliation_lis.jsonLIS move-minimized reconciliation; stable entries not invalidated
collections/semtree_incremental.jsonmemoized semantic tree: ancestor-chain-only recompute, sibling isolation, memo guard
collections/seqcrdt_convergence.jsonmove-aware sequence CRDT: single-LWW move, concurrent-move/value-edit independence, tombstone convergence, commutativity
collections/mergecell_algebra.jsonSource<T, M> merge algebra (#relaycell): KeepLatest/Sum/Max policies; per-op converged value + whether ⊕(old,op)==old suppresses the cascade (idempotent/identity no-op = free dedup); Cell ≡ Source<KeepLatest>
collections/textcrdt_convergence.jsonFugue/RGA character CRDT: concurrent same-point inserts, sticky tombstone, commutative/idempotent merge, GC
collections/textcrdt_delta_sync.jsonTextCrdt delta sync (#lztextsync): version_vector (insert + tombstone ids), delta_since / apply_delta; bidirectional exchange convergence, whole-snapshot fork identity preservation, idempotent apply
collections/stableid_alignment.jsonmanufactured text identity: anchors / content hashes / word-LCS similarity alignment
collections/workqueue_competing_delivery.jsoncompeting-consumer exclusive FIFO claims, delivery ownership, ack/nack, identity-preserving redelivery
collections/workqueue_lease_deadletter.jsonstrict visibility-timeout boundary, at-least-once requeue, max-delivery poison routing to DLQ

ComputedMap materialization conformance

The conformance/materialization/ directory pins the eager-vs-lazy materialization behavior (#lzmatmode) of a ComputedMap — eager is a pre-mint loop over the keyset; lazy is get_or_insert_with mint-on-access — proved in lazily-formal’s Materialization module. These are compute fixtures: a binding reads the spec (each key’s canonical value, and — for the mixed fixture — its cell/slot entry kind), builds the keyed map under both strategies, replays the reads sequence against the lazy build, and asserts observational transparency plus the memory / entry-kind laws. Because materialization is not observable on the value axis, there is no wire schema — only the compute effects below.

FixtureCovers
materialization/observational_transparency.jsonidentical observe values under eager vs. lazy; eager materializes all keys; lazy materializes only read keys; default mode eager (observe_canonical, eager_materializes_all, lazy_defers_slots, default_mode_eager)
materialization/deferral_not_deallocation.jsonlazy present set grows monotonically and is unchanged by re-reads; final lazy set is a subset of the eager set; no churn from allocation (materialize_present_monotone, lazy_present_subset_eager, materialize_preserves_observe)
materialization/entry_kind_orthogonal_to_mode.jsonentry kind ⟂ mode: cell (input) entries are present under either mode; slot (derived) entries are deferred under lazy until read (cell_entries_materialized_in_every_mode, slot_entries_deferred_under_lazy)

Reactive graph disposal conformance

The conformance/reactive-graph/ directory pins disposal and teardown scopes (#lzspecedgeindex) — the explicit-lifetime half of the graph contract, whose scope law is proved as disposeScope_eq_disposeAll in lazily-formal’s Reactive module. These are compute fixtures: a binding builds the graph by replaying each step’s op against a fresh Context and asserts that step’s expect. Disposal is not observable on the wire — it changes what a context holds and what a publish reaches, never a serialized frame — so there is no wire schema, only the compute effects below.

Every assertion is on observable state: dependent/dependency-set sizes, whether a node is readable, a read’s value or error, and which effects ran on a publish. Nothing here fixes a promotion threshold, a hash strategy, or an index layout — those are explicitly implementation-free per the implementation note, and a binding that dedups edges by linear scan at every degree conforms exactly as well as one that promotes to a hash index.

Op vocabulary (all ids are fixture-local labels, never a binding’s internal id):

OpMeaning
cell {id, value}Create a source cell
computed {id, reads[], offset, scope?}Create a derived, guarded Computed whose value is sum(reads) + offset; owned by scope when named
effect {id, reads[], scope?}Register an effect over reads; runs on creation and on each flush after a tracked invalidation
read {id}Read a node — expect.value, or expect.error for a disposed one
set {id, value}Publish; expect.observed_by names the effects that ran, expect.observed_count their number
dispose {id}Dispose one node, dispatching on its own kind
fanout {id_prefix, reads[], count, read_each} / dispose_fanout {id_prefix, count}Create / dispose count sibling readers, for widths a literal step list would bloat
churn {source, id_prefix, live_width, cycles, mode, read_each}Run cycles subscribe/unsubscribe cycles holding live_width subscribers live (mode: dispose_then_create or scope_per_cycle)
begin_scope {scope} / end_scope {scope}Open / end a teardown scope
disarm {scope}Cancel the scope’s teardown — ending it then disposes nothing
dispose_stale_handle {handle_of, handle_kind}Dispose through a handle whose id may have been recycled; a no-op unless the id still names a node of handle_kind
subscribe {id, cell, callback?, on_notify?, on_notify_once?}Register a Cell observer labelled id on cell. callback names a shared callable, so two registrations naming the same callback subscribe the same function (default: the callback is unique to id). on_notify is a list of subscribe/unsubscribe ops the callback performs reentrantly when it runs; on_notify_once restricts them to the first invocation. A reentrant subscribe may use id_prefix instead of id, minting <prefix>_0, <prefix>_1, … per invocation
unsubscribe {id, times?}Invoke the disposer returned by subscribe {id}, times times (default 1) — repeat calls exercise idempotency

A fixture uses top-level steps, or scenarios plus expected when the claim is that two differently-built runs agree (expected.observationally_equal).

FixtureCovers
reactive-graph/dispose_detaches_edges_both_directions.jsondisposal detaches upstream and downstream edges; a publish to a former source does not reach the disposed node; the surviving source is unaffected
reactive-graph/read_after_dispose_is_an_error.jsonreading a disposed Computed, a disposed source cell, or through a live reader that names one is an error — never a stale or default value; double-dispose is an idempotent no-op
reactive-graph/recycled_id_inherits_nothing.jsona node minted on a recycled id starts with an empty edge set in both directions — the owner-keyed-side-table aliasing hazard; a stale cross-kind handle disposes nothing
reactive-graph/scope_teardown_equals_fold_of_disposals.jsonending a scope is observationally equal to disposing each member individually (disposeScope_eq_disposeAll), including reverse-creation-order cleanup
reactive-graph/scoping_bounds_teardown_not_visibility.jsona scope’s nodes read parent- and sibling-owned nodes freely in every direction; propagation crosses scope boundaries unchanged
reactive-graph/disarm_disposes_nothing.jsondisarm() leaves node state untouched — the nodes stay readable, keep propagating, and stay individually disposable; ending the scope disposes nothing
reactive-graph/cross_scope_teardown_hazard.jsonending a scope tears down its nodes even when a node outside still reads them (required failure — a binding that keeps them alive is non-conforming); the mirror case is symmetric
reactive-graph/churn_returns_to_baseline.jsona subscribe/unsubscribe cycle that disposes what it creates leaves the source’s dependent set at its starting size, under both individual disposal and one scope per cycle

Cell observer conformance (#lzdartobservercow)

The same directory pins observer semantics — the hand-registered Cell.subscribe callbacks, which are a separate mechanism from the tracked dependency edges above. Same compute-fixture shape, same op replay, two further assertion keys:

AssertionMeaning
observed_orderThe exact sequence of observer labels invoked by this step
observed_countsPer-observer invocation counts for this step, where a shared callback runs more than once

observed_order is deliberately a sequence and not a set. An unordered observer collection satisfies a set-valued observed_by while firing in a fresh order on every notification, which is precisely how the family’s divergence went unnoticed — so an observer fixture that asserts only observed_by is rejected by the structural guard.

These fixtures fail against some bindings today, by design. The observer contract was unwritten until now and four bindings answered it differently; the fixtures encode the family position, and the gaps are listed as known divergences with migrations. A red result here is a binding bug, not a fixture bug.

FixtureCovers
reactive-graph/observer_order_is_registration_order.jsonobservers fire in registration order, stably across notifications; a removal closes the gap without reordering survivors; a re-registered callback appends rather than resuming its old position; the == guard still suppresses the notification entirely (fails: py, zig)
reactive-graph/observer_duplicate_registrations_are_independent.jsonsubscribing one callback twice yields two registrations that both fire and dispose independently — no dedup by identity or by equality (fails: py, zig)
reactive-graph/observer_subscribe_during_notify_is_deferred.jsonan observer registered mid-notification first runs on the next one, including when observers registered earlier are still unvisited; a self-feeding subscriber terminates because the pass is bounded by the pre-callback count
reactive-graph/observer_unsubscribe_during_notify_takes_effect_immediately.jsonan observer disposed mid-notification does not run in that pass even when unvisited; an already-visited observer’s invocation stands; self-unsubscribe completes the call it is in; the tail observer still runs, catching a swap-remove under a live cursor (fails: dart, go)
reactive-graph/observer_disposer_is_idempotent.jsona disposer latches — repeat calls are silent no-ops that remove nothing else, and a spent disposer never reaches a later registration of an equal callable; cell teardown drops observers without invoking them, and a disposer outliving its cell is a no-op

Signaling conformance

The conformance/signaling/ directory pins the WebSocket signaling wire protocol (see protocol.md § Signaling Protocol), the cross-language contract shared by every distributed-plane binding and the reference TypeScript signaling server.

  • signaling/frames.json is a wire fixture: each entry’s wire field is the canonical JSON for one ClientMessage / ServerMessage variant and validates against schemas/signaling.json. A binding encodes its typed message to the same JSON and decodes the JSON back. Tags are kebab-case (peer-joined, peer-left); peer ids are bare numbers ≤ 2⁵³−1. Client-directed frames carry to; server-forwarded frames carry a server-stamped from.
  • signaling/anti_spoof_session.json is a compute fixture: a binding that implements the server room replays each input and asserts the emitted expect frames. It pins the load-bearing invariants — the welcome roster excludes the joining peer, a forwarded frame’s from is the sender’s server-registered id (never client-supplied), and an unknown target yields an error frame. Its three-peer join makes roster sorting observable: the third welcome must be [1, 2], not reverse order [2, 1]. Peer-joined broadcast order remains outside this fixture’s claim.

Distributed (CRDT plane) conformance

The conformance/distributed/ directory pins the CRDT anti-entropy plane (see protocol.md § Distributed).

  • distributed/crdt_sync_frames.json is a wire fixture: each wire is a {"CrdtSync": {frontier, ops}} envelope validating against schemas/distributed.json (empty, keyed+keyless ops, and a multi-peer frontier).
  • distributed/anti_entropy_converge.json is a compute fixture: a binding replays each scenario’s ops through its CrdtPlaneRuntime and asserts convergence to expect.converged independent of delivery order, plus state-based idempotence (re-ingesting a seen frame applies 0 new ops). It models LWW cells where the plane WireStamp is the decisive stamp under lexicographic (wall_time, logical, peer) order.

Lossless tree conformance

The conformance/lossless-tree/ directory pins the lossless full-document tree CRDT (see Lossless Tree CRDT, #lzlosstree). These are compute fixtures with the same {scenarios: [{seed, steps, expect}]} shape as the collections fixtures: a binding builds the seed.tree on replica a (addressing nodes by stable string labels), replays each step (fork / clone / sync / deliver an op subset / on a replica an op — create, edit_leaf, split, merge_leaves, reorder, tombstone), and asserts the expect fields: render / render_on (exact rendered text per replica), live_nodes (live element+leaf count, excluding the root), and converged (a set of replicas that must render identically). Byte offsets in ops (at_byte) are UTF-8 and must land on a char boundary.

The step vocabulary itself is schema’d by schemas/lossless-tree-fixture.json (compute, not wire), so a step form the corpus can express is a step every runner has been told how to read. Two properties of the schedule are load-bearing and easy to miss:

  • A mutation step may follow a sync/deliver into the same replica. There is no fork → edit → sync phase structure; steps are simply applied in order. A post-sync mutation is the only way to observe that ingest advanced the ingesting replica’s Lamport counter, because a replica that only mutates before its first ingest never has to mint an id that outranks an ingested stamp.
  • deliver carries exactly one selector, only or order — they are mutually exclusive and are never composed. Both are 0-based indexes into the canonically ordered diff (diff(to.frontier()) on from, sorted by dotted (counter, peer) id), must be distinct, and must be in range; a runner fails the fixture rather than clamping. only delivers that subset in canonical order however it is listed — a delivery hole, where the omitted dots stay missing and re-requestable. order delivers the listed entries in the listed sequence, as one apply_update batch: a runner must not re-sort them and must not split them across calls, because the point is to hand a replica an op whose parent/target or prev has not arrived yet and require it to buffer and retry rather than drop. order need not be a permutation of the whole diff. Composing the two is disallowed on purpose: nothing would say whether order’s indexes address the diff or the only subset, and ten runners would answer that ten ways.
FixtureCovers
lossless-tree/exact_roundtrip.jsonToken/Trivia/Raw/Error leaves incl. an invalid span + multi-byte text; render == source
lossless-tree/one_leaf_edit_delta.jsonone-leaf edit at a UTF-8 byte offset in multi-byte text, delivered by anti-entropy
lossless-tree/split_merge.jsonsplit a leaf then merge back; render preserved; live-node count grows then restores
lossless-tree/concurrent_insert_same_parent.jsontwo replicas insert into the same gap; both survive, deterministic order, converge
lossless-tree/concurrent_reorder_and_leaf_edit.jsonconcurrent reorder + text edit both apply (position/text are independent registers)
lossless-tree/non_contiguous_anti_entropy.jsona delivery hole is representable in the dotted frontier, re-requested, and converges
lossless-tree/token_trivia_preservation.jsona leaf edit leaves adjacent Token/Trivia leaves byte-for-byte unchanged
lossless-tree/invalid_source_roundtrip.jsonunclosed fence/comment kept as Error leaves round-trips exactly; editing an adjacent Raw leaf keeps the Error spans
lossless-tree/concurrent_conflict_preserves_text.jsonincompatible concurrent shapes both survive with no bytes dropped (text preservation wins over semantic shape)
lossless-tree/apply_update_advances_counter.jsoningest advances the Lamport counter, so a write minted AFTER a sync outranks the stamps that sync delivered (post-sync mutation step)
lossless-tree/out_of_order_delivery_buffers.jsona reversed delivery batch (deliver.order) drains through the dependency buffer; a dropped op is never repaired by a later sync

The op-delta wire form additionally validates against schemas/lossless-tree.json (vocabulary) + schemas/lossless-tree-delta.json (the TreeUpdate message); the Rust reference and each port validate their serialized TreeUpdate against these.

Causal receipt conformance

The conformance/receipts/ directory pins lazily’s generic outcome vocabulary for commands and effect requests (see protocol.md § Causal Receipts).

  • receipts/causal_receipts.json is a wire + compute fixture: its wire field validates against schemas/receipts.json, and bindings replay the receipts into their projection. The stale-generation receipt is ignored by the current projection; observed / accepted remain non-terminal; applied is the terminal outcome.

Examples

snapshot_minimal.json

{
  "description": "Minimal snapshot with one payload node and no edges",
  "protocol_version": 1,
  "kind": "Snapshot",
  "assertions": {
    "epoch": 1,
    "node_count": 1,
    "edge_count": 0,
    "root_count": 1,
    "first_node_type_tag": "i32"
  },
  "wire": {
    "Snapshot": {
      "epoch": 1,
      "nodes": [
        {
          "node": 1,
          "type_tag": "i32",
          "state": {
            "Payload": [1, 2, 3, 4]
          }
        }
      ],
      "edges": [],
      "roots": [1]
    }
  }
}

snapshot_multi_node.json

{
  "description": "Snapshot with payload nodes, opaque node, edges, and roots",
  "protocol_version": 1,
  "kind": "Snapshot",
  "assertions": {
    "epoch": 7,
    "node_count": 3,
    "edge_count": 2,
    "root_count": 2,
    "has_opaque_node": true,
    "opaque_node_id": 3
  },
  "wire": {
    "Snapshot": {
      "epoch": 7,
      "nodes": [
        {
          "node": 1,
          "type_tag": "i32",
          "state": {
            "Payload": [1, 2, 3]
          }
        },
        {
          "node": 2,
          "type_tag": "f64",
          "state": {
            "Payload": [0, 0, 0, 0, 0, 0, 240, 63]
          }
        },
        {
          "node": 3,
          "type_tag": "opaque-type",
          "state": "Opaque"
        }
      ],
      "edges": [
        { "dependent": 2, "dependency": 1 },
        { "dependent": 3, "dependency": 1 }
      ],
      "roots": [1, 2]
    }
  }
}

snapshot_shared_blob.json

{
  "description": "Snapshot with a shared-blob node referencing shared memory",
  "protocol_version": 1,
  "kind": "Snapshot",
  "assertions": {
    "epoch": 9,
    "node_count": 1,
    "edge_count": 0,
    "root_count": 1,
    "first_node_state_kind": "SharedBlob",
    "blob_offset": 0,
    "blob_len": 16,
    "blob_epoch": 9
  },
  "wire": {
    "Snapshot": {
      "epoch": 9,
      "nodes": [
        {
          "node": 7,
          "type_tag": "text/plain",
          "state": {
            "SharedBlob": {
              "offset": 0,
              "len": 16,
              "generation": 1,
              "epoch": 9,
              "checksum": 123456789
            }
          }
        }
      ],
      "edges": [],
      "roots": [7]
    }
  }
}

delta_sequential.json

{
  "description": "Sequential delta covering all 7 DeltaOp variants",
  "protocol_version": 1,
  "kind": "Delta",
  "assertions": {
    "base_epoch": 40,
    "epoch": 41,
    "is_sequential": true,
    "op_count": 7,
    "has_all_op_variants": true
  },
  "wire": {
    "Delta": {
      "base_epoch": 40,
      "epoch": 41,
      "ops": [
        { "CellSet": { "node": 1, "payload": { "Inline": [10] } } },
        { "SlotValue": { "node": 2, "payload": { "Inline": [20] } } },
        { "Invalidate": { "node": 3 } },
        {
          "NodeAdd": {
            "node": 4,
            "type_tag": "u64",
            "state": { "Payload": [64] }
          }
        },
        { "NodeRemove": { "node": 5 } },
        { "EdgeAdd": { "dependent": 2, "dependency": 1 } },
        { "EdgeRemove": { "dependent": 3, "dependency": 1 } }
      ]
    }
  }
}

delta_non_sequential.json

{
  "description": "Non-sequential delta with gap (requires resync)",
  "protocol_version": 1,
  "kind": "Delta",
  "assertions": {
    "base_epoch": 12,
    "epoch": 13,
    "is_sequential": true,
    "resync_after_epoch_10": true
  },
  "wire": {
    "Delta": {
      "base_epoch": 12,
      "epoch": 13,
      "ops": []
    }
  }
}

delta_shared_blob.json

{
  "description": "Delta with shared-blob payload referencing shared memory",
  "protocol_version": 1,
  "kind": "Delta",
  "assertions": {
    "base_epoch": 8,
    "epoch": 9,
    "op_count": 1,
    "first_op_kind": "SlotValue",
    "first_op_payload_kind": "SharedBlob"
  },
  "wire": {
    "Delta": {
      "base_epoch": 8,
      "epoch": 9,
      "ops": [
        {
          "SlotValue": {
            "node": 7,
            "payload": {
              "SharedBlob": {
                "offset": 40,
                "len": 17,
                "generation": 2,
                "epoch": 9,
                "checksum": 987654321
              }
            }
          }
        }
      ]
    }
  }
}

Portable standard-library primitives

This document specifies the language-neutral Timer, Timeout<T>, and RevisionBarrier convenience APIs. Bindings may expose idiomatic names and types, but an implementation earns the corresponding coverage feature only by replaying the canonical fixtures against its public production API.

The three versioned features are:

  • stdlib_timer_v1
  • stdlib_timeout_v1
  • stdlib_revision_barrier_v1

All time in this contract is monotone logical time. Production implementations normally read a platform monotone clock; conformance runners inject a clock and wait driver and must never sleep or depend on wall-clock scheduling.

Shared rules

  • Logical instants and durations are unsigned 64-bit integers.
  • start + duration is checked. Overflow returns typed Unavailable and never wraps.
  • A clock observation lower than the most recently accepted observation returns Unavailable(clock_regression) without changing the stored state.
  • A deadline is inclusive: now >= deadline reaches the deadline.
  • The first terminal result is latched. Later reads return the same result and invoke no caller-supplied adapters.
  • The adapter owns no executor, async runtime, or thread. Waiting is driven by a caller-owned clock/wait seam.

Timer

Starting a timer at start for duration produces a pending timer whose deadline is start + duration. A zero duration therefore fires on the first observation at start.

observe(now) has two non-error observations:

  • Pending(deadline) when now < deadline;
  • Fired(fired_at) when now >= deadline.

The first firing latches its actual observation instant as fired_at; later observations preserve that value. A timer cannot return to pending.

Timeout<T>

Timeout<T> is a caller-driven adapter around an operation probe and a cancellation probe. It is not a future executor and it is not the reactive TimeoutCell. TimeoutCell emits a reactive timeout edge; this adapter resolves a single operation to one of:

  • Completed(T)
  • TimedOut
  • Cancelled
  • Unavailable(reason)

Each nonterminal poll(now, operation, cancellation) uses this precedence:

  1. Reject a regressed clock without polling either adapter.
  2. If now >= deadline, return TimedOut without polling either adapter.
  3. Sample the operation and cancellation adapters once each.
  4. A completed operation wins, including when cancellation is observed in the same poll.
  5. An unavailable operation returns Unavailable(operation_unavailable).
  6. Cancellation returns Cancelled; an unavailable cancellation adapter returns Unavailable(cancellation_unavailable).
  7. Otherwise remain pending.

This order makes exact-deadline behavior and simultaneous completion/cancellation deterministic in every binding.

RevisionBarrier

A revision barrier waits for both:

  1. current_revision >= required_revision; and
  2. a caller-supplied derived predicate to be true.

Revisions never decrease. Registration increments a wake generation, then the waiter re-reads the revision and predicate after registration. This register-then-recheck rule closes the check-to-sleep race: an advance that lands between the initial check and waiter registration is observed immediately.

A waiter may have an inclusive deadline and a cancellation adapter. Deadline dominance is checked before cancellation. Its terminal outcomes are Satisfied(revision), TimedOut, Cancelled, Disposed, and Unavailable(reason). Disposal wakes all waiters and latches.

Effect receipts and transport acknowledgements are application-owned observations. Recording a receipt cannot advance a barrier revision, change its wake generation, or satisfy a waiter.

Canonical fixtures and mutation gates

The canonical corpus is under conformance/stdlib/ and validates against schemas/stdlib-fixture.schema.json. Each family declares a scenario and assertion floor. A runner that opens zero fixtures, executes fewer than the declared floor, or merely echoes fixture expectations fails conformance.

The spec test runner also applies named mutations. At minimum it proves that the corpus detects:

  • changing >= to > at a deadline;
  • checking cancellation before pre-deadline completion;
  • failing to latch a terminal result;
  • omitting the barrier’s post-registration recheck;
  • treating an effect receipt as barrier authority; and
  • replacing production transitions with fixture bookkeeping.

Bindings may advertise a feature to the peer suite only after their public API passes the matching canonical family and its mutation checks. Unsupported bindings stay visible as staged and receive no pass credit.

JSON Schemas

Schemas are provided as JSON Schema (Draft 2020-12). Each implementation must validate against these schemas. The JSON representation of the wire format is normative; future binary codecs encode the same shapes.

SchemaLayer
defs.jsonShared wire primitives (NodeId, NodeKey, NodeState, IpcValue, ShmBlobRef, WireStamp)
snapshot.jsonIPC — Snapshot message (externally-tagged {"Snapshot": …} envelope)
delta.jsonIPC — Delta message (externally-tagged {"Delta": …}, all 7 DeltaOp variants)
ffi.jsonCross-language FFI boundary
signaling.jsonSignaling (WebSocket)
distributed.jsonDistributed — CrdtSync message ({"CrdtSync": …}) + CRDT/cell-model types
receipts.jsonCausal receipts ({"CausalReceipts": …}) + terminal outcome projection
message-passing.jsonCommand / RPC message plane (CommandSubmit / CommandCancel / CommandEvents / CommandProjection)
statechart.jsonCompute (Harel/SCXML chart form — not a wire message)
stdlib-fixture.schema.jsonDeterministic Timer, Timeout, and RevisionBarrier conformance scenarios
assertion-blocks.jsonRouted, fail-closed schemas for every canonical assertions / expect / expected object

The IPC schemas describe the normative externally-tagged envelope that every binding serializes (the single-key {"Snapshot": …} / {"Delta": …} / {"CrdtSync": …} form), with node addressing and value bytes as JSON arrays of u8 (not base64). Shared wire primitives live in defs.json and are referenced via absolute $ref so the primitive definitions never copy-drift. Every conformance fixture’s wire field validates against its schema — enforced by make test-schemas (see tests/test_schema_conformance.py).

Assertion blocks are validated independently of their fixture’s wire or compute schema, so families without a whole-fixture schema still reject unknown claim keys and stale value shapes. See Assertion-Block Schemas.

defs.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/defs.json",
  "title": "Shared wire primitive definitions",
  "description": "Primitive types shared across the lazily IPC schemas (snapshot, delta, distributed). Referenced via absolute $ref so every wire schema stays codec-faithful to protocol.md without copy-drift.",
  "$defs": {
    "NodeId": {
      "type": "integer",
      "minimum": 0,
      "maximum": 18446744073709551615,
      "description": "Wire-stable node identifier (u64). The full u64 range is wire-valid; the 2^53-1 bound is a PRODUCER obligation on peers whose runtime represents integers as IEEE-754 doubles, and a decoder that cannot represent a received value exactly MUST reject the frame rather than round it (protocol.md § NodeId / PeerId, #lzspecdecoderbound). Schema validity is therefore u64-wide — a narrower binding refuses at decode, not here."
    },
    "PeerId": {
      "type": "integer",
      "minimum": 0,
      "maximum": 18446744073709551615,
      "description": "Wire-stable peer identifier (u64). Same producer bound and decoder obligation as NodeId (#lzspecdecoderbound). This carried maximum 9007199254740991 until that audit, which contradicted both the u64 wire type and protocol.md's producer-only wording."
    },
    "NodeKey": {
      "type": "string",
      "minLength": 1,
      "maxLength": 1024,
      "pattern": "^[^/]+(/[^/]+)*$",
      "description": "Optional wire-stable keyed address: a '/'-joined path (e.g. 'scores/alice'). The pattern rejects empty/leading/trailing/double slashes; construction-time code also enforces the ≤1024 *byte* and ≤32 *segment* bounds (see protocol.md § NodeKey). Serialized as a bare string; the field is omitted in self-describing codecs when absent."
    },
    "ShmBlobRef": {
      "type": "object",
      "description": "Descriptor into a blob backend (zero-copy transport). The standard fields locate and integrity-check a byte range within the backend's resolved buffer; `backend` selects which pluggable backend resolves it ('shm' = POSIX shared memory, the default for backward compatibility; 'arrow' = Apache Arrow IPC/Flight; 'in_process' = an in-process arena). See protocol.md § Zero-copy transport. `backend` is optional and defaults to 'shm' so legacy descriptors validate unchanged.",
      "required": ["offset", "len", "generation", "epoch", "checksum"],
      "additionalProperties": false,
      "properties": {
        "offset": { "type": "integer", "minimum": 0 },
        "len": { "type": "integer", "minimum": 0 },
        "generation": { "type": "integer", "minimum": 0 },
        "epoch": { "type": "integer", "minimum": 0 },
        "checksum": { "type": "integer", "minimum": 0 },
        "backend": { "type": "string", "enum": ["shm", "arrow", "in_process"], "default": "shm", "description": "Which pluggable backend holds the blob. Optional; defaults to 'shm' (POSIX shared memory) for backward compatibility. The receiver routes resolution by this discriminator." }
      }
    },
    "NodeState": {
      "description": "Externally-tagged node body: concrete bytes, a shared-memory value, or an opaque (visible but non-serializable) node.",
      "oneOf": [
        {
          "type": "object",
          "title": "Payload",
          "required": ["Payload"],
          "additionalProperties": false,
          "properties": {
            "Payload": {
              "type": "array",
              "items": { "type": "integer", "minimum": 0, "maximum": 255 },
              "description": "Serialized value bytes as JSON array of u8 (NOT base64)."
            }
          }
        },
        {
          "type": "object",
          "title": "SharedBlob",
          "required": ["SharedBlob"],
          "additionalProperties": false,
          "properties": { "SharedBlob": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/ShmBlobRef" } }
        },
        { "type": "string", "title": "Opaque", "const": "Opaque" }
      ]
    },
    "IpcValue": {
      "description": "Externally-tagged delta payload: inline bytes or a shared-memory blob reference.",
      "oneOf": [
        {
          "type": "object",
          "title": "Inline",
          "required": ["Inline"],
          "additionalProperties": false,
          "properties": {
            "Inline": {
              "type": "array",
              "items": { "type": "integer", "minimum": 0, "maximum": 255 },
              "description": "Inline serialized bytes as JSON array of u8 (NOT base64)."
            }
          }
        },
        {
          "type": "object",
          "title": "SharedBlob",
          "required": ["SharedBlob"],
          "additionalProperties": false,
          "properties": { "SharedBlob": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/ShmBlobRef" } }
        }
      ]
    },
    "WireStamp": {
      "type": "object",
      "description": "Wire mirror of the runtime HLC stamp — a total order (wall_time, logical, peer).",
      "required": ["wall_time", "logical", "peer"],
      "additionalProperties": false,
      "properties": {
        "wall_time": { "type": "integer", "minimum": 0, "description": "Wall-clock microseconds since the Unix epoch." },
        "logical": { "type": "integer", "minimum": 0, "description": "Logical counter advancing causality within equal wall_time." },
        "peer": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/PeerId", "description": "Originating peer; final tiebreak so equal (wall, logical) is a total order." }
      }
    }
  }
}

snapshot.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/snapshot.json",
  "title": "Snapshot",
  "description": "Full graph state IPC message. Normative form (protocol.md § IPC) is the EXTERNALLY-TAGGED envelope {\"Snapshot\": {...}} — not a \"type\" discriminant. Serialized value bytes are JSON arrays of u8, not base64. A NodeSnapshot's optional `key` (NodeKey) is omitted when absent.",
  "type": "object",
  "required": ["Snapshot"],
  "additionalProperties": false,
  "properties": {
    "Snapshot": {
      "type": "object",
      "required": ["epoch", "nodes", "edges", "roots"],
      "additionalProperties": false,
      "properties": {
        "epoch": { "type": "integer", "minimum": 0, "description": "Current IPC epoch." },
        "nodes": { "type": "array", "items": { "$ref": "#/$defs/NodeSnapshot" }, "description": "All serialized nodes." },
        "edges": { "type": "array", "items": { "$ref": "#/$defs/EdgeSnapshot" }, "description": "Dependency edges (dependent → dependency)." },
        "roots": { "type": "array", "items": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" }, "description": "Cell and source slot ids." }
      }
    }
  },
  "$defs": {
    "NodeSnapshot": {
      "type": "object",
      "description": "Full state for one node in a snapshot.",
      "required": ["node", "type_tag", "state"],
      "additionalProperties": false,
      "properties": {
        "node": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" },
        "type_tag": { "type": "string", "description": "Stable cross-process type key for decoding state." },
        "state": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeState" },
        "key": {
          "description": "Optional wire-stable keyed address. A conforming ENCODER omits it when absent; an explicit null is also wire-valid and a decoder MUST read both forms as absent (protocol.md § NodeKey, #lzkeynullstrict). This carried a bare NodeKey $ref until that audit, which made a frame the spec now requires decoders to accept schema-invalid.",
          "oneOf": [
            { "type": "null" },
            { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeKey" }
          ]
        }
      }
    },
    "EdgeSnapshot": {
      "type": "object",
      "description": "Directed dependency edge (dependent → dependency).",
      "required": ["dependent", "dependency"],
      "additionalProperties": false,
      "properties": {
        "dependent": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" },
        "dependency": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" }
      }
    }
  }
}

delta.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/delta.json",
  "title": "Delta",
  "description": "Incremental change-set IPC message. Normative form (protocol.md § IPC) is the EXTERNALLY-TAGGED envelope {\"Delta\": {...}} — not a \"type\" discriminant. Each DeltaOp variant is itself externally tagged by its PascalCase name (single-key object). Serialized value bytes are JSON arrays of u8, not base64. NodeAdd's optional `key` (NodeKey) is omitted in JSON when absent.",
  "type": "object",
  "required": ["Delta"],
  "additionalProperties": false,
  "properties": {
    "Delta": {
      "type": "object",
      "required": ["base_epoch", "epoch", "ops"],
      "additionalProperties": false,
      "properties": {
        "base_epoch": { "type": "integer", "minimum": 0, "description": "Epoch this delta applies to (must equal the receiver's last_epoch)." },
        "epoch": { "type": "integer", "minimum": 0, "description": "New epoch, >= base_epoch + 1. Usually base_epoch + 1; a multi-epoch-span delta has epoch > base_epoch + 1 (epoch - base_epoch = accepted-event span). See protocol.md § Multi-epoch-span delta." },
        "ops": { "type": "array", "items": { "$ref": "#/$defs/DeltaOp" }, "description": "Ordered operations." }
      }
    }
  },
  "$defs": {
    "DeltaOp": {
      "description": "One incremental graph mutation. Externally tagged: a single-key object whose key is the PascalCase variant name.",
      "oneOf": [
        {
          "type": "object",
          "title": "CellSet",
          "required": ["CellSet"],
          "additionalProperties": false,
          "properties": { "CellSet": { "$ref": "#/$defs/CellSetBody" } }
        },
        {
          "type": "object",
          "title": "SlotValue",
          "required": ["SlotValue"],
          "additionalProperties": false,
          "properties": { "SlotValue": { "$ref": "#/$defs/SlotValueBody" } }
        },
        {
          "type": "object",
          "title": "Invalidate",
          "required": ["Invalidate"],
          "additionalProperties": false,
          "properties": { "Invalidate": { "$ref": "#/$defs/NodeBody" } }
        },
        {
          "type": "object",
          "title": "NodeAdd",
          "required": ["NodeAdd"],
          "additionalProperties": false,
          "properties": { "NodeAdd": { "$ref": "#/$defs/NodeAddBody" } }
        },
        {
          "type": "object",
          "title": "NodeRemove",
          "required": ["NodeRemove"],
          "additionalProperties": false,
          "properties": { "NodeRemove": { "$ref": "#/$defs/NodeBody" } }
        },
        {
          "type": "object",
          "title": "EdgeAdd",
          "required": ["EdgeAdd"],
          "additionalProperties": false,
          "properties": { "EdgeAdd": { "$ref": "#/$defs/EdgeBody" } }
        },
        {
          "type": "object",
          "title": "EdgeRemove",
          "required": ["EdgeRemove"],
          "additionalProperties": false,
          "properties": { "EdgeRemove": { "$ref": "#/$defs/EdgeBody" } }
        },
        {
          "type": "object",
          "title": "QueuePush",
          "description": "Op-log delta form for a QueueCell node: append `payload` to the tail. The op-log delta is the incremental counterpart to the storage-snapshot form (cell-model.md § Wire and snapshot shape); a run of same-direction QueuePush ops fuses into one multi-epoch batch Delta (protocol.md § Backpressure & outbox coalescing).",
          "required": ["QueuePush"],
          "additionalProperties": false,
          "properties": { "QueuePush": { "$ref": "#/$defs/CellSetBody" } }
        },
        {
          "type": "object",
          "title": "QueuePop",
          "description": "Op-log delta form for a QueueCell node: remove the head. Carries no value (the popped element is determined by ordered replay); a re-delivered pop is Ignored by the epoch gap rule. Crosses the wire only when consumption is authoritatively replicated (mirror sync); in the default producer→consumer sync the remote consumer's pops are local. A QueuePop/QueueClose is a fusion boundary (order-sensitive).",
          "required": ["QueuePop"],
          "additionalProperties": false,
          "properties": { "QueuePop": { "$ref": "#/$defs/NodeBody" } }
        },
        {
          "type": "object",
          "title": "QueueClose",
          "description": "Op-log delta form for a QueueCell node: mark closed (idempotent, terminal — Closed is distinct from Empty). A fusion boundary.",
          "required": ["QueueClose"],
          "additionalProperties": false,
          "properties": { "QueueClose": { "$ref": "#/$defs/NodeBody" } }
        }
      ]
    },
    "CellSetBody": {
      "type": "object",
      "required": ["node", "payload"],
      "additionalProperties": false,
      "properties": {
        "node": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" },
        "payload": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/IpcValue" }
      }
    },
    "SlotValueBody": {
      "type": "object",
      "required": ["node", "payload"],
      "additionalProperties": false,
      "properties": {
        "node": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" },
        "payload": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/IpcValue" }
      }
    },
    "NodeBody": {
      "type": "object",
      "required": ["node"],
      "additionalProperties": false,
      "properties": { "node": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" } }
    },
    "NodeAddBody": {
      "type": "object",
      "required": ["node", "type_tag", "state"],
      "additionalProperties": false,
      "properties": {
        "node": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" },
        "type_tag": { "type": "string" },
        "state": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeState" },
        "key": {
          "description": "Optional wire-stable keyed address. A conforming ENCODER omits it when absent; an explicit null is also wire-valid and a decoder MUST read both forms as absent (protocol.md § NodeKey, #lzkeynullstrict). This carried a bare NodeKey $ref until that audit, which made a frame the spec now requires decoders to accept schema-invalid.",
          "oneOf": [
            { "type": "null" },
            { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeKey" }
          ]
        }
      }
    },
    "EdgeBody": {
      "type": "object",
      "required": ["dependent", "dependency"],
      "additionalProperties": false,
      "properties": {
        "dependent": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" },
        "dependency": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" }
      }
    }
  }
}

ffi.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/ffi.json",
  "title": "FFI Types",
  "description": "C ABI types for the lazily FFI boundary",

  "$defs": {
    "LazilyFfiBytes": {
      "type": "object",
      "description": "Owned byte buffer crossing the FFI boundary",
      "properties": {
        "ptr": { "type": "integer", "minimum": 0, "description": "Pointer to byte buffer" },
        "len": { "type": "integer", "minimum": 0, "description": "Buffer length in bytes" }
      },
      "required": ["ptr", "len"],
      "additionalProperties": false
    },
    "LazilyFfiStatus": {
      "type": "integer",
      "enum": [0, 1, 2, 3, 4, 5],
      "description": "FFI operation status code",
      "oneOf": [
        { "const": 0, "title": "Ok" },
        { "const": 1, "title": "Empty" },
        { "const": 2, "title": "NullPointer" },
        { "const": 3, "title": "InvalidMessage" },
        { "const": 4, "title": "EncodeFailed" },
        { "const": 5, "title": "Panic" }
      ]
    },
    "LazilyFfiMessageKind": {
      "type": "integer",
      "enum": [0, 1, 2, 3, 4, 5],
      "description": "IPC message kind discriminator. 4/5 are the reliable-sync (#lzsync) reverse-channel control-frame variants ResyncRequest/OutboxAck (protocol.md § Reliable Sync).",
      "oneOf": [
        { "const": 0, "title": "Unknown" },
        { "const": 1, "title": "Snapshot" },
        { "const": 2, "title": "Delta" },
        { "const": 3, "title": "CrdtSync" },
        { "const": 4, "title": "ResyncRequest" },
        { "const": 5, "title": "OutboxAck" }
      ]
    }
  }
}

signaling.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/signaling.json",
  "title": "Signaling Protocol",
  "description": "WebSocket signaling frames for lazily peer discovery",
  "$comment": "Validates both client→server and server→client frames",

  "oneOf": [
    {
      "title": "ClientMessage",
      "oneOf": [
        {
          "type": "object",
          "title": "Join",
          "properties": {
            "type": { "const": "join" },
            "peer": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
            "capabilities": { "type": "array", "items": { "type": "string" } }
          },
          "required": ["type", "peer"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "Offer",
          "properties": {
            "type": { "const": "offer" },
            "to": { "type": "integer", "minimum": 0 },
            "sdp": { "type": "string" }
          },
          "required": ["type", "to", "sdp"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "Answer",
          "properties": {
            "type": { "const": "answer" },
            "to": { "type": "integer", "minimum": 0 },
            "sdp": { "type": "string" }
          },
          "required": ["type", "to", "sdp"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "Ice",
          "properties": {
            "type": { "const": "ice" },
            "to": { "type": "integer", "minimum": 0 },
            "candidate": { "type": "string" }
          },
          "required": ["type", "to", "candidate"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "Relay",
          "properties": {
            "type": { "const": "relay" },
            "to": { "type": "integer", "minimum": 0 },
            "payload": {}
          },
          "required": ["type", "to", "payload"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "Leave",
          "properties": {
            "type": { "const": "leave" }
          },
          "required": ["type"],
          "additionalProperties": false
        }
      ]
    },
    {
      "title": "ServerMessage",
      "oneOf": [
        {
          "type": "object",
          "title": "Welcome",
          "properties": {
            "type": { "const": "welcome" },
            "peer": { "type": "integer", "minimum": 0 },
            "peers": {
              "type": "array",
              "items": { "type": "integer", "minimum": 0 }
            }
          },
          "required": ["type", "peer", "peers"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "PeerJoined",
          "properties": {
            "type": { "const": "peer-joined" },
            "peer": { "type": "integer", "minimum": 0 }
          },
          "required": ["type", "peer"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "PeerLeft",
          "properties": {
            "type": { "const": "peer-left" },
            "peer": { "type": "integer", "minimum": 0 }
          },
          "required": ["type", "peer"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "ForwardedOffer",
          "properties": {
            "type": { "const": "offer" },
            "from": { "type": "integer", "minimum": 0 },
            "sdp": { "type": "string" }
          },
          "required": ["type", "from", "sdp"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "ForwardedAnswer",
          "properties": {
            "type": { "const": "answer" },
            "from": { "type": "integer", "minimum": 0 },
            "sdp": { "type": "string" }
          },
          "required": ["type", "from", "sdp"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "ForwardedIce",
          "properties": {
            "type": { "const": "ice" },
            "from": { "type": "integer", "minimum": 0 },
            "candidate": { "type": "string" }
          },
          "required": ["type", "from", "candidate"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "ForwardedRelay",
          "properties": {
            "type": { "const": "relay" },
            "from": { "type": "integer", "minimum": 0 },
            "payload": {}
          },
          "required": ["type", "from", "payload"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "title": "Error",
          "properties": {
            "type": { "const": "error" },
            "code": { "type": "string" },
            "message": { "type": "string" }
          },
          "required": ["type", "code", "message"],
          "additionalProperties": false
        }
      ]
    }
  ]
}

distributed.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/distributed.json",
  "title": "Distributed (CRDT) Message",
  "description": "CRDT anti-entropy IPC message (protocol.md § Distributed). The CrdtSync message rides the same lazily-ipc transport as Snapshot/Delta as a third IpcMessage variant: the externally-tagged envelope {\"CrdtSync\": {...}}. Supporting distributed/cell-model types are kept under $defs.",
  "type": "object",
  "required": ["CrdtSync"],
  "additionalProperties": false,
  "properties": {
    "CrdtSync": {
      "type": "object",
      "description": "Anti-entropy sync frame: the sender advertises its per-peer stamp frontier and ships an op batch.",
      "required": ["ops"],
      "additionalProperties": false,
      "properties": {
        "frontier": {
          "type": "array",
          "items": { "$ref": "#/$defs/StampFrontierEntry" },
          "description": "Per-peer highest observed stamp. Retained in full (not permission-filtered) so the receiver can compute a sound causal-stability watermark. Optional under #lzspecfrontiersuppress: omit (or send []) when unchanged since the last accepted frame; the receiver reuses its last-merged frontier."
        },
        "ops": {
          "type": "array",
          "items": { "$ref": "#/$defs/CrdtOp" },
          "description": "Op batch this frame ships; permission-filtered by omission before serialization."
        }
      }
    }
  },
  "$defs": {
    "NodeId": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeId" },
    "PeerId": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/PeerId" },
    "NodeKey": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/NodeKey" },
    "WireStamp": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/WireStamp" },
    "IpcValue": { "$ref": "https://lazily.dev/schemas/defs.json#/$defs/IpcValue" },
    "OpKind": {
      "type": "string",
      "enum": ["read", "write", "trigger_effect"],
      "description": "Permission-gated operation kind. The three kinds are gated independently."
    },
    "RemoteOp": {
      "type": "object",
      "description": "Gated, serializable unit a peer requests.",
      "properties": {
        "kind": { "$ref": "#/$defs/OpKind" },
        "node": { "$ref": "#/$defs/NodeId" }
      },
      "required": ["kind", "node"],
      "additionalProperties": false
    },
    "MergeMechanism": {
      "type": "string",
      "enum": ["crdt", "lww", "ot", "lease", "custom"],
      "description": "Convergence mechanism for a multi-write cell. `crdt` is the first normative mechanism (converges without coordination); the rest are reserved extension points (see cell-model.md). Every mechanism MUST be deterministic. An implementation MUST reject an unimplemented mechanism explicitly rather than aliasing it to `crdt`."
    },
    "CellKind": {
      "description": "Static classification of a cell by concurrent-writer count. Single-writer cells take no merge; multi-write cells carry a pluggable merge mechanism. Multi-write is NOT a hardcoded `crdt` kind — see cell-model.md.",
      "oneOf": [
        {
          "type": "object",
          "properties": { "kind": { "const": "single_writer" } },
          "required": ["kind"],
          "additionalProperties": false
        },
        {
          "type": "object",
          "properties": {
            "kind": { "const": "multi_write" },
            "merge": { "$ref": "#/$defs/MergeMechanism" }
          },
          "required": ["kind", "merge"],
          "additionalProperties": false
        }
      ]
    },
    "CellRegisterType": {
      "type": "string",
      "enum": ["lww", "mv", "pn-counter"],
      "description": "CRDT register type for a `merge: crdt` cell (the value shape within the CRDT mechanism); distinct from MergeMechanism."
    },
    "StampFrontierEntry": {
      "type": "array",
      "description": "A (peer, WireStamp) tuple in the per-peer stamp frontier.",
      "prefixItems": [
        { "$ref": "#/$defs/PeerId" },
        { "$ref": "#/$defs/WireStamp" }
      ],
      "minItems": 2,
      "maxItems": 2,
      "items": false
    },
    "CrdtOp": {
      "type": "object",
      "description": "One CRDT cell op on the wire (state-based / CvRDT): the converged register/sequence/text state for `node`, tagged with the WireStamp that produced it and an optional wire-stable NodeKey. State-based, idempotent — safe to resend.",
      "required": ["node", "key", "stamp", "state"],
      "additionalProperties": false,
      "properties": {
        "node": { "$ref": "#/$defs/NodeId", "description": "Volatile target id; pair with `key` for stable addressing." },
        "key": {
          "description": "Wire-stable keyed address, or null when unset. Mirrors the lazily-rs derived struct: `key` is always present (null when unset), unlike NodeSnapshot/NodeAdd which omit it.",
          "oneOf": [
            { "type": "null" },
            { "$ref": "#/$defs/NodeKey" }
          ]
        },
        "stamp": { "$ref": "#/$defs/WireStamp" },
        "state": { "$ref": "#/$defs/IpcValue", "description": "The converged CRDT state to merge." }
      }
    }
  }
}

receipts.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/receipts.json",
  "title": "Causal Receipts",
  "description": "Generic receipt/outcome projection for causally-linked commands or effect requests. This is not a transport ACK plane; observed/accepted are non-terminal, applied/rejected are terminal.",
  "type": "object",
  "required": ["CausalReceipts"],
  "additionalProperties": false,
  "properties": {
    "CausalReceipts": {
      "type": "object",
      "required": ["receipts"],
      "additionalProperties": false,
      "properties": {
        "receipts": {
          "type": "array",
          "items": { "$ref": "#/$defs/CausalReceipt" }
        }
      }
    }
  },
  "$defs": {
    "ReceiptOutcome": {
      "type": "string",
      "enum": ["observed", "accepted", "applied", "rejected"],
      "description": "Outcome vocabulary. observed/accepted are non-terminal; applied/rejected are terminal."
    },
    "OptionalString": {
      "oneOf": [
        { "type": "null" },
        { "type": "string" }
      ]
    },
    "CausalReceipt": {
      "type": "object",
      "required": [
        "receipt_id",
        "causation_id",
        "observer",
        "generation",
        "outcome",
        "reason",
        "payload_hash"
      ],
      "additionalProperties": false,
      "properties": {
        "receipt_id": {
          "type": "string",
          "minLength": 1,
          "description": "Idempotency key for this receipt event."
        },
        "causation_id": {
          "type": "string",
          "minLength": 1,
          "description": "Stable id of the command, event, or effect request this receipt observes."
        },
        "observer": {
          "type": "string",
          "minLength": 1,
          "description": "Peer, process, or subsystem that produced the receipt."
        },
        "generation": {
          "type": "integer",
          "minimum": 0,
          "description": "Producer/editor generation. Consumers discard receipts outside the current generation for the causation id."
        },
        "outcome": { "$ref": "#/$defs/ReceiptOutcome" },
        "reason": { "$ref": "#/$defs/OptionalString" },
        "payload_hash": { "$ref": "#/$defs/OptionalString" }
      }
    }
  }
}

message-passing.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/message-passing.json",
  "title": "Command / RPC Message Plane",
  "description": "Evented command message plane (command-plane-v1). CommandSubmit/CommandCancel/CommandEvents/CommandProjection are an additive sibling family to Snapshot/Delta/CrdtSync. RPC is a facade over this plane; transport ACKs and non-terminal events are NOT authority. Terminal command outcomes fold through CausalReceipt (receipts.json).",
  "oneOf": [
    { "$ref": "#/$defs/CommandSubmitFrame" },
    { "$ref": "#/$defs/CommandCancelFrame" },
    { "$ref": "#/$defs/CommandEventsFrame" },
    { "$ref": "#/$defs/CommandProjectionFrame" }
  ],
  "$defs": {
    "CommandId": {
      "type": "string",
      "minLength": 1,
      "description": "Stable, replay-safe id for a command. Dedupe and reconnect projection key off this id."
    },
    "OptionalString": {
      "oneOf": [{ "type": "null" }, { "type": "string" }]
    },
    "DedupePolicy": {
      "type": "string",
      "enum": ["none", "same_idempotency_key", "same_command_id"],
      "description": "How the admitter collapses concurrent/duplicate submits."
    },
    "CommandPolicy": {
      "type": "object",
      "required": ["dedupe", "supersede", "cancel_on_preempt"],
      "additionalProperties": false,
      "properties": {
        "dedupe": { "$ref": "#/$defs/DedupePolicy" },
        "supersede": {
          "type": "boolean",
          "description": "If true, a newer submit with the same idempotency key supersedes an older non-terminal command."
        },
        "cancel_on_preempt": {
          "type": "boolean",
          "description": "If true, the admitter may cancel a still-non-terminal command when it is preempted."
        }
      }
    },
    "CommandSubmit": {
      "type": "object",
      "required": [
        "command_id",
        "causation_id",
        "source",
        "target",
        "namespace",
        "name",
        "authority_generation",
        "idempotency_key",
        "deadline_ms",
        "policy",
        "payload_type",
        "payload_hash",
        "payload",
        "required_features"
      ],
      "additionalProperties": false,
      "properties": {
        "command_id": { "$ref": "#/$defs/CommandId" },
        "causation_id": {
          "type": "string",
          "minLength": 1,
          "description": "Causal parent id (a command id or event id). Self-caused submits set this equal to command_id."
        },
        "source": {
          "type": "string",
          "minLength": 1,
          "description": "Identity of the submitter (e.g. 'vscode-plugin', 'jetbrains-plugin')."
        },
        "target": {
          "type": "string",
          "minLength": 1,
          "description": "Identity of the intended handler (e.g. 'project-controller')."
        },
        "namespace": {
          "type": "string",
          "minLength": 1,
          "description": "Domain namespace that owns the payload schema (e.g. 'agent-doc'). Lazily owns the envelope, not the namespace."
        },
        "name": {
          "type": "string",
          "minLength": 1,
          "description": "Command name within the namespace (e.g. 'editor_route')."
        },
        "authority_generation": {
          "type": "integer",
          "minimum": 0,
          "description": "Authority/controller generation. Receipts and events outside this generation are stale and never update the projection."
        },
        "idempotency_key": {
          "type": "string",
          "minLength": 1,
          "description": "Dedupe/supersede key (e.g. 'project-root:doc:run')."
        },
        "deadline_ms": {
          "type": "integer",
          "minimum": 0,
          "description": "Deadline in milliseconds. 0 means no deadline."
        },
        "policy": { "$ref": "#/$defs/CommandPolicy" },
        "payload_type": {
          "type": "string",
          "minLength": 1,
          "description": "Fully-qualified domain payload type (e.g. 'agent-doc.editor_route.v1')."
        },
        "payload_hash": {
          "type": "string",
          "minLength": 1,
          "description": "Content hash of the payload body (e.g. 'sha256:...')."
        },
        "payload": {
          "$ref": "https://lazily.dev/schemas/defs.json#/$defs/IpcValue",
          "description": "Domain payload as inline bytes or shared-memory blob reference. Lazily does not interpret the body."
        },
        "required_features": {
          "type": "array",
          "items": { "type": "string" },
          "description": "Features the target must advertise or the submit fails closed."
        }
      }
    },
    "CommandCancel": {
      "type": "object",
      "required": [
        "command_id",
        "causation_id",
        "source",
        "authority_generation",
        "reason"
      ],
      "additionalProperties": false,
      "properties": {
        "command_id": { "$ref": "#/$defs/CommandId" },
        "causation_id": {
          "type": "string",
          "minLength": 1,
          "description": "Id of the cancel request itself (for its own receipt/replay)."
        },
        "source": { "type": "string", "minLength": 1 },
        "authority_generation": {
          "type": "integer",
          "minimum": 0,
          "description": "Generation the cancel targets. A stale-generation cancel is ignored."
        },
        "reason": { "$ref": "#/$defs/OptionalString" }
      }
    },
    "CommandEventKind": {
      "type": "string",
      "enum": [
        "observed",
        "accepted",
        "started",
        "progress",
        "cancelled",
        "superseded",
        "timed_out"
      ],
      "description": "Progress/detail kinds. These are UX/diagnostics only and are NEVER terminal proof; terminal proof folds through CausalReceipt. cancelled/superseded/timed_out are surfaced here for UX but their terminal authority is a matching rejected receipt."
    },
    "CommandEvent": {
      "type": "object",
      "required": ["event_id", "command_id", "kind", "generation", "detail"],
      "additionalProperties": false,
      "properties": {
        "event_id": {
          "type": "string",
          "minLength": 1,
          "description": "Idempotency key for this event. Duplicate event_ids are no-ops."
        },
        "command_id": { "$ref": "#/$defs/CommandId" },
        "kind": { "$ref": "#/$defs/CommandEventKind" },
        "generation": {
          "type": "integer",
          "minimum": 0,
          "description": "Authority generation. Events outside the current generation for the command are ignored."
        },
        "detail": {
          "$ref": "#/$defs/OptionalString",
          "description": "Optional human/diagnostics detail (queue position, retry advice, copied CLI output). Not proof of effect."
        }
      }
    },
    "CommandEventsFrame": {
      "type": "object",
      "required": ["CommandEvents"],
      "additionalProperties": false,
      "properties": {
        "CommandEvents": {
          "type": "object",
          "required": ["events"],
          "additionalProperties": false,
          "properties": {
            "events": {
              "type": "array",
              "items": { "$ref": "#/$defs/CommandEvent" }
            }
          }
        }
      }
    },
    "CommandStatus": {
      "type": "string",
      "enum": [
        "submitted",
        "accepted",
        "running",
        "applied",
        "rejected",
        "cancelled",
        "superseded",
        "timed_out"
      ],
      "description": "Folded projection status. submitted/accepted/running are non-terminal; applied/rejected/cancelled/superseded/timed_out are terminal and backed by a terminal CausalReceipt."
    },
    "CommandProjectionEntry": {
      "type": "object",
      "required": [
        "command_id",
        "status",
        "terminal",
        "generation",
        "reason",
        "terminal_receipt_id",
        "last_event_id"
      ],
      "additionalProperties": false,
      "properties": {
        "command_id": { "$ref": "#/$defs/CommandId" },
        "status": { "$ref": "#/$defs/CommandStatus" },
        "terminal": {
          "type": "boolean",
          "description": "True iff a terminal CausalReceipt has folded into this command. accepted/queued/admission never sets terminal."
        },
        "generation": {
          "type": "integer",
          "minimum": 0,
          "description": "Current authority generation for the command."
        },
        "reason": {
          "$ref": "#/$defs/OptionalString",
          "description": "Terminal reason (rejection cause, cancel reason, timeout); null while non-terminal or applied without reason."
        },
        "terminal_receipt_id": {
          "$ref": "#/$defs/OptionalString",
          "description": "Receipt id that made the command terminal, or null when non-terminal."
        },
        "last_event_id": {
          "$ref": "#/$defs/OptionalString",
          "description": "Last folded event id for incremental resync, or null when none."
        }
      }
    },
    "CommandProjectionFrame": {
      "type": "object",
      "required": ["CommandProjection"],
      "additionalProperties": false,
      "properties": {
        "CommandProjection": {
          "type": "object",
          "required": ["generation", "commands"],
          "additionalProperties": false,
          "properties": {
            "generation": {
              "type": "integer",
              "minimum": 0,
              "description": "Authority generation this projection image was taken at."
            },
            "commands": {
              "type": "array",
              "items": { "$ref": "#/$defs/CommandProjectionEntry" }
            }
          }
        }
      }
    },
    "CommandSubmitFrame": {
      "type": "object",
      "required": ["CommandSubmit"],
      "additionalProperties": false,
      "properties": { "CommandSubmit": { "$ref": "#/$defs/CommandSubmit" } }
    },
    "CommandCancelFrame": {
      "type": "object",
      "required": ["CommandCancel"],
      "additionalProperties": false,
      "properties": { "CommandCancel": { "$ref": "#/$defs/CommandCancel" } }
    }
  }
}

statechart.json

This is a compute schema, not a wire message. It normatively defines the declarative Harel/SCXML chart form used by conformance fixtures and cross-language chart definitions. A chart is never serialized as a distinct wire kind; only its converged active configuration crosses IPC/FFI as an ordinary cell Payload. See State Charts.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/statechart.json",
  "title": "StateChart",
  "description": "Declarative form of a lazily state chart (Harel/SCXML subset). This is COMPUTE, not a wire message: a chart is never serialized over IPC/FFI as a distinct type — only its converged active configuration crosses the wire as an ordinary cell Payload. This schema fixes the cross-language declarative form used by conformance fixtures and chart definitions.",
  "type": "object",
  "required": ["initial", "states"],
  "additionalProperties": false,
  "properties": {
    "initial": {
      "description": "Default entry state of the root region. MUST resolve to a leaf by descending compound `initial`s (or, for a parallel root, to one leaf per region).",
      "type": "string"
    },
    "context": {
      "description": "Optional extended-state schema: host-resolved caller state over which guard expressions evaluate. Never serialized as part of the active configuration.",
      "type": "object"
    },
    "states": {
      "type": "object",
      "minProperties": 1,
      "additionalProperties": {"$ref": "#/$defs/State"}
    }
  },
  "$defs": {
    "State": {
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "parent": {
          "description": "Parent state id. Exactly one state (the root) has no parent.",
          "type": "string"
        },
        "kind": {
        "description": "Authoritative structural kind. If present, it MUST agree with the state's structural fields and child relation; contradictory declarations are malformed. If omitted, it is inferred: `history` when `history` is set; `parallel` when `parallel` is true; `compound` when the state has children; otherwise `atomic`. `final` cannot be inferred.",
          "enum": ["atomic", "compound", "parallel", "history", "final"]
        },
        "parallel": {
          "description": "If true, this state is an AND-state: its children are concurrent regions, all of which are active whenever this state is active. Mutually exclusive with `initial`.",
          "type": "boolean"
        },
        "initial": {
          "description": "Default child entered when this compound state is entered. Required for compound states; forbidden for parallel states.",
          "type": "string"
        },
        "history": {
          "description": "Marks this state as a history pseudo-state. `shallow` records/restores the direct children of the parent region; `deep` records/restores the full active leaf configuration. Resolve to `default` on first entry (no recorded history).",
          "enum": ["shallow", "deep"]
        },
        "default": {
          "description": "Default target for a history state when no configuration has been recorded for its region yet.",
          "type": "string"
        },
        "on": {
          "description": "Event → transition table. An event maps to a target id or a transition object. Transitions on compound/parallel states are reachable from any active descendant (event bubbling).",
          "type": "object",
          "additionalProperties": {"$ref": "#/$defs/Transition"}
        },
        "entry": {
          "description": "Ordered actions fired when this state is entered, after its ancestors' entry actions.",
          "type": "array",
          "items": {"$ref": "#/$defs/Action"}
        },
        "exit": {
          "description": "Ordered actions fired when this state is exited, before its ancestors' exit actions.",
          "type": "array",
          "items": {"$ref": "#/$defs/Action"}
        },
        "run": {
          "description": "Ongoing (do) actions started on entry and cancelled on exit. Host-managed; not part of conformance replay.",
          "type": "array",
          "items": {"$ref": "#/$defs/Action"}
        }
      },
      "allOf": [
        {
          "if": {"required": ["parallel"], "properties": {"parallel": {"const": true}}},
          "then": {"not": {"required": ["initial"]}}
        },
        {
          "if": {"required": ["history"]},
          "then": {"not": {"anyOf": [{"required": ["initial"]}, {"required": ["parallel"]}]}}
        }
      ]
    },
    "Transition": {
      "description": "A transition. A bare string is shorthand for `{\"target\": <string>}`.",
      "oneOf": [
        {"type": "string"},
        {
          "type": "object",
          "required": ["target"],
          "additionalProperties": false,
          "properties": {
            "target": {
              "description": "Target state id. MAY be compound or parallel; entry descends via `initial` (compound) or enters every region (parallel). MAY target a history state to resume its parent region.",
              "type": "string"
            },
            "guard": {"$ref": "#/$defs/Guard"},
            "action": {
              "description": "Ordered actions fired after the exit set and before the enter set.",
              "type": "array",
              "items": {"$ref": "#/$defs/Action"}
            },
            "internal": {
              "description": "If true, an internal transition: the source state is not exited/re-entered even when the target is the source or a descendant. Defaults to false (external).",
              "type": "boolean"
            }
          }
        }
      ]
    },
    "Guard": {
      "description": "A guard predicate. A bare string is a named guard resolved by the caller's guard resolver (fail-closed if absent). An object is an extended-state expression the host evaluates against `context`.",
      "oneOf": [
        {"type": "string"},
        {
          "type": "object",
          "required": ["expr"],
          "additionalProperties": false,
          "properties": {
            "expr": {"type": "string"}
          }
        }
      ]
    },
    "Action": {
      "description": "An action. A bare string is a named action resolved by the caller's action handler. An object carries an action name plus an opaque payload.",
      "oneOf": [
        {"type": "string"},
        {
          "type": "object",
          "required": ["name"],
          "additionalProperties": false,
          "properties": {
            "name": {"type": "string"},
            "payload": {}
          }
        }
      ]
    }
  }
}

stdlib-fixture.schema.json

This compute schema validates the portable standard-library fixture corpus. It is not a wire message. See Portable Standard Library.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lazily.dev/schemas/stdlib-fixture.schema.json",
  "title": "Portable Lazily stdlib conformance fixture",
  "type": "object",
  "required": [
    "fixture_version",
    "feature",
    "scenario_floor",
    "assertion_floor",
    "mutation_floor",
    "scenarios",
    "mutations"
  ],
  "properties": {
    "$schema": {
      "type": "string",
      "const": "https://lazily.dev/schemas/stdlib-fixture.schema.json"
    },
    "fixture_version": {
      "const": 1
    },
    "feature": {
      "enum": [
        "stdlib_timer_v1",
        "stdlib_timeout_v1",
        "stdlib_revision_barrier_v1"
      ]
    },
    "scenario_floor": {
      "type": "integer",
      "minimum": 1
    },
    "assertion_floor": {
      "type": "integer",
      "minimum": 1
    },
    "mutation_floor": {
      "type": "integer",
      "minimum": 1
    },
    "scenarios": {
      "type": "array",
      "minItems": 1
    },
    "mutations": {
      "type": "array",
      "minItems": 1,
      "items": {
        "$ref": "#/$defs/mutation"
      }
    }
  },
  "allOf": [
    {
      "if": {
        "properties": {
          "feature": {
            "const": "stdlib_timer_v1"
          }
        }
      },
      "then": {
        "properties": {
          "scenarios": {
            "items": {
              "$ref": "#/$defs/timerScenario"
            }
          }
        }
      }
    },
    {
      "if": {
        "properties": {
          "feature": {
            "const": "stdlib_timeout_v1"
          }
        }
      },
      "then": {
        "properties": {
          "scenarios": {
            "items": {
              "$ref": "#/$defs/timeoutScenario"
            }
          }
        }
      }
    },
    {
      "if": {
        "properties": {
          "feature": {
            "const": "stdlib_revision_barrier_v1"
          }
        }
      },
      "then": {
        "properties": {
          "scenarios": {
            "items": {
              "$ref": "#/$defs/barrierScenario"
            }
          }
        }
      }
    }
  ],
  "additionalProperties": false,
  "$defs": {
    "u64": {
      "type": "integer",
      "minimum": 0,
      "maximum": 18446744073709551615
    },
    "nullableU64": {
      "oneOf": [
        {
          "$ref": "#/$defs/u64"
        },
        {
          "type": "null"
        }
      ]
    },
    "expect": {
      "type": "object",
      "required": [
        "outcome"
      ],
      "properties": {
        "outcome": {
          "enum": [
            "pending",
            "fired",
            "completed",
            "timed_out",
            "cancelled",
            "unavailable",
            "satisfied",
            "disposed"
          ]
        },
        "deadline": {
          "$ref": "#/$defs/nullableU64"
        },
        "fired_at": {
          "$ref": "#/$defs/nullableU64"
        },
        "value": {
          "type": [
            "string",
            "null"
          ]
        },
        "reason": {
          "type": [
            "string",
            "null"
          ]
        },
        "revision": {
          "$ref": "#/$defs/nullableU64"
        },
        "generation": {
          "$ref": "#/$defs/nullableU64"
        },
        "operation_calls": {
          "type": "integer",
          "minimum": 0,
          "maximum": 1
        },
        "cancellation_calls": {
          "type": "integer",
          "minimum": 0,
          "maximum": 1
        }
      },
      "additionalProperties": false
    },
    "mutation": {
      "type": "object",
      "required": [
        "operator",
        "must_fail"
      ],
      "properties": {
        "operator": {
          "type": "string",
          "minLength": 1
        },
        "must_fail": {
          "type": "array",
          "minItems": 1,
          "items": {
            "type": "string",
            "minLength": 1
          }
        }
      },
      "additionalProperties": false
    },
    "timerScenario": {
      "type": "object",
      "required": [
        "id",
        "steps"
      ],
      "properties": {
        "id": {
          "type": "string",
          "minLength": 1
        },
        "steps": {
          "type": "array",
          "minItems": 1,
          "items": {
            "$ref": "#/$defs/timerStep"
          }
        }
      },
      "additionalProperties": false
    },
    "timerStep": {
      "oneOf": [
        {
          "type": "object",
          "required": [
            "op",
            "now",
            "duration",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "start"
            },
            "now": {
              "$ref": "#/$defs/u64"
            },
            "duration": {
              "$ref": "#/$defs/u64"
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        },
        {
          "type": "object",
          "required": [
            "op",
            "now",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "observe"
            },
            "now": {
              "$ref": "#/$defs/u64"
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        }
      ]
    },
    "timeoutScenario": {
      "type": "object",
      "required": [
        "id",
        "steps"
      ],
      "properties": {
        "id": {
          "type": "string",
          "minLength": 1
        },
        "steps": {
          "type": "array",
          "minItems": 1,
          "items": {
            "$ref": "#/$defs/timeoutStep"
          }
        }
      },
      "additionalProperties": false
    },
    "timeoutStep": {
      "oneOf": [
        {
          "type": "object",
          "required": [
            "op",
            "now",
            "duration",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "start"
            },
            "now": {
              "$ref": "#/$defs/u64"
            },
            "duration": {
              "$ref": "#/$defs/u64"
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        },
        {
          "type": "object",
          "required": [
            "op",
            "now",
            "operation",
            "cancellation",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "poll"
            },
            "now": {
              "$ref": "#/$defs/u64"
            },
            "operation": {
              "enum": [
                "pending",
                "completed",
                "unavailable"
              ]
            },
            "value": {
              "type": "string"
            },
            "cancellation": {
              "enum": [
                "pending",
                "cancelled",
                "unavailable"
              ]
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        }
      ]
    },
    "barrierScenario": {
      "type": "object",
      "required": [
        "id",
        "steps"
      ],
      "properties": {
        "id": {
          "type": "string",
          "minLength": 1
        },
        "steps": {
          "type": "array",
          "minItems": 1,
          "items": {
            "$ref": "#/$defs/barrierStep"
          }
        }
      },
      "additionalProperties": false
    },
    "barrierStep": {
      "oneOf": [
        {
          "type": "object",
          "required": [
            "op",
            "revision",
            "required_revision",
            "deadline",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "start"
            },
            "revision": {
              "$ref": "#/$defs/u64"
            },
            "required_revision": {
              "$ref": "#/$defs/u64"
            },
            "deadline": {
              "$ref": "#/$defs/nullableU64"
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        },
        {
          "type": "object",
          "required": [
            "op",
            "now",
            "predicate",
            "cancellation",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "observe"
            },
            "now": {
              "$ref": "#/$defs/u64"
            },
            "predicate": {
              "type": "boolean"
            },
            "cancellation": {
              "enum": [
                "pending",
                "cancelled",
                "unavailable"
              ]
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        },
        {
          "type": "object",
          "required": [
            "op",
            "now",
            "observed_revision",
            "predicate",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "register_recheck"
            },
            "now": {
              "$ref": "#/$defs/u64"
            },
            "observed_revision": {
              "$ref": "#/$defs/u64"
            },
            "predicate": {
              "type": "boolean"
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        },
        {
          "type": "object",
          "required": [
            "op",
            "revision",
            "predicate",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "advance"
            },
            "revision": {
              "$ref": "#/$defs/u64"
            },
            "predicate": {
              "type": "boolean"
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        },
        {
          "type": "object",
          "required": [
            "op",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "dispose"
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        },
        {
          "type": "object",
          "required": [
            "op",
            "key",
            "expect"
          ],
          "properties": {
            "op": {
              "const": "receipt"
            },
            "key": {
              "type": "string",
              "minLength": 1
            },
            "expect": {
              "$ref": "#/$defs/expect"
            }
          },
          "additionalProperties": false
        }
      ]
    }
  }
}

Lean Formal Model

formal/lean is a small Lean 4 Lake package for the IPC Snapshot/Delta state machine. It is a spec companion, not an implementation replacement.

The model proves the invariants that are easiest to blur across language bindings:

  • delta epochs are strictly sequential (epoch = base_epoch + 1);
  • gap, reorder, and restart cases fail closed to snapshot resync;
  • equal set writes are silent;
  • equal memo recomputes suppress slot_value and downstream invalidation;
  • eager Signal changes publish concrete slot_value ops rather than bare invalidate ops for their backing slot;
  • batch flushes carry a coalesced frontier and advance the IPC epoch once;
  • command-plane projection (LazilyFormal.Command): progress events never complete a command, only a terminal receipt does; stale generations are discarded; duplicate submits are idempotent; a cancel cannot override an applied command; conflicting terminal outcomes fail closed; reconnect projection is fold-equivalent; and an RPC call cannot resolve before a terminal receipt. This mirrors the standalone lazily-formal model.

Run it through the local check target:

make check

Keep the Lean package narrow. JSON Schema, Rust implementation behavior, cross-language conformance fixtures, Loom/thread-safe tests, and live transport validation remain separate verification layers.

PRD: Native Distributed Queue Support

Status: Active — TopicCell semantic core shipped; distributed storage remains post-v1 Created: 2026-07-08 Depends on: v1 QueueCell + QueueStorage adapter seam

Problem

Applications using lazily need distributed queue semantics — producer/consumer across process boundaries, work distribution, event delivery — but face a dilemma:

  1. External brokers (Kafka, RabbitMQ, NATS, Redis Streams, SQS) are production-grade but require provisioning and operating a separate service. This is heavy for embedded, edge, small-to-medium-scale, or prototype deployments where the operational overhead of a broker exceeds the application’s complexity budget.

  2. CRDT-replicated queues (via lazily’s existing CrdtPlaneRuntime) converge without coordination but cannot provide the semantics production queues need: destructive pop requires agreement (not merge), FIFO order requires a single sequencer, and exactly-once delivery requires a leader. CRDT is the right tool for collaborative editing (TextCrdt, SeqCrdt) and the wrong tool for queues. See § Background: Why Consensus, Not CRDT.

The embeddability advantage

lazily is embeddable as a library/app. Its advantage: a distributed queue can be built into the application — no external service to provision, no broker to operate, no ops team required — while still providing consensus-based strong consistency. This is the same value proposition as SQLite vs PostgreSQL: trade raw performance and operational features for zero-provisioning embedded simplicity.

Positioning

Use an external broker when one is already available. Established distributed queues have years of production hardening, tooling, client libraries, and operational experience. Until lazily’s native distributed queue reaches feature parity, the external broker is the better choice for production-scale workloads. The v1 QueueStorage adapter seam enables this today — a KafkaStorage or RedisStreamStorage backend plugs into the same reactive shell.

Use lazily’s native distributed queue when embeddability matters. For embedded, edge, small-to-medium scale, zero-provisioning, or prototype deployments, a library-embedded distributed queue eliminates the operational cost of provisioning and managing a broker. When the application is already shipping lazily, the native queue adds zero new dependencies.

The native distributed queue is not a Kafka replacement. It is the right choice when provisioning a broker is more expensive than the problem it solves.

Target Use Cases

Use caseWhy native (not external broker)
Embedded / edge deploymentNo ops team, no broker to provision
Small-to-medium scale work distributionBroker overhead exceeds app complexity
Prototyping / developmentZero-provisioning; promote to broker later via adapter
Single-binary distributed appsShip one artifact, not app + broker
Applications already using lazilyNo new dependency; reuses reactive substrate
Partition-tolerant control planesIn-process queue with Raft-level consistency

Goals

  • Consensus-based distributed queue (Raft replicated log), not CRDT.
  • Zero external service dependencies — the queue is embedded in the application process; peers are other application instances.
  • Builds on v1 — the QueueStorage adapter seam is the integration point. The native distributed queue is a new backend (RaftQueueStorage), not a new primitive.
  • Path to parity — phased delivery that converges toward feature-comparability with established distributed queues for the embeddable niche.
  • Cross-language — consistent semantics across every lazily binding via the shared conformance fixtures, same as every other lazily primitive.

Non-Goals (v1)

  • Not replacing external brokers for production-scale workloads. The adapter seam (KafkaStorage, RedisStreamStorage, SqsStorage) is the recommended path when a broker is available.
  • Not CRDT-based. CRDT is the wrong algebra for destructive pop.
  • Not implementing Raft from scratch. Use an existing, proven Raft library per binding (or a Rust core + FFI for other languages).
  • Not partitioning / consumer groups / exactly-once transactions in Phase 1. These are Phase 4 parity features.

Background: Why Consensus, Not CRDT

A queue’s defining operation is destructive pop — “I claim this element exclusively; no one else may have it.” This is an agreement problem, not a merge problem.

Property neededCRDT provides?Consensus provides?
Concurrent writes all survive mergeN/A (serialized by leader)
Exclusive destructive pop (exactly-once)❌ (at-least-once)
FIFO order matching push order❌ (fractional-index, may reorder)✅ (log index = total order)
Immediate capacity rejection (no overcommit)❌ (convergent is_full lags)✅ (leader rejects)
Single delivery ID for ack/dedup✅ (leader assigns)
No tombstone growth under high churn❌ (head-pointer optimization needed)✅ (log compaction)

CRDT is excellent when all operations are commutative and non-destructive (collaborative text editing, counters, sets, shared maps). Queues are none of these. This is why every production distributed queue (Kafka, RabbitMQ, NATS, Redis Streams, SQS) uses consensus, not CRDT — and why lazily uses CRDT for TextCrdt/SeqCrdt but uses consensus for the native distributed queue.

For the full analysis (CRDT tombstone growth under load, head-pointer vs tombstone GC, resurrection safety), see the design discussion captured in the distributed queue pressure-test (session document).

Architecture

Core insight: the Raft log IS the queue

A replicated log (via Raft) provides everything a distributed queue needs:

  • Total order: log entries are committed with monotonically-increasing indices → FIFO is free.
  • Leader-based writes: all pushes go through the leader → no concurrent-write conflicts, no fractional-index reordering.
  • Durability: committed log entries survive leader failure → queue state is durable.
  • GC: log compaction (snapshot + truncate below the lowest consumer cursor) reclaims consumed entries → bounded memory.

The queue is a cursor over the replicated log:

  ┌──────────────────────────────────────────────────────┐
  │               Replicated Log (Raft)                   │
  │                                                       │
  │  [Push A] [Push B] [Push C] [Push D] [Push E] ...    │
  │     1        2        3        4        5            │
  │                                                       │
  │  committed ──────────────────────────── ▲ ── tail    │
  │                                  consumer cursor      │
  └──────────────────────────────────────────────────────┘
                     │
                     ▼
  Queue contents = entries (consumer_cursor, tail]
  GC: entries ≤ consumer_cursor are compactable
  • Push = append Entry { value } to the Raft log (leader serializes).
  • Pop = read entry at consumer_cursor + 1, advance cursor (replicated via Raft state machine).
  • GC = log compaction below the lowest cursor (or a TTL/expiry floor).

This is architecturally closer to Kafka (log + consumer offset) than to RabbitMQ (AMQP delivery model). The log is append-only; destructive semantics live in the cursor, not in tombstones.

RaftQueueStorage as a QueueStorage backend

The native distributed queue is not a new primitive. It is a QueueStorage backend — the same adapter seam that v1 defines for VecDequeStorage, KafkaStorage, etc.

  ┌─────────────────────────────────────────────────┐
  │            QueueCell reactive shell              │
  │  (head/tail/closed version cells, invalidation)  │
  └───────────────────┬─────────────────────────────┘
                      │ QueueStorage trait
                      │
   ┌──────────────────┼──────────────────┐
   │                  │                  │
   ▼                  ▼                  ▼
 VecDeque        RaftQueue          KafkaStorage
 (local,         (embedded          (external broker,
  default)       consensus)          via adapter)

This means:

  • The reactive shell (invalidation, backpressure, closure) is shared across all backends.
  • The consensus logic lives entirely in RaftQueueStorage.
  • Users switch between local / native-distributed / external-broker by swapping the storage backend, with no change to the reactive API.

Transport

RaftQueueStorage reuses lazily’s existing transport infrastructure:

  • IPC (in-process, cross-thread) — same machine.
  • WebSocket — cross-machine, cloud/edge.
  • WebRTC — peer-to-peer, NAT traversal.

No new transport layer. The Raft RPCs (RequestVote, AppendEntries, etc.) ride the same DataChannel abstraction that the CRDT plane and command plane already use.

Consensus implementation

Use an existing, proven Raft library — do not implement Raft from scratch.

BindingCandidate
lazily-rsopenraft or raft-rs
lazily-pyFFI to Rust RaftQueueStorage core
lazily-zigFFI to Rust core, or native Zig Raft if available
lazily-jsWASM-compiled Rust core, or WebSocket client to a remote leader
lazily-goNative Go Raft (etcd/raft) — idiomatic for Go
lazily-ktFFI to Rust core, or native JVM Raft (copy-cat)

The Rust core is the reference implementation; other bindings either FFI to it or use a native Raft library in their language. The QueueStorage trait guarantees semantic parity regardless of the underlying Raft implementation.

Relationship to v1

The v1 deliverables are the foundation for native distributed queue support:

v1 deliverableRole in the distributed queue
QueueCell reactive shellThe API surface — unchanged whether storage is local or distributed
QueueStorage adapter traitThe integration point — RaftQueueStorage is a new backend
VecDequeStorage defaultThe local reference; RaftQueueStorage must match its observable FIFO contract
FIFO-order spec clauseThe cross-backend invariant — consensus-backed or broker-backed, FIFO must hold
Closure observable contractShared across backends — close semantics are shell-level

No v1 spec/formal work is blocked by the distributed queue PRD. The adapter seam carries the distributed story: v1 ships the seam; this PRD ships the consensus backend.

Phased Delivery

Phase 0 — v1 foundation (current scope)

  • Local QueueCell (SPSC primitive + MPSC usage rule).
  • QueueStorage adapter trait + VecDequeStorage default.
  • TopicCell local semantic contract + conformance + Lean reference.
  • WorkQueueCell portable local-authority lifecycle + conformance + Lean safety reference; distributed/HA claim serialization remains a Phase 2 integration.
  • Reactive shell: closure, bounded/backpressure, ordering contract.

Deliverable: a local queue primitive with a pluggable backend seam, ready for distributed backends.

Phase 1 — Consensus core

  • RaftQueueStorage: Raft replicated log + consumer cursor.
  • Single-partition, single-consumer distributed queue.
  • Log compaction (GC below consumer cursor).
  • Transport over existing DataChannel (IPC / WebSocket / WebRTC).
  • Conformance fixtures for distributed FIFO, durability under leader failover, GC safety.

Deliverable: an embeddable distributed queue with strong consistency, no external dependencies. This is the milestone that delivers the PRD’s core value proposition.

Phase 2 — WorkQueueCell (exactly-once handoff)

  • Reuse the shipped local push / claim / ack / nack / reap_expired lifecycle and cross-language fixtures unchanged.
  • Leader-based exclusive handoff over the Raft log.
  • Pop = advance cursor with ack; unacked entries are redelivered.
  • Pending entries list (consumer failure recovery).
  • Dead-letter queue (poison-message handling).
  • Receipt integration for at-most-once effect authority.

Deliverable: distributed exactly-once assignment authority for the shipped competing-consumer shell — the semantic that CRDT cannot provide and the reason production queues use consensus.

Phase 3 — TopicCell (multi-cursor broadcast)

  • Each subscriber maintains its own cursor over the Raft log.
  • Cursor persistence (survives subscriber restart).
  • Log GC bounded by the slowest subscriber’s cursor.
  • Fan-out semantics (one push → all subscribers receive).

Deliverable: pub/sub broadcast — the event-delivery use case.

Status: the storage-independent semantic contract, replay fixtures, and universal Lean proofs shipped in v0.31.0. Wiring those cursors to a Raft-backed durable log remains part of the post-v1 distributed-storage implementation.

Phase 4 — Parity features

FeatureParity targetSource
PartitioningKey-based routing across multiple Raft groupsKafka partitions
Consumer groupsShared cursor among group members; rebalance on join/leaveKafka consumer groups
Exactly-once deliveryTransactional consumer (consume + ack + commit in one Raft round)Kafka transactions
PersistenceDurable log (fsync on commit); WAL recoveryKafka / RabbitMQ durability
Visibility timeout / leaseConsumer lease with TTL; redelivery on expirySQS
PriorityPriority-weighted cursor advancementRabbitMQ priority queues
MonitoringDepth / lag / throughput / consumer-position metricsKafka / RabbitMQ dashboards
Flow controlBounded queue with quota; push rejection with backpressure signalKafka quota / RabbitMQ prefetch

Deliverable: feature-comparability with established distributed queues for the embeddable niche.

Parity Boundary

lazily’s native distributed queue will not match established brokers on:

  • Raw throughput — dedicated brokers (Kafka) are optimized for millions of ops/sec with zero-copy kernel-bypass. lazily targets the embeddable niche, not the hyperscale niche.
  • Operational tooling — Kafka’s ecosystem (Connect, Streams, Schema Registry, KSQL) is decades of engineering. lazily provides the queue, not the platform.
  • Language-specific client libraries — Kafka has clients in 30+ languages. lazily has its own bindings; external-broker integration uses the adapter seam.

The parity target is semantic (FIFO, exactly-once, persistence, ack/nack, dead-letter, consumer groups) and embedding-grade operational (zero-provisioning, in-process, no external services) — not hyperscale performance.

Risks / Open Questions

RiskMitigation
Raft library correctness / maturityUse proven libraries (openraft, etcd/raft); conformance fixtures validate semantics
Performance vs dedicated brokersExplicitly non-goal for hyperscale; target embeddable niche
Multi-language Raft divergenceRust core + FFI for bindings that can’t host a native Raft; conformance fixtures enforce parity
Log compaction correctness under failoverFormal model + conformance fixture for GC safety under leader crash
Partitioning strategy (Phase 4)Defer to Phase 4 design; single-partition is sufficient for embeddable niche in Phase 1
When to recommend native vs external brokerClear decision criteria in docs: scale threshold, existing infrastructure, operational capacity
Raft group membership changesUse the Raft library’s built-in membership change protocol; don’t reinvent
Cross-backend snapshot interopRaftQueueStorage defines its own snapshot format; cross-backend interop requires explicit format agreement (per v1 wire/snapshot clause)

Decision Criteria: Native vs External Broker

CriterionNative (RaftQueueStorage)External (KafkaStorage etc.)
No external service to provision
Zero new dependencies (already using lazily)
Embeddable / single-binary deployment
Edge / resource-constrained environment
Production-scale throughput (>100K ops/sec)
Mature operational tooling / monitoring
Rich client ecosystem (30+ languages)
Existing infrastructure / team expertise
Exactly-once transactions (until Phase 4)

Rule of thumb: if you already have a broker, use it. If provisioning one is more expensive than the problem, use the native queue.

References