lazily Wire Protocol

Language-agnostic protocol reference for the lazily reactive-graph family (lazily-rs, lazily-py, lazily-zig). This document describes the wire format, message schemas, and transport contracts. Language-specific APIs live in each binding’s own documentation.

Message Plane

All channels (FFI, IPC, WebSocket, WebRTC data) carry the same two message kinds:

• Snapshot — full graph image, sent on connect and on resync
• Delta — incremental change set, sent once per outermost batch flush

These are tagged as IpcMessage:

{ "Snapshot": { ... } }
{ "Delta": { ... } }

Wire Types

NodeId

Stable wire identifier for a reactive node (cell or slot). Decoupled from language-internal allocation IDs.

{ "node": 1 }

Wire format: u64 wrapped in a "node" field.

PeerId

Identifies a remote peer.

{ "peer": 42 }

Wire format: u64. JavaScript peers must keep this at or below Number.MAX_SAFE_INTEGER.

OpKind

Access category for a remote operation.

RemoteOp

A single operation a remote peer may request.

{ "kind": "Read", "node": 1 }

IpcPayload

Opaque serialized value bytes. The producing language owns type-aware encoding through type_tag; the channel only moves bytes.

Wire format: array of u8 (JSON array of integers).

NodeState

Serialization state for a node.

{ "Payload": [1, 2, 3, 4] }
"Opaque"
{ "SharedBlob": { "offset": 0, "len": 16, "generation": 1, "epoch": 9, "checksum": 123456789 } }

ShmBlobRef

Descriptor for a payload stored in a shared-memory blob arena.

IpcValue

Value stored inline or by shared-memory blob reference.

{ "Inline": [10, 20, 30] }
{ "SharedBlob": { "offset": 40, "len": 17, "generation": 2, "epoch": 9, "checksum": 987654321 } }

Snapshot Message

Full graph image sent on connect or resync.

Schema

Snapshot {
  epoch: u64,
  nodes: Vec<NodeSnapshot>,
  edges: Vec<EdgeSnapshot>,
  roots: Vec<NodeId>
}

NodeSnapshot

NodeSnapshot {
  node: NodeId,
  type_tag: string,
  state: NodeState
}

EdgeSnapshot

EdgeSnapshot {
  dependent: NodeId,
  dependency: NodeId
}

Example: Minimal snapshot

{
  "Snapshot": {
    "epoch": 1,
    "nodes": [
      {
        "node": 1,
        "type_tag": "i32",
        "state": { "Payload": [1, 2, 3, 4] }
      }
    ],
    "edges": [],
    "roots": [1]
  }
}

Example: Multi-node snapshot with opaque node

{
  "Snapshot": {
    "epoch": 7,
    "nodes": [
      { "node": 1, "type_tag": "i32", "state": { "Payload": [1, 2, 3] } },
      { "node": 2, "type_tag": "f64", "state": { "Payload": [0, 0, 0, 0, 0, 0, 240, 63] } },
      { "node": 3, "type_tag": "opaque-type", "state": "Opaque" }
    ],
    "edges": [
      { "dependent": 2, "dependency": 1 },
      { "dependent": 3, "dependency": 1 }
    ],
    "roots": [1, 2]
  }
}

Example: Snapshot with shared-blob node

{
  "Snapshot": {
    "epoch": 9,
    "nodes": [
      {
        "node": 7,
        "type_tag": "text/plain",
        "state": {
          "SharedBlob": {
            "offset": 0,
            "len": 16,
            "generation": 1,
            "epoch": 9,
            "checksum": 123456789
          }
        }
      }
    ],
    "edges": [],
    "roots": [7]
  }
}

Delta Message

Incremental change set emitted after one outermost batch flush.

Schema

Delta {
  base_epoch: u64,
  epoch: u64,
  ops: Vec<DeltaOp>
}

Sequential deltas satisfy epoch == base_epoch + 1. A receiver detects gaps, reorders, or sender restarts by checking base_epoch == last_epoch.

DeltaOp Variants

Example: Sequential delta with all op variants

{
  "Delta": {
    "base_epoch": 40,
    "epoch": 41,
    "ops": [
      { "CellSet": { "node": 1, "payload": { "Inline": [10] } } },
      { "SlotValue": { "node": 2, "payload": { "Inline": [20] } } },
      { "Invalidate": { "node": 3 } },
      { "NodeAdd": { "node": 4, "type_tag": "u64", "state": { "Payload": [64] } } },
      { "NodeRemove": { "node": 5 } },
      { "EdgeAdd": { "dependent": 2, "dependency": 1 } },
      { "EdgeRemove": { "dependent": 3, "dependency": 1 } }
    ]
  }
}

Example: Non-sequential delta (gap)

{
  "Delta": {
    "base_epoch": 12,
    "epoch": 13,
    "ops": []
  }
}

When the receiver’s last_epoch is 10, this delta has a gap (expected 10→11, got 12→13). The receiver must discard it and request a fresh Snapshot.

Example: Delta with shared-blob payload

{
  "Delta": {
    "base_epoch": 8,
    "epoch": 9,
    "ops": [
      {
        "SlotValue": {
          "node": 7,
          "payload": {
            "SharedBlob": {
              "offset": 40,
              "len": 17,
              "generation": 2,
              "epoch": 9,
              "checksum": 987654321
            }
          }
        }
      }
    ]
  }
}

Epoch Contract

• ipc_epoch is a monotonic u64 that advances once per outermost batch flush.
• Snapshot carries epoch.
• Delta carries { base_epoch, epoch } with epoch == base_epoch + 1.
• On Delta where base_epoch != last_epoch: discard the delta, request a fresh Snapshot, resume from the snapshot’s epoch.

Consistency Invariants

• PartialEq cell guard: equal CellSet produces no wire ops.
• Memo equality suppression: a dirty memo slot that recomputes to an equal value emits no SlotValue or downstream Invalidate.
• Coalesced frontier: a dependent reached through many changed cells in one batch appears at most once per delta.
• Eager Signal nodes always carry a value: an eager Signal (see below) is recomputed during the invalidation flush, so when it changes it appears in the delta as a concrete SlotValue (never a bare Invalidate). A purely lazy slot that was not read before the flush may instead appear as Invalidate with no value. Both are valid wire states for the same SlotValue/Invalidate op set; the distinction is computation timing, not message format.

Eager Signal Nodes

A Signal is the eager derived value in the Slot -> Cell -> Signal family: it recomputes the instant a dependency invalidates rather than on next read. It is not a new wire type. A Signal is composed from a memoized backing slot plus a local puller effect, and only the backing slot is graph state, so on the wire a Signal node is an ordinary slot node:

• Snapshot: the backing slot appears as a NodeSnapshot with its materialized value in NodeState (Payload/SharedBlob), like any other readable slot.
• Delta: a value change appears as SlotValue for the backing slot’s NodeId. Because the value is eagerly materialized at flush time it is always concrete; eager nodes do not emit bare Invalidate.
• Memo guard still applies: an eager recompute that yields an equal value (PartialEq) suppresses the SlotValue and any downstream Invalidate, exactly as for ctx.memo slots.
• The puller effect is local: it drives eager recomputation but is not serialized as a node and produces no TriggerEffect op. Eagerness is a producer-side scheduling property; remote peers receive the same permission-filtered Snapshot/Delta state plane regardless of whether a node is lazy or eager.

Peers therefore need no protocol change to consume signals from an eager producer — a Signal is observed as a slot that is reliably present in every delta that changes it.

Permission Boundary

Only nodes on the per-peer allowlist are serialized. Non-allowlisted nodes are omitted entirely (not even as Opaque) so a peer cannot infer their existence. Edges are retained only when both endpoints are readable.

This filter is applied at snapshot/delta construction time, before serialization, on all channels without exception.

Serialization

JSON (default)

serde_json with derived Serialize/Deserialize. All examples above use JSON. This is the cross-language default.

Binary (optional)

postcard compact binary encoding via the ipc-binary feature. Smaller and faster than JSON, but not self-describing — peers must agree on the schema. For same-language or postcard-aware transports only.

Binary frames decode through IpcMessage::decode_binary(bytes) and encode through IpcMessage::encode_binary().

Transport Contracts

FFI (C ABI)

• Opaque channel handle + owned byte buffers
• Functions: channel_new, channel_free, channel_send, channel_recv, ipc_message_validate, ipc_message_kind, ipc_message_clone, bytes_free
• Binary variants: same functions with _binary suffix
• Ownership: caller owns input bytes; Rust owns output buffers until the paired free function is called
• Errors return LazilyFfiStatus enum; panics are caught before the C ABI

IPC (Unix socket / pipe / local TCP)

• Length-prefixed serialized IpcMessage frames
• Shared-memory optional for large IpcValue::SharedBlob payloads
• IpcSink / IpcSource trait interface

WebSocket

• One WebSocket text/binary frame carries one serialized IpcMessage
• Signaling server (#yxjw) relays frames as opaque payload
• Server must not parse CRDT/IPC state

WebRTC Data Channel

• Reliable ordered data channels only (for graph state)
• Length-prefixed framing: 4-byte LE length + payload
• JSON or binary codec negotiated during capability handshake
• On channel failure: re-signaling via SignalingClient, delta resync covers gaps
• Unordered/unreliable channels only for optional lossy telemetry

Capability Negotiation

Each non-local session starts with a handshake:

If peers disagree on protocol major version, codec, or ordering guarantees, they fail closed before applying any Snapshot or Delta.

Cross-Language Family Rules

• Compute closures are language-local. Cross-language sync shares the cell state plane; derived slots converge remotely only when peers use a shared compiled graph or explicit compute descriptors.
• Permission filtering happens before serialization on every channel.
• Channel code must preserve back-pressure and resync behavior.
• All channels carry the same permission-filtered IpcMessage state plane.

Conformance Test Vectors

Canonical JSON fixtures in tests/conformance/ validate wire-format agreement across all language bindings:

Each fixture contains:

{
  "description": "...",
  "protocol_version": 1,
  "kind": "Snapshot" | "Delta",
  "assertions": { ... },
  "wire": { <IpcMessage> }
}

Language bindings should:

1. Parse wire into native types
2. Validate assertions (field values, counts, state kinds)
3. Re-serialize and verify byte-exact match