A replica is a copy of a log you do not own. It holds the same records, ciphertexts, delete markers and, if the owner's peers share them, entry keys. It is read-only: only the owner can add entries.
Opening a replica
A replica starts from three things: the owner's public key, the log ID and
the owner's signed log header. It never needs a private key. A
kolo-log Replica keeps only tree hashes and
records in memory, and its entries in a Storage. NewReplica starts one on
an empty storage, and RestoreReplica opens it again from that storage and
its stored heads, reading the entries one at a time and checking each.
kolo-store keeps a replica on disk, with about
250 bytes of memory per entry. A store holds one log per log ID, so a
replica of another owner's log with the same ID as a stored log needs
another store.
Growing a replica
Update takes a newer signed head, a consistency proof and the new
entries. It checks the head's signature, the proof, every record and
ciphertext, each key against its commitment, and the rebuilt root. If any
check fails it keeps nothing, and the replica stays exactly as it was. Every
change is committed to the storage before it is shown.
- If the owner signed two different histories,
UpdatereturnsErrDiverged, and the two signed heads are the proof. - An older head of the same history returns
ErrRollback. An older head whose root differs from the entries held returnsErrDiverged.
An entry can arrive without its key, if the serving peer does not share
keys. It reads as ErrKeyUnavailable until AddKey gives it the key, after
checking it against the record's commitment. Another key for an entry that
already has one is ErrBadEntry.
Key and EntryInfo tell a sync layer an entry's key, and whether it is
deleted or keyed, without reading its ciphertext. OpenEntry checks one
entry fetched on its own against a verified head and its inclusion proof.
Catching up in steps
A replica far behind catches up in chunks. Update accepts entries that
stop short of the new head, checked by a consistency proof against the root
the replica rebuilt itself. Head stays the last head whose every entry the
replica holds, so Len can be larger during a catch-up.
Target returns the head being filled and its proof. The replica commits
them with each chunk, so its storage always holds what RestoreReplica
needs, and the target is replayed through the checks of Update on a
restore (kolo-store's OpenReplica), so a catch-up resumes after a restart.
While a target is pending, a head between Len and the target's size is
refused unchecked with ErrRollback, which there does not mean the head
shares the replica's history. A chunk toward a larger head must reach the
target's size (ErrBadEntry), and the root rebuilt at the target's size must
match it (ErrDiverged).
Pull only
A pull between two peers
The puller asks and checks; the server only answers. Nothing is stored until it is verified.
- The puller opens a stream on its kolo-secure connection and sends HEAD_REQ with the owner's key and the log ID.
- The server asks its authorize function about this peer and this log. A log it does not offer and a peer with no access get the same not-found error. Otherwise it answers HEAD: the signed head and the log header.
- The puller checks the head's signature. If it already has an older head, PROOF_REQ asks for a consistency proof that the new head extends it.
- ENTRIES_REQ asks for the missing range. Answers hold at most 4 MiB, and Replica.Update checks every record and proof before it stores anything.
- STATE_REQ asks for delete markers and keys. The puller checks and applies the markers, then adds the keys.
- The puller closes the stream. The server never writes to the puller: kolo-sync is pull only.
kolo-sync copies a log between peers, and only by pulling. The peer that wants the copy asks; the peer that serves never writes to anyone. Any peer with a verified copy can serve it, because trust comes from the owner's signatures and not from the peer that sent the bytes.
A log is named by its owner's key and its log ID when it is served and synced. The serving side decides, for each peer and each log, whether the peer gets nothing, the ciphertext only, or the ciphertext and the keys.
One STATE answer looks at most at 32768 entries and reads only the delete markers and keys it sends, and the puller asks for the rest. A pull with nothing new reads no ciphertext on either side. A pull that stops in the middle leaves a target in kolo-store; the next pull fills it first, and returns nil from a peer whose head is below it. An entry whose padded ciphertext does not fit in one message cannot sync, deleted or not, and the log does not sync past it.
Fetch reads a range of entries without a replica, against a head the
client trusts: one from FetchHead or a replica's head. Each entry is
checked against that head with its inclusion proof, and a peer whose head is
behind it gives ErrBehind.
Pinning peers
A pinning peer from kolo-pin is a replica that
keeps an owner's log available while the owner is offline. It pulls the log
with Ciphertext access, so it holds the records, ciphertexts and delete
markers but never the entry keys, and cannot read the entries. Other peers
pull the ciphertext from it, or fetch a range of entries, like from any
replica, by owner key and log ID, and get the keys from the owner's own
devices.
Readers in kolo-kv
A kolo-kv reader shows the longest prefix of its
replica it can replay exactly, up to the last complete verified head. It
stops at the first entry held without its key, and before a destroyed entry
whose delete entry it does not hold yet. Len reports that prefix and can go
down after a Refresh: the reader moves back rather than show a state the
writer never had. It changes state only on Refresh, apart from destroyed
values, which read as missing at once.
What best effort means here
A replica applies the delete markers it pulls: it destroys the entry key and keeps the signed record and marker. That holds on every peer that follows the protocol. A peer that already had the key before the marker arrived can keep a copy of it. See Deletion.