kolo docsbeta

Deletion

Kolo deletes an entry by destroying its key. Deletion is best effort, and this page says exactly what that means.

On this page

A log only grows, so Kolo cannot remove an entry from it. Instead, each entry is encrypted with its own key, and deleting the entry destroys that key.

How it works

All steps

Deleting an entry

Deletion removes the content, not the entry: the record stays, so every head and proof still checks out.

  1. Entry 2 is stored as its signed record, its ciphertext and its entry key.
  2. Delete makes a delete marker: the log, the index and the leaf hash, signed by the owner.
  3. The ciphertext and the key are dropped; the record and the marker stay. The tree and the head do not change.
  4. A replica checks the marker, then drops its own copy of that ciphertext and key. Reading the entry gives ErrDeleted.
  5. A copy made before, or a key someone kept, is out of reach. Deletion is best effort.
  1. The owner calls Delete with one or more indexes. kolo-log signs a delete marker for each entry and commits them all at once, all or nothing: an index out of range deletes nothing, and entries already deleted are skipped.
  2. The signed record stays in the log, so every signed head and every proof still holds.
  3. Replicas pull the markers with kolo-sync and apply them with ApplyDelete, which destroys their copy of each key. It takes many markers and applies them all or none: one bad marker applies none. kolo-sync sends markers to every peer with Ciphertext or Keys access, also for entries the puller already holds, and applies the markers of one STATE answer in one commit.
  4. The storage keeps the record and the marker in place of the ciphertext and the key, in one commit. kolo-store writes it as one synced batch.

After that, reading the entry returns ErrDeleted, and so does kolo-sync's Fetch for an entry with a verified marker. The content, its real size and the time it was written were never public, and they stay private. Restore refuses a deleted entry that still has a key (ErrRestoreMismatch). A peer can leave out a marker and keep serving the live entry, but nobody can make a live entry look deleted without the owner's signed marker.

Deletion is best effort

On your own disk it is best effort too. Pebble no longer references the old value, and compaction later drops it from its files, but the bytes are not erased: a backup, a filesystem snapshot, the write-ahead log or freed disk blocks from before the delete can still hold a working key. Putting an old entry value back is refused while the log is open, but after a reopen it makes the deleted entry readable again with no error. A whole log removed with Store.Delete leaves its old bytes the same way until compaction. In memory, the cipher's own copy of the key stays until the Go runtime reuses that memory, and with the JSON encoding, encoding/json keeps its own buffers, which kolo-log cannot clear.

On a pinning peer

A kolo-pin pinning peer never gets entry keys, so it cannot read an entry before or after a delete. It applies an owner's new delete markers at the owner's next Add, as kolo-sync hands them over, and drops their ciphertexts in the same write. That is still best effort: a pinning peer that does not follow the protocol keeps the ciphertext, and old disk blocks can still hold deleted bytes.

Deleting a key in kolo-kv

A Put over an existing key keeps the older value in the log as its own entry. So kolo-kv Delete appends a delete entry that lists every value of the key since its last delete, then destroys all of them in one write (one Log.Delete). Once a key holds 65536 values since its last delete, the next Put first deletes them.

The deleted key's name stays in its delete entry, so keep secrets in values, not in keys. A reader that gets a delete marker before the delete entry goes back to its state before the destroyed entry until the delete entry arrives.

What you can rely on

  • Every peer that follows the protocol destroys the key once it has the marker.
  • A peer that only ever got the ciphertext, never the key, cannot read the entry, before or after the delete.
  • The delete marker is signed by the owner, so nobody else can delete an entry.

What you cannot rely on: a peer that does not follow the protocol, a peer that copied the key or the plaintext before the delete, and old copies of a disk.