kolo docsbeta

kolo-log

Un log firmado append-only que cualquier peer puede comprobar y del que su propietario aún puede borrar.

En esta página

Las entradas están en un árbol de Merkle cuyo hash y cuyas pruebas siguen Certificate Transparency 2.0 (RFC 9162), con BLAKE2b-256 (RFC 7693) en lugar de SHA-256, y el propietario firma cada registro y cada head con Ed25519ctx (RFC 8032), así que cualquier peer puede verificar que una entrada está en el log y que el log solo ha crecido. Cada entrada se cifra con su propia clave, y borrar una entrada destruye esa clave mientras todas las pruebas siguen siendo válidas, como propusieron Boneh y Lipton. Un log también puede derivar las claves de sus entradas de un secreto de grupo, para que sus lectores lo abran desde cualquier copia mientras quien escribe está sin conexión. Las identidades vienen de kolo-id y las codificaciones de kolo-codec. Un log guarda sus entradas en un Storage: NewMemory para un log que no se persiste, o kolo-store en disco. Consulta Logs y Réplicas.

Todos los pasos

Añadir una entrada

Las entradas no se encadenan unas a otras con hashes. El árbol de Merkle las une, y cada head firmado cubre todas las entradas anteriores.

  1. Append crea una clave de entrada nueva de 32 bytes. El valor se rellena y se cifra con XChaCha20-Poly1305.
  2. El registro guarda el índice, el tamaño, un hash del texto cifrado y un compromiso con la clave. El propietario lo firma.
  3. La hoja es el hash BLAKE2b-256 de 0x00 y el registro firmado. Entra en el árbol por la derecha.
  4. Cada nodo de arriba es el hash de 0x01 y sus dos hijos, hasta una raíz nueva.
  5. El propietario firma un head nuevo: el log, el tamaño y la raíz. Entrada, clave y head se guardan en un solo commit.

Instalación

go get github.com/ciranda-dev/kolo-log

Compila sin cgo.

Inicio rápido

Añade una entrada, demuestra que está en el log y después bórrala.

main.go
package main

import (
	"errors"
	"fmt"
	"log"

	koloid "github.com/ciranda-dev/kolo-id"
	kololog "github.com/ciranda-dev/kolo-log"
)

func main() {
	me, err := koloid.Generate()
	check(err)
	l, err := kololog.New(me, kololog.NewMemory(), kololog.WithValueEncoding(kololog.UTF8))
	check(err)
	defer l.Close()
	i, err := l.Append("hello")
	check(err)

	// A reader trusts only the owner's public key and the log ID.
	h, err := l.Head()
	check(err)
	head, err := kololog.VerifyHead(me.PublicKey(), l.ID(), h)
	check(err)
	rec, err := l.Record(i)
	check(err)
	proof, err := l.InclusionProof(i, head.Size())
	check(err)
	fmt.Println("included:", kololog.VerifyInclusion(head, rec, proof))

	check(l.Delete(i))
	var s string
	fmt.Println("deleted:", errors.Is(l.Read(i, &s), kololog.ErrDeleted))
	fmt.Println("still included:", kololog.VerifyInclusion(head, rec, proof))
}

func check(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

Imprime included: true, deleted: true y still included: true: la entrada ya no se puede leer, pero el head firmado y la prueba siguen siendo válidas.

Referencia de la API

La referencia de la API se genera a partir del código Go y está en inglés.

Paquete kololog · github.com/ciranda-dev/kolo-log · v0.1.0

Package kololog is a signed append-only log on a Merkle tree for Kolo. Each entry has its own key, random by default or derived from a group secret with WithDerivedKeys; deleting an entry destroys its stored key, not the proofs.

This package is experimental and its API may change before v1.0.0.

Constantes

const (
	// DerivedHeaderContext is the signing context of the header of a log with [DerivedKeys].
	DerivedHeaderContext = "kolo-log/derived-header/v0"
	// EntryKeyContext starts the HKDF info of a derived entry key.
	EntryKeyContext = "kolo-log/entry-key/v0"
)
const (
	// HeadContext is the signing context of tree heads.
	HeadContext = "kolo-log/tree-head/v0"
	// RecordContext is the signing context of entry records.
	RecordContext = "kolo-log/record/v1"
	// KeyContext starts the hashed input of an entry key commitment.
	KeyContext = "kolo-log/key/v0"
)
const DeleteContext = "kolo-log/delete/v0"

DeleteContext is the signing context of delete markers.

const EntrySecretSize = 32

EntrySecretSize is the size in bytes of an EntrySecret.

const HashSize = blake2b.Size256

HashSize is the size in bytes of a tree hash.

const HeaderContext = "kolo-log/header/v0"

HeaderContext is the signing context of log headers.

const LogIDSize = 32

LogIDSize is the size in bytes of a log ID.

const MaxAddKeys = 32768

MaxAddKeys is the most keys one Replica.AddKeys call takes: as many entries as one kolo-sync answer can carry, its 4 MiB message bound over the 128-byte smallest signed record (kolo-sync wire.go, MaxMessageSize and minSigned).

Variables

var (
	// ErrNotDerived is returned by [NewKeyDeriver] for a header with [RandomKeys].
	ErrNotDerived = errors.New("kololog: log does not use derived keys")
	// ErrKeyModeMismatch is returned by [Restore] when the header's key mode or epoch differs from the options.
	ErrKeyModeMismatch = errors.New("kololog: key mode mismatch")
	// ErrSecretMismatch is returned by [Restore] when the entry secret does not derive the newest entry's key.
	ErrSecretMismatch = errors.New("kololog: entry secret mismatch")
	// ErrMissingSecret is returned by [New], [Restore] and [NewKeyDeriver] for a nil entry secret.
	ErrMissingSecret = errors.New("kololog: missing entry secret")
	// ErrDeriverClosed is returned by [KeyDeriver.Key] after [KeyDeriver.Close].
	ErrDeriverClosed = errors.New("kololog: key deriver closed")
)
var (
	// ErrWrongValueType is returned when a value or output does not match the log's value encoding.
	ErrWrongValueType = errors.New("kololog: wrong value type")
	// ErrInvalidUTF8 is returned by a UTF8 log for a string that is not valid UTF-8.
	ErrInvalidUTF8 = errors.New("kololog: invalid UTF-8")
	// ErrUnknownEncoding is returned for a value encoding other than Bytes, UTF8 or JSON.
	ErrUnknownEncoding = errors.New("kololog: unknown value encoding")
)
var (
	// ErrInvalidHead is returned by [VerifyHead] for any rejected head; the errors below wrap it.
	ErrInvalidHead = errors.New("kololog: invalid head")
	// ErrBadHeadSignature is returned when a head is not signed by the given key or names another key.
	ErrBadHeadSignature error = &headError{"kololog: invalid head signature"}
	// ErrWrongLogID is returned when a correctly signed head belongs to another log.
	ErrWrongLogID error = &headError{"kololog: head belongs to another log"}
)
var (
	// ErrDeleted is returned by [Log.Get] for an entry whose key was destroyed by [Log.Delete].
	ErrDeleted = errors.New("kololog: entry deleted")
	// ErrOutOfRange is returned for an index or tree size the log does not have.
	ErrOutOfRange = errors.New("kololog: index out of range")
	// ErrMalformed is returned when an encoding cannot be parsed.
	ErrMalformed = errors.New("kololog: malformed encoding")
)
var (
	// ErrBadLogHeader is returned when a log header is not signed by the expected key.
	ErrBadLogHeader = errors.New("kololog: bad log header")
	// ErrMissingHeader is returned by [Restore] and [RestoreReplica] for a nil or empty header.
	ErrMissingHeader = errors.New("kololog: missing log header")
	// ErrEncodingMismatch is returned by [Restore] when the header's value encoding differs from the options.
	ErrEncodingMismatch = errors.New("kololog: value encoding mismatch")
)
var (
	// ErrLogOpen is returned by [New], [Restore] and the Reserve functions when a writable log with the same author key and log ID
	// is already open in this process, or that pair or log ID is reserved.
	ErrLogOpen = errors.New("kololog: log already open in this process")
	// ErrLogClosed is returned by writes and Head after [Log.Close].
	ErrLogClosed = errors.New("kololog: log closed")
)
var (
	// ErrDiverged is returned by [Replica.Update] when the owner signed two histories: a head whose root differs from the held
	// entries at its size, a head of the target's size with another root, or entries toward a newer head that miss the target's root.
	ErrDiverged = errors.New("kololog: head diverges from the held entries or the pending target")
	// ErrBadProof is returned by [Replica.Update] when the consistency proof does not verify, and by [OpenEntry] when the inclusion proof does not.
	ErrBadProof = errors.New("kololog: consistency proof does not verify")
	// ErrBadEntry is returned by [Replica.Update], [Replica.AddKey], [Replica.ApplyDelete] and [OpenEntry] for an entry, key or marker that does not match its signed record or the head, and wrapped by [BadKeyError].
	ErrBadEntry = errors.New("kololog: entry does not match its signed record or the head")
	// ErrNoHead is returned by [Replica.Head] before the replica has a verified head.
	ErrNoHead = errors.New("kololog: replica has no verified head")
)
var (
	// ErrRestoreMismatch is returned by [Restore] when stored data does not match the signed head or records.
	ErrRestoreMismatch = errors.New("kololog: restored data does not match the signed head or records")
	// ErrRollback is returned by [Restore] when the snapshot is older than or forks from a trusted head, and by [Replica.Update]
	// for a head of the held history smaller than the verified one, or, unchecked, smaller than a pending target.
	ErrRollback = errors.New("kololog: snapshot or head is older than a known head or forks from it")
	// ErrKeyUnavailable is returned when an entry's ciphertext is present but its key is not in this log.
	ErrKeyUnavailable = errors.New("kololog: entry key not available")
)
var (
	// ErrNoStorage is returned by [New], [Restore], [NewReplica] and [RestoreReplica] for a nil [Storage].
	ErrNoStorage = errors.New("kololog: nil storage")
	// ErrStorageMismatch is returned by a read whose [Storage] returns an entry or key that does not match the entry's verified record.
	ErrStorageMismatch = errors.New("kololog: stored entry does not match its verified record")
	// ErrStorageNotEmpty is returned by [New] and [NewReplica] for a [Storage] that already holds an entry.
	ErrStorageNotEmpty = errors.New("kololog: storage already holds entries")
)
var ErrEntryTooLarge = errors.New("kololog: entry too large")

ErrEntryTooLarge is returned by Log.Append for an entry over the size cap. The cap is about 254 GiB on 64-bit and about 512 MiB on 32-bit platforms.

var ErrMalformedEntry = errors.New("kololog: malformed entry")

ErrMalformedEntry is returned by Log.Get when a stored ciphertext does not open to a valid entry.

var ErrNotDecodable = errors.New("kololog: a VerifiedHead cannot be decoded; decode a Head and call VerifyHead")

ErrNotDecodable is returned by VerifiedHead.UnmarshalJSON, always.

var ErrTooManyKeys = errors.New("kololog: too many keys in one call")

ErrTooManyKeys is returned by Replica.AddKeys for more than MaxAddKeys keys.

Funciones

func OpenEntry

func OpenEntry(h VerifiedHead, p InclusionProof, e Entry) ([]byte, error)

OpenEntry checks an entry received on its own against a verified head and the entry's inclusion proof in it, then returns its stored bytes as Log.Get does. Its record, ciphertext or marker, and key are checked by the same code as in Replica.Update. It returns ErrInvalidHead for a zero head, ErrBadProof for a proof that does not put the record in h, ErrBadEntry for a part that does not match the record, and, after those checks, ErrDeleted for an entry with a valid delete marker and ErrKeyUnavailable for one without its key.

func Reserve

func Reserve(logKey ed25519.PublicKey, id LogID) (release func(), err error)

Reserve keeps New and Restore from opening a writable log with this key and ID until release is called. It returns ErrLogOpen if such a log is open or the pair is already reserved.

func ReserveID

func ReserveID(id LogID) (release func(), err error)

ReserveID reserves a log ID under every key, for storage that cannot trust which key a log belongs to. It returns ErrLogOpen if a log with that ID is open or reserved under any key; release is idempotent.

func VerifyConsistency

func VerifyConsistency(old, new VerifiedHead, p ConsistencyProof) bool

VerifyConsistency reports whether old and new belong to the same log and p proves old is a prefix of new.

func VerifyDeleteMarker

func VerifyDeleteMarker(pub ed25519.PublicKey, logID LogID, r Record, m DeleteMarker) bool

VerifyDeleteMarker reports whether m is pub's valid marker for record r of the log logID. It never panics on untrusted input.

func VerifyInclusion

func VerifyInclusion(h VerifiedHead, r Record, p InclusionProof) bool

VerifyInclusion reports whether r is signed by h's log key, belongs to h's log, and p proves r is in h.

func VerifyLogHeader

func VerifyLogHeader(pub ed25519.PublicKey, h LogHeader) error

VerifyLogHeader returns ErrBadLogHeader unless h is signed by pub under the context of its key mode; it refuses an unknown key mode and a header with RandomKeys and a nonzero epoch. It never panics on untrusted input.

func VerifyRecord

func VerifyRecord(pub ed25519.PublicKey, logID LogID, r Record) bool

VerifyRecord reports whether r is signed by pub and belongs to the log logID. It never panics on untrusted input.

Tipos

type BadKeyError

type BadKeyError struct {
	Indexes []uint64 // ascending
}

BadKeyError is the error of Replica.AddKeys for the keys that do not belong to their entries; it wraps ErrBadEntry.

func (BadKeyError) Error
func (e *BadKeyError) Error() string

Error returns how many keys were refused and the first refused index.

func (BadKeyError) Unwrap
func (e *BadKeyError) Unwrap() error

Unwrap returns ErrBadEntry.

type Change

type Change struct {
	Start   uint64     // the log's length before the change, in every Change: Entries are stored from this index on
	Entries []Entry    // new entries, each with its key when it has one
	Deleted []Deletion // entries whose ciphertext and key are replaced by their delete marker
	Keys    []KeyAt    // keys of entries stored without one, in ascending index order
	Heads   *Heads     // when not nil, replaces the stored heads
}

Change is one atomic change a log makes to its Storage.

type ConsistencyProof

type ConsistencyProof struct {
	OldSize uint64
	NewSize uint64
	Path    [][HashSize]byte
}

ConsistencyProof shows that the tree of OldSize entries is a prefix of the tree of NewSize entries.

func DecodeConsistencyProof
func DecodeConsistencyProof(b []byte) (ConsistencyProof, error)

DecodeConsistencyProof parses an encoding made by ConsistencyProof.Encode.

func (ConsistencyProof) Encode
func (p ConsistencyProof) Encode() []byte

Encode returns the canonical encoding of p.

func (ConsistencyProof) LogValue
func (p ConsistencyProof) LogValue() slog.Value

LogValue returns p as a slog group with full hex values.

func (ConsistencyProof) MarshalJSON
func (p ConsistencyProof) MarshalJSON() ([]byte, error)

MarshalJSON encodes p as a JSON view.

func (ConsistencyProof) String
func (p ConsistencyProof) String() string

String returns a short summary of p.

func (ConsistencyProof) UnmarshalJSON
func (p *ConsistencyProof) UnmarshalJSON(b []byte) error

UnmarshalJSON strictly decodes the view made by ConsistencyProof.MarshalJSON.

type DeleteMarker

type DeleteMarker struct {
	LogKey    ed25519.PublicKey
	LogID     LogID
	Index     uint64
	LeafHash  [HashSize]byte
	Signature []byte
}

DeleteMarker is the log owner's signed statement that the entry with this leaf hash was deleted.

func DecodeDeleteMarker
func DecodeDeleteMarker(b []byte) (DeleteMarker, error)

DecodeDeleteMarker parses an encoding made by DeleteMarker.Encode.

func (DeleteMarker) Encode
func (m DeleteMarker) Encode() []byte

Encode returns the canonical encoding of m.

func (DeleteMarker) LogValue
func (m DeleteMarker) LogValue() slog.Value

LogValue returns m as a slog group with full hex values.

func (DeleteMarker) MarshalJSON
func (m DeleteMarker) MarshalJSON() ([]byte, error)

MarshalJSON encodes m as a JSON view; the binary encoding is the only one that is signed.

func (DeleteMarker) String
func (m DeleteMarker) String() string

String returns a short summary of m.

func (DeleteMarker) UnmarshalJSON
func (m *DeleteMarker) UnmarshalJSON(b []byte) error

UnmarshalJSON strictly decodes the view made by DeleteMarker.MarshalJSON.

type Deletion

type Deletion struct {
	Index  uint64
	Record []byte
	Marker []byte
}

Deletion is the stored form of a deleted entry: its record and its delete marker.

type Entry

type Entry struct {
	Record     []byte   // Record.Encode() of the entry, exactly as hashed into the tree
	Ciphertext []byte   // nonce || sealed padded frame; nil after Delete
	Marker     []byte   // DeleteMarker.Encode() of a deleted entry; nil otherwise
	Key        EntryKey `json:"-"` // the entry key; nil when unknown or deleted
}

Entry is the stored form of one entry: its signed record, either its ciphertext or its delete marker, and its key when known. encoding/json leaves Key out; an encoder that ignores MarshalJSON, such as encoding/gob, writes the key in clear.

type EntryInfo

type EntryInfo struct {
	Deleted bool // a delete marker is stored in place of the ciphertext and key
	Keyed   bool // the entry key is stored
}

EntryInfo is what the log keeps in memory about the stored form of an entry.

type EntryKey

type EntryKey []byte

EntryKey is the encryption key of one entry; formatted or logged on its own, or as a field fmt and slog can reach, it shows only as [redacted].

func (EntryKey) Format
func (EntryKey) Format(f fmt.State, _ rune)

Format writes [redacted] for every verb, so no fmt verb prints the key.

func (EntryKey) LogValue
func (EntryKey) LogValue() slog.Value

LogValue returns [redacted].

func (EntryKey) MarshalJSON
func (EntryKey) MarshalJSON() ([]byte, error)

MarshalJSON returns "[redacted]"; send a key only in a binary encoding over a secure channel.

func (EntryKey) String
func (EntryKey) String() string

String returns [redacted].

type EntrySecret

type EntrySecret [EntrySecretSize]byte

EntrySecret is the group secret of a log with DerivedKeys: 32 uniformly random bytes used for nothing else. A guessable secret can be found offline from any record, because a record's KeyCommit is public. Every copy of the array is another copy of the secret. fmt, slog and JSON show it as [redacted], but it prints raw inside an unexported struct field (by value or behind one pointer), with %p of a value, and in encoding/gob.

func (EntrySecret) Format
func (EntrySecret) Format(f fmt.State, _ rune)

Format writes [redacted] for every verb.

func (EntrySecret) GoString
func (EntrySecret) GoString() string

GoString returns [redacted].

func (EntrySecret) LogValue
func (EntrySecret) LogValue() slog.Value

LogValue returns [redacted].

func (EntrySecret) MarshalJSON
func (EntrySecret) MarshalJSON() ([]byte, error)

MarshalJSON returns "[redacted]".

func (EntrySecret) String
func (EntrySecret) String() string

String returns [redacted].

type Head

type Head struct {
	LogKey    ed25519.PublicKey
	LogID     LogID
	Size      uint64
	Root      [HashSize]byte
	Signature []byte
}

Head is a signed statement of the size and root hash of a log, as sent over the network.

func DecodeHead
func DecodeHead(b []byte) (Head, error)

DecodeHead parses an encoding made by Head.Encode.

func (Head) Encode
func (h Head) Encode() []byte

Encode returns the canonical encoding of h.

func (Head) LogValue
func (h Head) LogValue() slog.Value

LogValue returns h as a slog group with full hex values.

func (Head) MarshalJSON
func (h Head) MarshalJSON() ([]byte, error)

MarshalJSON encodes h as a JSON view; the binary encoding is the only one that is signed.

func (Head) String
func (h Head) String() string

String returns a short summary of h.

func (Head) UnmarshalJSON
func (h *Head) UnmarshalJSON(b []byte) error

UnmarshalJSON strictly decodes the view made by Head.MarshalJSON; verify the result with VerifyHead.

type Heads

type Heads struct {
	Head   *Head            // the last head whose every entry is stored; nil for a replica with no head yet
	Target *Head            // a replica's catch-up target, past Head; nil when none is pending
	Proof  ConsistencyProof // with Target: the proof that the stored entries are a prefix of it
}

Heads is what a log stores about its heads.

type InclusionProof

type InclusionProof struct {
	Index uint64
	Size  uint64
	Path  [][HashSize]byte
}

InclusionProof shows that the entry at Index is in the tree of Size entries.

func DecodeInclusionProof
func DecodeInclusionProof(b []byte) (InclusionProof, error)

DecodeInclusionProof parses an encoding made by InclusionProof.Encode.

func (InclusionProof) Encode
func (p InclusionProof) Encode() []byte

Encode returns the canonical encoding of p.

func (InclusionProof) LogValue
func (p InclusionProof) LogValue() slog.Value

LogValue returns p as a slog group with full hex values.

func (InclusionProof) MarshalJSON
func (p InclusionProof) MarshalJSON() ([]byte, error)

MarshalJSON encodes p as a JSON view.

func (InclusionProof) String
func (p InclusionProof) String() string

String returns a short summary of p.

func (InclusionProof) UnmarshalJSON
func (p *InclusionProof) UnmarshalJSON(b []byte) error

UnmarshalJSON strictly decodes the view made by InclusionProof.MarshalJSON.

type KeyAt

type KeyAt struct {
	Index uint64
	Key   EntryKey
}

KeyAt is the key of entry Index, in a Change.

type KeyDeriver

type KeyDeriver struct {
	// contains filtered or unexported fields
}

KeyDeriver derives the entry keys of one log with DerivedKeys; it is safe for concurrent use. fmt and slog show a *KeyDeriver as [redacted]; a KeyDeriver value prints its log key, log ID and epoch, never the PRK.

func NewKeyDeriver
func NewKeyDeriver(secret *EntrySecret, owner ed25519.PublicKey, h LogHeader) (*KeyDeriver, error)

NewKeyDeriver checks h with VerifyLogHeader(owner, h) and returns its deriver, or ErrNotDerived for a header with RandomKeys.

func (KeyDeriver) Close
func (d *KeyDeriver) Close()

Close clears the PRK; Key calls that hold the read lock finish first. HMAC's own copies stay in memory until the runtime reuses them.

func (KeyDeriver) Format
func (*KeyDeriver) Format(f fmt.State, _ rune)

Format writes [redacted] for every verb.

func (KeyDeriver) GoString
func (*KeyDeriver) GoString() string

GoString returns [redacted].

func (KeyDeriver) Key
func (d *KeyDeriver) Key(index uint64) (EntryKey, error)

Key returns the key of entry index, or ErrDeriverClosed after Close; the caller should clear it.

func (KeyDeriver) LogValue
func (*KeyDeriver) LogValue() slog.Value

LogValue returns [redacted].

func (KeyDeriver) String
func (*KeyDeriver) String() string

String returns [redacted].

type KeyMode

type KeyMode uint8

KeyMode is how a log makes its entry keys.

const (
	// RandomKeys gives every entry a fresh random key that only the writer holds; it is the default.
	RandomKeys KeyMode = iota
	// DerivedKeys derives every entry key from an [EntrySecret], the log and the entry's index.
	DerivedKeys
)
func (KeyMode) String
func (m KeyMode) String() string

String returns "random" or "derived".

type Log

type Log struct {
	// contains filtered or unexported fields
}

Log is a signed append-only log owned by one writer; it is safe for concurrent use. It keeps its tree hashes and a compact copy of each record in memory, and its entries in its Storage.

func New
func New(id *koloid.Identity, s Storage, opts ...Option) (*Log, error)

New returns an empty Log with a random log ID and a signed LogHeader, which keeps its entries in s. s must belong to this log only. New stores nothing: the caller stores the header if it keeps the log. It returns koloid.ErrInvalidIdentity for a nil or zero identity instead of panicking, ErrNoStorage for a nil s, ErrStorageNotEmpty for an s that returns an entry at index 0, and the storage's error if it fails to read index 0.

func Restore
func Restore(id *koloid.Identity, header, head []byte, s Storage, opts ...Option) (*Log, error)

Restore rebuilds a writable log from its persisted header and head and the entries in s, verifying all of it. It reads the head's entries from s once each in increasing order, checks them in parallel and keeps only their records; the error is the first bad entry in log order. A live entry without its key returns ErrRestoreMismatch.

func (Log) Append
func (l *Log) Append(v any) (uint64, error)

Append encodes v with the log's value encoding, encrypts it, stores it with a head signed over it in one Storage.Commit and returns its index. If the commit fails, the log is unchanged and the error wraps the storage's. With JSON, encoding/json keeps its own internal buffers, which kololog cannot clear.

func (Log) Close
func (l *Log) Close() error

Close releases the log ID for another writable log in this process and clears a derived log's PRK; reads keep working. Append, Head and Delete then return ErrLogClosed. Closing twice is not an error.

func (Log) ConsistencyProof
func (l *Log) ConsistencyProof(oldSize, newSize uint64) (ConsistencyProof, error)

ConsistencyProof returns a proof that the tree of oldSize entries is a prefix of the tree of newSize entries.

func (Log) Delete
func (l *Log) Delete(indexes ...uint64) error

Delete signs a DeleteMarker for each entry at indexes and stores them in place of the entries' ciphertexts and keys in one Storage.Commit, so deleting many entries costs one durable write; the records and every proof stay valid. An index out of range returns ErrOutOfRange and deletes nothing. Entries already deleted, and repeated indexes, are skipped; if the commit fails, every entry stays as it was.

func (Log) Entry
func (l *Log) Entry(index uint64) (Entry, error)

Entry returns the stored form of the entry at index, its key included; clear Key when done with it. It reads the entry from storage and checks it against the entry's record: a deleted entry must come with its marker only.

func (Log) EntryInfo
func (l *Log) EntryInfo(index uint64) (EntryInfo, error)

EntryInfo returns whether the entry at index is deleted and whether its key is stored, without reading storage.

func (Log) Format
func (l *Log) Format(f fmt.State, _ rune)

Format writes l.String() for every verb, so no verb prints the log's storage and the entry keys in it.

func (Log) Get
func (l *Log) Get(index uint64) ([]byte, error)

Get returns the stored bytes of the entry at index, ErrDeleted after Log.Delete, or ErrKeyUnavailable for an entry stored without its key. It reads the entry from storage and checks it against the entry's record first.

func (Log) GoString
func (l *Log) GoString() string

GoString returns l.String().

func (Log) Head
func (l *Log) Head() (Head, error)

Head returns a signed head for the current tree.

func (Log) Header
func (l *Log) Header() LogHeader

Header returns a copy of the signed log header.

func (Log) ID
func (l *Log) ID() LogID

ID returns the random ID of the log.

func (Log) InclusionProof
func (l *Log) InclusionProof(index, size uint64) (InclusionProof, error)

InclusionProof returns a proof that the entry at index is in the tree of the first size entries.

func (Log) Key
func (l *Log) Key(index uint64) (EntryKey, error)

Key returns the key of the entry at index, read from storage without its ciphertext and checked against the record's key commitment; clear it when done. A deleted entry returns ErrDeleted, one stored without its key ErrKeyUnavailable.

func (Log) Len
func (l *Log) Len() uint64

Len returns the number of entries in the log, deleted ones included.

func (Log) LogValue
func (l *Log) LogValue() slog.Value

LogValue returns l's log ID, value encoding, key mode and epoch as a slog group.

func (Log) Read
func (l *Log) Read(index uint64, out any) error

Read decodes the entry at index into out with the log's value encoding. With JSON, encoding/json keeps its own internal buffers, which kololog cannot clear.

func (Log) Record
func (l *Log) Record(index uint64) (Record, error)

Record returns the public record of the entry at index, which stays available after Log.Delete.

func (Log) String
func (l *Log) String() string

String returns a short summary of l: its log ID, value encoding, key mode and epoch.

type LogHeader

type LogHeader struct {
	LogKey    ed25519.PublicKey
	LogID     LogID
	Encoding  ValueEncoding
	Keys      KeyMode
	Epoch     uint64 // 0 with RandomKeys
	Signature []byte
}

LogHeader is the signed description of a log, persisted next to its data by the storage layer. A header with RandomKeys has the encoding of kolo-log v0.1.6; one with DerivedKeys has its own tag and signing context.

func DecodeLogHeader
func DecodeLogHeader(b []byte) (LogHeader, error)

DecodeLogHeader parses an encoding made by LogHeader.Encode.

func (LogHeader) Encode
func (h LogHeader) Encode() []byte

Encode returns the canonical encoding of h.

func (LogHeader) LogValue
func (h LogHeader) LogValue() slog.Value

LogValue returns h as a slog group with full hex values.

func (LogHeader) MarshalJSON
func (h LogHeader) MarshalJSON() ([]byte, error)

MarshalJSON encodes h as a JSON view; the binary encoding is the only one that is signed. A header with DerivedKeys adds "keys": "derived" and "epoch".

func (LogHeader) String
func (h LogHeader) String() string

String returns a short summary of h.

func (LogHeader) UnmarshalJSON
func (h *LogHeader) UnmarshalJSON(b []byte) error

UnmarshalJSON strictly decodes the view made by LogHeader.MarshalJSON: the four fields of a header with random keys, or those and both "keys": "derived" and "epoch".

type LogID

type LogID [LogIDSize]byte

LogID is the random ID that separates logs under the same identity.

func (LogID) LogValue
func (id LogID) LogValue() slog.Value

LogValue returns the log ID as lowercase hex.

func (LogID) MarshalJSON
func (id LogID) MarshalJSON() ([]byte, error)

MarshalJSON encodes the log ID as a lowercase hex string.

func (LogID) String
func (id LogID) String() string

String returns the log ID as lowercase hex.

func (LogID) UnmarshalJSON
func (id *LogID) UnmarshalJSON(b []byte) error

UnmarshalJSON decodes a lowercase hex string of exactly 32 bytes.

type Option

type Option func(*options)

Option configures New and Restore.

func ReserveForRestore
func ReserveForRestore(logKey ed25519.PublicKey, id LogID) (opt Option, release func(), err error)

ReserveForRestore reserves like Reserve; passing opt to Restore of that key and ID lets it take the reservation over. release is idempotent and does nothing after a takeover; after a failed Restore the reservation stays until release.

func WithDerivedKeys
func WithDerivedKeys(secret *EntrySecret, epoch uint64) Option

WithDerivedKeys makes New derive entry keys from secret under epoch, and makes Restore require a log that does. It copies secret when called; the copy lives as long as the Option. A nil secret makes New and Restore return ErrMissingSecret.

func WithTrustedHead
func WithTrustedHead(h Head) Option

WithTrustedHead makes Restore reject a snapshot that does not extend h; New ignores it. Every trusted head given is checked.

func WithValueEncoding
func WithValueEncoding(e ValueEncoding) Option

WithValueEncoding sets the value encoding of the log; the default is Bytes.

type Record

type Record struct {
	LogKey      ed25519.PublicKey
	LogID       LogID
	Index       uint64
	Size        uint64         // padded ciphertext length in bytes
	ContentHash [HashSize]byte // BLAKE2b-256 of the ciphertext
	KeyCommit   [HashSize]byte // BLAKE2b-256 of KeyContext and the entry key, so one record opens to one plaintext only
	Signature   []byte
}

Record is the signed public metadata of one entry; its encoding is the Merkle leaf.

func DecodeRecord
func DecodeRecord(b []byte) (Record, error)

DecodeRecord parses an encoding made by Record.Encode.

func (Record) Encode
func (r Record) Encode() []byte

Encode returns the canonical encoding of r.

func (Record) LogValue
func (r Record) LogValue() slog.Value

LogValue returns r as a slog group with full hex values.

func (Record) MarshalJSON
func (r Record) MarshalJSON() ([]byte, error)

MarshalJSON encodes r as a JSON view; the binary encoding is the only one that is signed.

func (Record) String
func (r Record) String() string

String returns a short summary of r.

func (Record) UnmarshalJSON
func (r *Record) UnmarshalJSON(b []byte) error

UnmarshalJSON strictly decodes the view made by Record.MarshalJSON.

type Replica

type Replica struct {
	// contains filtered or unexported fields
}

Replica is a read-only copy of a log, opened from the owner's public key; it is safe for concurrent use. It grows only through Replica.Update, which verifies everything before it commits any of it to its Storage.

func NewReplica
func NewReplica(pub ed25519.PublicKey, logID LogID, header []byte, s Storage) (*Replica, error)

NewReplica returns an empty replica of the log logID owned by pub, from the log's signed header, keeping its entries in s. s must belong to this replica only: one that returns an entry at index 0 gives ErrStorageNotEmpty, and a failed read of index 0 the storage's error. NewReplica stores nothing: the caller stores the header if it keeps the replica.

func RestoreReplica
func RestoreReplica(pub ed25519.PublicKey, logID LogID, header []byte, heads Heads, s Storage) (*Replica, error)

RestoreReplica rebuilds a replica from its persisted header and heads and the entries in s, verifying all of it like Restore, then replays a pending heads.Target through the checks of Replica.Update. Zero heads are a replica that has not grown yet. Errors in the stored data wrap ErrRestoreMismatch.

func (Replica) AddKey
func (r *Replica) AddKey(index uint64, key []byte) error

AddKey gives a held entry that came without its key the key, after checking it against the record's key commitment and that it opens the ciphertext read from storage, and commits it. The key an entry already has does nothing; another key returns ErrBadEntry. The caller should clear key.

func (Replica) AddKeys
func (r *Replica) AddKeys(start uint64, keys []EntryKey) error

AddKeys checks every key in keys against the held entries from start on, and stores every key that passes in one Storage.Commit. A nil or empty key skips its entry, and so does an entry that is deleted or already keyed. Keys the signed record proves wrong are refused together in one *BadKeyError, and the keys that passed are still stored. A storage error or ErrStorageMismatch stores nothing. More than MaxAddKeys keys is ErrTooManyKeys, and keys past Len ErrOutOfRange. The caller should clear keys.

func (Replica) ApplyDelete
func (r *Replica) ApplyDelete(markers ...[]byte) error

ApplyDelete verifies encoded [DeleteMarker]s for entries the replica holds, then commits them in place of those entries' ciphertexts and keys in one Storage.Commit. The records and the markers stay, and reads return ErrDeleted. A marker that does not decode, is out of range or does not match its entry returns an error and applies none of them; a marker for an entry already deleted, or repeated, is skipped.

func (Replica) Close
func (r *Replica) Close() error

Close stops Update, AddKey, AddKeys and ApplyDelete, which then return ErrLogClosed; reads keep working.

func (Replica) ConsistencyProof
func (r *Replica) ConsistencyProof(oldSize, newSize uint64) (ConsistencyProof, error)

ConsistencyProof returns a proof that the tree of oldSize entries is a prefix of the tree of newSize entries.

func (Replica) Entry
func (r *Replica) Entry(index uint64) (Entry, error)

Entry returns the stored form of the entry at index, its key included; clear Key when done with it.

func (Replica) EntryInfo
func (r *Replica) EntryInfo(index uint64) (EntryInfo, error)

EntryInfo returns whether the entry at index is deleted and whether its key is stored, without reading storage.

func (Replica) Format
func (r *Replica) Format(f fmt.State, _ rune)

Format writes r.String() for every verb.

func (Replica) Get
func (r *Replica) Get(index uint64) ([]byte, error)

Get returns the stored bytes of the entry at index, ErrKeyUnavailable without its key, or ErrDeleted.

func (Replica) GoString
func (r *Replica) GoString() string

GoString returns r.String().

func (Replica) Head
func (r *Replica) Head() (Head, error)

Head returns the last verified head whose every entry the replica holds, exactly as received, or ErrNoHead.

func (Replica) Header
func (r *Replica) Header() LogHeader

Header returns a copy of the signed log header.

func (Replica) ID
func (r *Replica) ID() LogID

ID returns the log ID.

func (Replica) InclusionProof
func (r *Replica) InclusionProof(index, size uint64) (InclusionProof, error)

InclusionProof returns a proof that the entry at index is in the tree of the first size entries.

func (Replica) Key
func (r *Replica) Key(index uint64) (EntryKey, error)

Key returns the key of the entry at index without reading its ciphertext, as Log.Key does.

func (Replica) Len
func (r *Replica) Len() uint64

Len returns the number of verified entries, deleted ones included; during a catch-up it can exceed Head's size.

func (Replica) LogValue
func (r *Replica) LogValue() slog.Value

LogValue returns r's log ID, value encoding, key mode and epoch as a slog group.

func (Replica) Read
func (r *Replica) Read(index uint64, out any) error

Read decodes the entry at index into out with the log's value encoding.

func (Replica) Record
func (r *Replica) Record(index uint64) (Record, error)

Record returns the public record of the entry at index.

func (Replica) String
func (r *Replica) String() string

String returns a short summary of r: its log ID, value encoding, key mode and epoch.

func (Replica) Target
func (r *Replica) Target() (h Head, p ConsistencyProof, ok bool)

Target returns the head an unfinished catch-up is filling and the proof that the first Len entries are a prefix of it, or ok false. The replica commits both with each chunk, and RestoreReplica replays them.

func (Replica) Update
func (r *Replica) Update(h Head, p ConsistencyProof, entries []Entry) error

Update verifies h, the entries from Len on and a consistency proof p, then commits them all in one Storage.Commit; on any error the replica is unchanged. The entries may stop at any size m up to h's size, so a replica far behind catches up in chunks; p then proves the tree of m entries, rebuilt from every verified leaf, is a prefix of h (RFC 9162). When the entries reach h's size, p may also be the proof from Len, or anything for an empty replica, and the rebuilt root must equal h's root. Until a chunk reaches h, h is the target (Replica.Target) and Replica.Head stays the last head whose every entry is held. A head whose entries the replica already holds becomes its head or is a no-op; an older one returns ErrRollback if it matches the held entries and ErrDiverged if not, as does a head of the size of the head or target with another root. While a target is pending, a head between Len and the target's size is refused unchecked with ErrRollback, a chunk toward a larger head must reach the target's size (ErrBadEntry otherwise), and the rebuilt root there must equal the target's root (ErrDiverged otherwise). An entry may come without a key and then reads as ErrKeyUnavailable; a deleted one carries its marker.

type Storage

type Storage interface {
	// Entry returns the stored form of the entry at index, its key included, as copies the log may keep or clear.
	// For an index that holds no entry it returns an error wrapping [ErrOutOfRange]; any other error means the storage failed.
	// Restore and RestoreReplica ask for each index once, in increasing order, from one goroutine at a time that is not the caller's,
	// and after a bad entry may still ask for a few more.
	Entry(index uint64) (Entry, error)
	// Key returns the stored key of the entry at index, nil if it has none, as a copy the log clears.
	Key(index uint64) (EntryKey, error)
	// Commit stores c atomically and durably, or nothing at all when it returns an error. It must copy what it keeps:
	// the log zeroes the keys in c after Commit returns.
	Commit(c Change) error
}

Storage keeps one log's entries and heads. A log or replica keeps only tree hashes and a compact copy of each record in memory, reads entries from its Storage and checks them against the record, and makes every change through Commit. The log calls Storage with its own lock held: Commit never runs at the same time as another call, reads may run together.

func NewMemory
func NewMemory() Storage

NewMemory returns an empty in-memory Storage, for one log or replica that is not persisted.

type ValueEncoding

type ValueEncoding uint8

ValueEncoding is how Log.Append encodes values and Log.Read decodes them.

const (
	// Bytes stores []byte values unchanged; it is the default.
	Bytes ValueEncoding = iota
	// UTF8 stores string values, which must be valid UTF-8.
	UTF8
	// JSON stores any value as its encoding/json form.
	JSON
)
func (ValueEncoding) String
func (e ValueEncoding) String() string

String returns "bytes", "utf8" or "json".

type VerifiedHead

type VerifiedHead struct {
	// contains filtered or unexported fields
}

VerifiedHead is a Head whose signature and log ID VerifyHead has checked. Only VerifyHead can build one; the zero value verifies nothing.

func VerifyHead
func VerifyHead(pub ed25519.PublicKey, logID LogID, h Head) (VerifiedHead, error)

VerifyHead checks that h is signed by pub and belongs to the log logID. It never panics on untrusted input.

func (VerifiedHead) Head
func (v VerifiedHead) Head() Head

Head returns a copy of the signed head, for sending it on.

func (VerifiedHead) LogID
func (v VerifiedHead) LogID() LogID

LogID returns the ID of the log the head belongs to.

func (VerifiedHead) LogKey
func (v VerifiedHead) LogKey() ed25519.PublicKey

LogKey returns the public key that signed the head.

func (VerifiedHead) LogValue
func (v VerifiedHead) LogValue() slog.Value

LogValue returns v like Head.LogValue.

func (VerifiedHead) MarshalJSON
func (v VerifiedHead) MarshalJSON() ([]byte, error)

MarshalJSON encodes v like Head.MarshalJSON.

func (VerifiedHead) Root
func (v VerifiedHead) Root() [HashSize]byte

Root returns the root hash of the tree.

func (VerifiedHead) Size
func (v VerifiedHead) Size() uint64

Size returns the number of entries in the tree.

func (VerifiedHead) String
func (v VerifiedHead) String() string

String returns a short summary of v.

func (VerifiedHead) UnmarshalJSON
func (v *VerifiedHead) UnmarshalJSON([]byte) error

UnmarshalJSON always returns ErrNotDecodable.

Conviene saber

Todos los pasos

Comprobar una prueba de inclusión

Una prueba muestra que una entrada está en el log sin enviar las demás. Sigue el RFC 9162, con BLAKE2b-256.

  1. El lector tiene un head firmado por el propietario: tamaño 5 y la raíz. Primero comprueba la firma.
  2. Pide la entrada 2 y recibe el registro firmado y un camino de auditoría de tres hashes: la hoja 3, el nodo 0-1 y la hoja 4.
  3. Comprueba la firma del registro, lo convierte en la hoja 2 y sube por el árbol con cada hash del camino.
  4. Si el resultado es igual a la raíz firmada, la entrada 2 está en el log con ese tamaño. Cualquier otro registro da otra raíz.
Todos los pasos

Borrar una entrada

Borrar quita el contenido, no la entrada: el registro se queda, así que todos los heads y las pruebas siguen siendo válidos.

  1. La entrada 2 se guarda como su registro firmado, su texto cifrado y su clave de entrada.
  2. Delete crea una marca de borrado: el log, el índice y el hash de la hoja, firmados por el propietario.
  3. El texto cifrado y la clave se eliminan; el registro y la marca se quedan. El árbol y el head no cambian.
  4. Una réplica comprueba la marca y elimina su copia de ese texto cifrado y esa clave. Leer la entrada da ErrDeleted.
  5. Una copia hecha antes, o una clave que alguien guardó, queda fuera de alcance. El borrado se hace lo mejor posible.

Un log o una réplica solo mantienen en memoria los hashes de su árbol y una copia compacta de cada registro. Los textos cifrados, las claves y las marcas de borrado viven en su Storage, una interfaz con tres métodos: Entry, Key y Commit. Cada cambio es un único Commit atómico, hecho con el bloqueo del log antes de mostrar el cambio: Append confirma la entrada con el head firmado justo después, Delete y ApplyDelete los registros y las marcas en lugar de los textos cifrados y las claves, AddKey y AddKeys las claves (Change.Keys, una lista de índice y clave en orden ascendente), y Update sus entradas con los nuevos heads. Un commit fallido deja el log o la réplica exactamente como estaban. Un almacenamiento pertenece a un solo log: New y NewReplica devuelven ErrStorageNotEmpty para uno que ya guarda una entrada.

Entry tiene que devolver un error que envuelva ErrOutOfRange para un índice que no guarda ninguna entrada. New y NewReplica leen el índice 0 para comprobar que el almacenamiento está vacío, y devuelven cualquier otro error de esa lectura, como una lectura fallida o un backend cerrado, en vez de empezar un log sobre él. Para una entrada que el log tiene, ErrOutOfRange del almacenamiento significa que el almacenamiento la ha perdido: las lecturas devuelven ErrStorageMismatch, y Restore y RestoreReplica ErrRestoreMismatch, nunca ErrOutOfRange. Un Storage propio tiene que cumplir este contrato y guardar todas las claves de Change.Keys. kolo-store lo hace.

Cada lectura comprueba lo que devuelve el almacenamiento frente al registro en memoria y devuelve ErrStorageMismatch si hay una diferencia. Get y Key de una entrada borrada devuelven ErrDeleted sin leer el almacenamiento. Key y EntryInfo le dan a una capa de sincronización la clave de una entrada, y si está borrada o tiene clave, sin leer su texto cifrado. Restore y RestoreReplica leen cada entrada del almacenamiento una vez, en orden, y verifican cada una: una goroutine lee el almacenamiento, hasta GOMAXPROCS trabajadores (como mucho 16) comprueban las entradas, y los resultados se aplican en el orden de los índices, así que el error es siempre el de la primera entrada mala en el orden del log. Por eso un Storage se lee desde una goroutine distinta de la de quien llama, de una llamada en una, y puede que se le pidan hasta 2 x trabajadores + 1 entradas más allá de la primera mala. Restore devuelve ErrRestoreMismatch para una entrada viva de un log propio guardada sin su clave. Restaurar 200 entradas de 1 MiB tardó 164 ms en lugar de 471 ms, y 20.000 entradas de 100 bytes 140 ms en lugar de 1,38 s, en una máquina de 10 núcleos con carga; el tiempo crece con los bytes guardados divididos entre los trabajadores, más lo que tarde el almacenamiento en leerlos, mientras que Append, Head y las pruebas nunca leen las entradas guardadas.

Delete(indexes...) y ApplyDelete(markers...) reciben muchas entradas o marcas y las confirman todas a la vez, todo o nada. Un índice fuera de rango o una marca incorrecta no borra nada, y las entradas ya borradas se saltan.

Un Log tiene un solo escritor. No hay log con varios escritores: el uso en varios dispositivos y el chat usan un log por dispositivo o persona, y los demás replican cada uno como una Replica de solo lectura. Una Replica solo necesita la clave pública del propietario, el ID de log y la cabecera firmada del log, nunca una clave privada, y solo crece mediante Update, que lo verifica todo antes de guardar nada. Una réplica muy atrasada puede ponerse al día por partes: Update acepta entradas que se quedan antes del head, confirma ese head y su prueba como objetivo pendiente junto con las entradas, y RestoreReplica reproduce el objetivo después de un reinicio. OpenEntry comprueba una entrada obtenida por separado frente a un head verificado y su prueba de inclusión.

Claves de entrada derivadas

Un log tiene uno de dos modos de clave, fijado en su cabecera firmada. Con RandomKeys, el modo por defecto, cada entrada recibe una clave aleatoria nueva que solo tiene quien escribe, y un lector recibe las claves de quien escribe. Con DerivedKeys, de New(id, s, WithDerivedKeys(&secret, epoch)), la clave de cada entrada se deriva de un secreto de grupo, del log y del índice de la entrada. Quien tiene el secreto deriva la clave de cualquier entrada y abre el texto cifrado que recibió de quien escribe, de una réplica o de un pin, sin que quien escribe esté conectado. Ninguna clave tiene que viajar. Un log creado sin WithDerivedKeys no cambia, byte a byte.

Usa DerivedKeys solo cuando todos los que tienen el secreto pueden leer todas las entradas del log. Un log que oculta las claves a algunos lectores se queda con RandomKeys.

  • La derivación es HKDF-SHA256 del EntrySecret, con la clave del log, el ID del log, la época y el índice en su info. La clave derivada pasa por el mismo cifrado y el mismo KeyCommit que una aleatoria.
  • El EntrySecret tiene que ser de 32 bytes aleatorios uniformes, de crypto/rand, y no usarse para nada más. Con un secreto adivinable, cualquiera que tenga un registro puede encontrarlo sin conexión.
  • La época la fija quien llama y está en la cabecera firmada. Un secreto nuevo va con logs nuevos en la época siguiente, y un log nunca mezcla épocas. kolo-log nunca compara épocas entre logs: una aplicación que pasa a una época nueva tiene que rechazar una cabecera cuya época sea inferior a la que usa.
  • Restore devuelve ErrKeyModeMismatch cuando la cabecera y las opciones no coinciden en el modo o en la época, y ErrSecretMismatch cuando el secreto no deriva la clave de la entrada más reciente. WithDerivedKeys(nil, epoch) da ErrMissingSecret.
  • Un lector crea un KeyDeriver con NewKeyDeriver(secret, owner, header) (ErrNotDerived para una cabecera con claves aleatorias). Close lo borra.
  • Replica.AddKeys(start, keys) comprueba muchas claves a la vez, como máximo MaxAddKeys (32.768) por llamada, y guarda en un solo commit cada clave que pasa. Las claves que un registro firmado demuestra erróneas vuelven juntas en un *BadKeyError, que envuelve ErrBadEntry.

Lo que el modo significa para los lectores, en palabras de la guía de kolo-log:

  • Quién lee. Todos los que alguna vez tuvieron el secreto leen cada entrada de la época cuyo texto cifrado puedan conseguir, de quien escribe, de una réplica o de cualquier pin, sin que quien escribe esté conectado, mientras se conserven copias.
  • Lector eliminado. kolo-log no tiene membresía. Un lector que tuvo el secreto de una época sigue leyendo todas las entradas de esa época. Dejarlo fuera de las entradas posteriores exige un secreto nuevo, hecho con bytes aleatorios nuevos y nunca derivado del anterior, entregado solo a los lectores que quedan, y logs nuevos en la época siguiente; conserva todas las épocas que tuvo y nada después.
  • Lector nuevo. Quien recibe el secreto lee toda la historia de la época, incluidas las entradas escritas antes de que recibiera el secreto, también de escritores que nunca llega a conocer.
  • Borrar no revoca la lectura. Borrar una entrada no puede quitársela a nadie que tenga el secreto y haya guardado una copia de su texto cifrado: una clave que se puede volver a derivar no se puede destruir.
  • Sin secreto hacia adelante dentro de una época. Quien consigue el secreto de una época, en cualquier momento, lee todas las entradas de esa época que sigan en un pin o en una copia, pasadas y futuras. Las claves aleatorias permiten a quien escribe destruir una clave; las derivadas renuncian a eso.

Un pin, o cualquier otro que tenga el texto cifrado sin el secreto, conoce por la cabecera el modo y la época de cada log, y no lee nada. Kolo Chat usa este modo, así que un mensaje se lee desde un pin mientras quien lo envió está sin conexión.

Límites

  • Tamaño de entrada: unos 254 GiB en plataformas de 64 bits y unos 512 MiB en las de 32 bits. Por encima, Append devuelve ErrEntryTooLarge.
  • Relleno: cada entrada de hasta 254 bytes ocupa un frame de 256 bytes antes de cifrarse, y 296 bytes de texto cifrado en disco (un nonce de 24 bytes y una etiqueta de 16 más). Las más grandes se redondean hacia arriba con relleno Padmé, como mucho un 6 % más.
  • Memoria: unos 64 bytes de hashes del árbol y 144 bytes de registro por entrada. Crece con el número de entradas, nunca con su tamaño.
  • ID de log: 32 bytes, aleatorio. Hashes del árbol: 32 bytes (BLAKE2b-256).
  • Rutas de prueba: como máximo 65 hashes.
  • Entrada JSON: como máximo 4 KiB, 8 KiB para pruebas. Una entrada más grande, campos desconocidos o duplicados, hex en mayúsculas, secuencias de escape y datos sobrantes devuelven un error que envuelve ErrMalformed.
  • Un log escribible por (clave de autor, ID de log) en cada proceso. New y Restore devuelven ErrLogOpen para un segundo, o mientras el par esté retenido por Reserve o ReserveForRestore, o el ID de log por ReserveID.

Seguridad

  • Los heads, los registros, las marcas de borrado y la cabecera del log se firman con Ed25519ctx, cada uno con su propio contexto: kolo-log/tree-head/v0, kolo-log/record/v1, kolo-log/delete/v0 y kolo-log/header/v0, o kolo-log/derived-header/v0 para la cabecera de un log con claves derivadas.
  • El árbol usa el hash del RFC 9162 con BLAKE2b-256. Las entradas se cifran con XChaCha20-Poly1305.
  • XChaCha20-Poly1305 no ofrece compromiso de clave (key commitment), así que cada registro lleva KeyCommit, el hash BLAKE2b-256 de kolo-log/key/v0 y la clave de la entrada. Restore, RestoreReplica, Update, AddKey y OpenEntry comprueban una clave contra él antes de descifrar, así que un registro firmado solo se abre en un único texto en claro.
  • El hash del texto cifrado se comprueba en cada lectura, porque la clave se guarda junto al texto cifrado: otros datos sellados con esa clave también se abrirían.
  • De cada entrada son públicos la clave del autor, el ID de log, el índice, el tamaño con relleno, el hash del texto cifrado y el compromiso de clave. El contenido, su tamaño real y el momento en que se escribió no lo son, tampoco después del borrado.
  • El log borra las claves que usa en cuanto termina, pero la copia interna de la clave en el cifrador se queda en memoria hasta que el runtime la reutiliza. Con la codificación JSON, encoding/json mantiene sus propios búferes, que kolo-log no puede limpiar.
  • Cada función Decode y cada verificador reciben bytes no fiables y devuelven un error o false, nunca un pánico.
  • JSON es una vista para APIs, depuración y configuración. Nunca se firma, y un Head leído de JSON solo es de confianza después de VerifyHead.
  • Pasa a Restore WithTrustedHead con el head más reciente que hayas visto, para que una instantánea más antigua o bifurcada devuelva ErrRollback y un log restaurado nunca vuelva a firmar un índice que ya se publicó.
  • Replica.Update devuelve ErrDiverged cuando el propietario ha firmado dos historiales, y ErrRollback para un head más antiguo del mismo historial. Ante cualquier error, la réplica se queda exactamente como estaba.

Límites conocidos

  • Experimental. La API puede cambiar antes de v1.0.0.
  • NewMemory guarda las entradas solo en memoria y no guarda heads, así que su log no se puede restaurar después de un reinicio. Para eso, usa un Storage persistente como kolo-store.
  • El almacenamiento es de confianza para las claves. Un valor de entrada antiguo que se vuelva a poner en él se rechaza mientras el log está abierto, pero después de un Restore se lee otra vez como la entrada viva.
  • Los heads no llevan marca de tiempo, así que una réplica los ordena solo por tamaño y raíz.
  • Mientras hay un objetivo de puesta al día pendiente, Update rechaza sin comprobarlo, con ErrRollback, un head entre Len y el tamaño del objetivo. Ahí no significa que el head comparta el historial de la réplica.
  • VerifyConsistency también acepta un tamaño antiguo de 0 y tamaños iguales, que el RFC 9162 deja sin definir.
  • Una entrada recibida sin su clave se lee como ErrKeyUnavailable hasta que AddKey o AddKeys le da la clave.

Errores

ErrorMensaje
ErrBadEntrykololog: entry does not match its signed record or the head
ErrBadHeadSignature
ErrBadLogHeaderkololog: bad log header
ErrBadProofkololog: consistency proof does not verify
ErrDeletedkololog: entry deleted
ErrDeriverClosedkololog: key deriver closed
ErrDivergedkololog: head diverges from the held entries or the pending target
ErrEncodingMismatchkololog: value encoding mismatch
ErrEntryTooLargekololog: entry too large
ErrInvalidHeadkololog: invalid head
ErrInvalidUTF8kololog: invalid UTF-8
ErrKeyModeMismatchkololog: key mode mismatch
ErrKeyUnavailablekololog: entry key not available
ErrLogClosedkololog: log closed
ErrLogOpenkololog: log already open in this process
ErrMalformedkololog: malformed encoding
ErrMalformedEntrykololog: malformed entry
ErrMissingHeaderkololog: missing log header
ErrMissingSecretkololog: missing entry secret
ErrNoHeadkololog: replica has no verified head
ErrNoStoragekololog: nil storage
ErrNotDecodablekololog: a VerifiedHead cannot be decoded; decode a Head and call VerifyHead
ErrNotDerivedkololog: log does not use derived keys
ErrOutOfRangekololog: index out of range
ErrRestoreMismatchkololog: restored data does not match the signed head or records
ErrRollbackkololog: snapshot or head is older than a known head or forks from it
ErrSecretMismatchkololog: entry secret mismatch
ErrStorageMismatchkololog: stored entry does not match its verified record
ErrStorageNotEmptykololog: storage already holds entries
ErrTooManyKeyskololog: too many keys in one call
ErrUnknownEncodingkololog: unknown value encoding
ErrWrongLogID
ErrWrongValueTypekololog: wrong value type

Comprueba los errores con errors.Is, porque la mayoría llegan envueltos con el índice o un motivo. ErrBadHeadSignature y ErrWrongLogID coinciden los dos con ErrInvalidHead, y todo lo que no se puede analizar coincide con ErrMalformed. Un texto cifrado guardado que no se abre devuelve ErrMalformedEntry. New y Restore envuelven koloid.ErrInvalidIdentity para una identidad nula o a cero, y New, Restore, NewReplica y RestoreReplica devuelven ErrNoStorage para un almacenamiento nulo. Un error del almacenamiento llega envuelto, así que sus propios errores siguen accesibles con errors.Is.

Referencias