A pinning peer keeps other people's kolo-log logs available while their owners are offline: chat history, or what one of your devices wrote, for your other devices. It is a kolo-sync replica without entry keys, and it verifies what it keeps as a Certificate Transparency monitor (RFC 9162) would. It checks and stores signed records, ciphertexts and delete markers, and serves them to any kolo-sync client, by pull or with Fetch, but it never has the keys to read them. An owner allows it with a signed capability that expires; there are no accounts. Select spreads data over pinning peers by rendezvous hashing. The Go package is named pin, and the kolo-pin command runs a pinning peer on a small server.
A pin holds ciphertext only
The pin keeps your log online while your devices are off. It sees sizes, counts and timing, not content.
- Your device dials the pin with kolo-secure and sends ADD with the log ID and a capability.
- The pin checks the capability, then pulls the log from your device with kolo-sync. Your device offers it with ciphertext access only, so no entry key is sent.
- The pin stores the records, the ciphertext and the delete markers under the owner, and answers with the entries, bytes used, quota and expiry.
- Any peer that knows the owner key and the log ID can pull the log from the pin, as ciphertext.
- Entry keys come only from the owner's devices. Without them the pin and other peers cannot read the entries.
Install
go get github.com/ciranda-dev/kolo-pinBuilds with CGO_ENABLED=0 for a static binary. The command is in cmd/kolo-pin; Run a pinning peer on a VPS goes through it step by step.
Quick start
This program runs a pinning peer on loopback with its data in a temporary directory. Alice pins a log and goes offline, then her phone pulls the log's ciphertext from the pinning peer.
package main
import (
"context"
"fmt"
"log"
"os"
"time"
koloid "github.com/ciranda-dev/kolo-id"
"github.com/ciranda-dev/kolo-pin"
kolosecure "github.com/ciranda-dev/kolo-secure"
kolostore "github.com/ciranda-dev/kolo-store"
kolosync "github.com/ciranda-dev/kolo-sync"
)
func main() {
dir, err := os.MkdirTemp("", "kolo-pin")
if err != nil {
log.Fatal(err)
}
defer os.RemoveAll(dir)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// The pinning peer: its key and store live in dir.
pid, err := pin.LoadIdentity(dir)
if err != nil {
log.Fatal(err)
}
ln, err := kolosecure.Listen(ctx, "127.0.0.1:0", pid, nil)
if err != nil {
log.Fatal(err)
}
defer ln.Close()
sctx, stop := context.WithCancel(ctx)
served := make(chan error, 1)
go func() { served <- pin.Serve(sctx, ln, pin.Options{Dir: dir}) }()
peer := pin.Peer{ID: pid.PublicKey(), Addr: ln.Addr().String()}
// Alice writes a log and lets the pinning peer keep it for 30 days, up to 100 MB.
alice, err := koloid.Generate()
if err != nil {
log.Fatal(err)
}
as := kolostore.New(kolostore.NewMemory())
l, err := as.Create(alice)
if err != nil {
log.Fatal(err)
}
for _, v := range []string{"hello", "from", "alice"} {
if _, err := l.Append([]byte(v)); err != nil {
log.Fatal(err)
}
}
capability, err := pin.Issue(alice, pin.Capability{Peer: peer.ID, Expires: time.Now().Add(30 * 24 * time.Hour), Quota: 100_000_000})
if err != nil {
log.Fatal(err)
}
info, err := pin.Add(ctx, alice, peer, l, capability)
if err != nil {
log.Fatal(err)
}
fmt.Printf("pinned %d entries, %d bytes used\n", info.Entries, info.Used)
logID := l.ID()
as.Close() // alice goes offline
// Her phone pulls the ciphertext from the pinning peer and checks it against her key.
phone, err := koloid.Generate()
if err != nil {
log.Fatal(err)
}
conn, err := kolosecure.Dial(ctx, peer.Addr, phone, peer.ID, nil)
if err != nil {
log.Fatal(err)
}
defer conn.Close()
ps := kolostore.New(kolostore.NewMemory())
defer ps.Close()
header, err := kolosync.FetchHeader(ctx, conn, alice.PublicKey(), logID)
if err != nil {
log.Fatal(err)
}
rep, err := ps.OpenReplica(alice.PublicKey(), logID, header)
if err != nil {
log.Fatal(err)
}
if err := kolosync.Pull(ctx, conn, rep); err != nil {
log.Fatal(err)
}
_, err = rep.Get(0)
fmt.Printf("phone holds %d verified entries; reading one without its key: %v\n", rep.Len(), err)
stop()
if err := <-served; err != context.Canceled {
log.Fatal(err)
}
}It prints:
pinned 3 entries, 2305 bytes used
phone holds 3 verified entries; reading one without its key: kololog: entry key not availableThe phone gets the entry keys later from Alice's own devices, with kolo-sync Keys access. Only keys travel then, since the ciphertext is already there. To keep a log on several pinning peers, see Choosing pinning peers.
API reference
The API reference is generated from the Go source and is in English.
Package pin runs and uses pinning peers: a peer that stores encrypted data for others without being able to read it.
A pinning peer keeps kolo-log logs available while their owners are offline. Serve runs one on a kolo-secure listener. Add makes it pull an owner's log over kolo-sync, as a replica without entry keys, under a capability the owner signed; Status and Remove ask about or delete a log, and Revoke delivers a revoke note. Any kolo-sync client pulls or fetches the hosted logs from it like from any other replica, as ciphertext only. Select and AddMany spread a log over several pinning peers.
Experimental: the API may change before v1.0.0.
Command
usage:
kolo-pin run -data DIR [flags] run a pinning peer
kolo-pin id -data DIR print the pinning peer's public key, creating it if needed
Run "kolo-pin run -h" for the flags and their environment variables.kolo-pin run
Usage of kolo-pin run:
-add-timeout time
the most time one add may spend pulling a log, such as 10m; default none (KOLO_PIN_ADD_TIMEOUT)
-data dir
data dir, private (0700): identity.key and the store (KOLO_PIN_DATA)
-health address
TCP address of the HTTP health endpoint /healthz, IPv4, or "off" (KOLO_PIN_HEALTH) (default "127.0.0.1:7451")
-listen address
UDP address for kolo-secure connections, IPv4; "0.0.0.0:7450" on a VPS (KOLO_PIN_LISTEN) (default "127.0.0.1:7450")
-log-format format
format of the logs on stderr: text or json (KOLO_PIN_LOG_FORMAT) (default "text")
-log-level level
level: debug, info, warn or error (KOLO_PIN_LOG_LEVEL) (default "info")
-max-bytes bytes
the most bytes stored for all owners, such as 500MB or 2TiB (KOLO_PIN_MAX_BYTES) (default "100GB")
-max-conns string
the most connections at once, handshaking or open; default 100, up to 10000 (KOLO_PIN_MAX_CONNS)
-max-conns-per-ip string
the most connections from one source address; default 4, at most -max-conns (KOLO_PIN_MAX_CONNS_PER_IP)
-max-owners string
the most owner records kept, one per owner key ever seen; default 100000 (KOLO_PIN_MAX_OWNERS)
-metrics address
TCP address of the Prometheus endpoint /metrics, IPv4, or "off" (KOLO_PIN_METRICS) (default "127.0.0.1:7452")
-owners keys
comma-separated hex owner keys to accept; empty accepts any owner, and anyone can then fill the disk (KOLO_PIN_OWNERS)kolo-pin id
Usage of kolo-pin id:
-data dir
data dir, private (0700): identity.key and the store (KOLO_PIN_DATA)Constants
const (
// CapabilityContext is the Ed25519ctx context of a capability signature.
CapabilityContext = "kolo-pin/capability/v0"
// RevokeContext is the Ed25519ctx context of a revoke note signature.
RevokeContext = "kolo-pin/revoke/v0"
// MaxLogs is the most log IDs one capability lists.
MaxLogs = 256
// MaxCapabilitySize is the size of a capability that lists MaxLogs logs.
MaxCapabilitySize = 1 + 2*32 + 3*10 + 2 + MaxLogs*32 + ed25519.SignatureSize
)const (
// DefaultCopies is the number of pinning peers Select and AddMany choose when n is 0 or less.
DefaultCopies = 2
// MaxFallback is how many more peers AddMany tries, in Select's order, when chosen ones are unreachable.
MaxFallback = 2
)const (
// MaxRequestSize is the largest request a pinning peer reads: an ADD with a capability of MaxLogs logs.
MaxRequestSize = 1 + 1 + kololog.LogIDSize + 2 + MaxCapabilitySize
// Timeout bounds reading one request and writing one answer.
Timeout = 60 * time.Second
)const DefaultMaxBytes = 100_000_000_000DefaultMaxBytes is the default Options.MaxBytes: 100 GB.
const DefaultMaxOwners = 100_000DefaultMaxOwners is the default Options.MaxOwners: a revoke-only record is about 50 bytes and a capability at most 8353, so 100,000 records take at most 0.8 GB.
Variables
var (
// ErrInvalidArgument means a nil identity, log or listener, a key of the wrong size (also in a log's header), or a missing option.
ErrInvalidArgument = errors.New("pin: invalid argument")
// ErrInvalidCapability means a capability or revoke note that is malformed or whose signature does not verify.
ErrInvalidCapability = errors.New("pin: invalid capability")
// ErrUnauthorized means the pinning peer refused the capability: another peer or owner, a log it does not
// cover, an owner the peer does not accept, a bad signature, or an issue time in the future.
ErrUnauthorized = errors.New("pin: not authorized")
// ErrExpired means the capability has expired.
ErrExpired = errors.New("pin: capability expired")
// ErrRevoked means the capability was revoked, or replaced by a newer one from the same owner.
ErrRevoked = errors.New("pin: capability revoked or replaced")
// ErrQuotaExceeded means the write would take the owner past the quota in its capability.
ErrQuotaExceeded = errors.New("pin: owner quota exceeded")
// ErrDiskFull means the write would take the pinning peer past its total byte limit.
ErrDiskFull = errors.New("pin: pinning peer disk full")
// ErrOwnerLimit means the pinning peer already keeps the most owner records it accepts and the owner is not one of them.
ErrOwnerLimit = errors.New("pin: owner limit reached")
// ErrNotFound means the pinning peer does not hold the log for this owner.
ErrNotFound = errors.New("pin: log not found")
// ErrInvalid means the log the pinning peer pulled does not verify; nothing past the last valid entry was kept.
ErrInvalid = errors.New("pin: log data does not verify")
// ErrTooLarge means a message or log entry over kolo-sync's MaxMessageSize, or a request over MaxRequestSize.
ErrTooLarge = errors.New("pin: too large")
// ErrBusy means the pinning peer could not finish now, for example a sync that timed out; try again later.
ErrBusy = errors.New("pin: pinning peer busy")
// ErrProtocol means a malformed, truncated or unexpected message, or another protocol version.
ErrProtocol = errors.New("pin: protocol error")
// ErrUnreachable means the pinning peer could not be reached, or the connection broke before it answered.
ErrUnreachable = errors.New("pin: pinning peer unreachable")
// ErrNoPeer means AddMany found no pinning peer that accepted the log.
ErrNoPeer = errors.New("pin: no pinning peer accepted the log")
)Functions
func Issue
func Issue(owner *koloid.Identity, c Capability) ([]byte, error)Issue signs c with the owner's identity and returns the encoded capability, in whole seconds.
func IssueMany
func IssueMany(owner *koloid.Identity, peers []Peer, c Capability) ([][]byte, error)IssueMany issues c for each peer, all with the same issue time, in the order of peers.
func IssueRevocation
func IssueRevocation(owner *koloid.Identity, peer ed25519.PublicKey, through time.Time) ([]byte, error)IssueRevocation signs a revoke note: peer drops owner's logs if its capability was issued at or before through.
func LoadIdentity
func LoadIdentity(dir string) (*koloid.Identity, error)LoadIdentity returns the pinning peer's identity in dir, creating it (mode 0600) on first use.
func Remove
func Remove(ctx context.Context, id *koloid.Identity, peer Peer, logID kololog.LogID) errorRemove asks the pinning peer to delete the log logID; id must be the log's owner.
func Revoke
func Revoke(ctx context.Context, id *koloid.Identity, peer Peer, note []byte) errorRevoke gives the pinning peer a revoke note from IssueRevocation; anyone may deliver it.
func Serve
func Serve(ctx context.Context, node *kolosecure.Listener, opts Options) errorServe runs a pinning peer on node, which must listen with LoadIdentity(opts.Dir), until ctx is done. It closes the connections it accepted and its store before it returns ctx.Err() or the listener's error.
Types
type Capability
type Capability struct {
Owner ed25519.PublicKey // the log owner who signs it; Issue sets it
Peer ed25519.PublicKey // the one pinning peer it is for
Logs []kololog.LogID // the logs it covers; empty means every log of Owner
Issued time.Time // newer capabilities replace older ones; zero means now in Issue
Expires time.Time // after this the pinning peer drops the logs
Quota uint64 // the most bytes Owner's logs may take on Peer; 0 means only the peer's own limit
}Capability lets one pinning peer keep and serve an owner's logs until it expires; the owner signs it.
func ParseCapability
func ParseCapability(b []byte) (Capability, error)ParseCapability decodes a capability and checks its owner's signature; it never panics on untrusted input. It does not check the time: the pinning peer does, against its own clock.
func (Capability) Covers
func (c Capability) Covers(logID kololog.LogID) boolCovers reports whether c covers the log logID.
type Info
type Info struct {
Entries uint64 // entries of the log it holds
Used uint64 // bytes the owner's logs take on it
Quota uint64 // the owner's quota from the capability in use; 0 means none
Expires time.Time // when that capability expires and the logs are dropped
}Info is what a pinning peer holds of one log and its owner.
func Add
func Add(ctx context.Context, id *koloid.Identity, peer Peer, log kolosync.Source, capability []byte) (Info, error)Add asks the pinning peer to keep log under capability, and serves it the log's ciphertext until it has pulled and verified it. Call it again after new entries or deletes. id only opens the connection.
func Status
func Status(ctx context.Context, id *koloid.Identity, peer Peer, logID kololog.LogID) (Info, error)Status returns what the pinning peer holds of the log logID; id must be the log's owner.
type Options
type Options struct {
// Dir is the data directory, holding identity.key and the store. It must be private; it is created with mode 0700.
Dir string
// MaxBytes is the most bytes the peer stores for all owners together. Default DefaultMaxBytes.
MaxBytes uint64
// Owners lists the owner keys the peer accepts capabilities and revoke notes from. Empty accepts any owner, and
// Serve then logs a warning: anyone can fill MaxBytes and MaxOwners.
Owners []ed25519.PublicKey
// MaxOwners is the most owner records the peer keeps: one per owner key with a capability in use, a held log or a
// revoke note, which stays after expiry so a revoked capability is never accepted again. A request that would
// add one past it is ErrOwnerLimit; owners already kept are not affected. Default DefaultMaxOwners.
MaxOwners uint64
// AddTimeout bounds the time one Add may spend pulling a log, entries already stored are kept. Zero means no
// limit but the connection's idle timeout and the owner's quota.
AddTimeout time.Duration
// Logger receives the peer's events; nil discards them.
Logger *slog.Logger
// Metrics, if set, gets the peer's kolo_pin_* metrics while Serve runs. Their labels never hold keys, IDs or addresses.
Metrics prometheus.Registerer
}Options configures Serve.
type Outcome
type Outcome intOutcome is what happened when AddMany tried one peer.
const (
// Accepted means the peer pulled and verified the log.
Accepted Outcome = iota + 1
// Refused means the peer answered with an error, or no capability named it.
Refused
// Unreachable means the peer could not be reached or the connection broke.
Unreachable
)func (Outcome) String
func (o Outcome) String() stringString returns "accepted", "refused" or "unreachable".
type Peer
type Peer struct {
ID ed25519.PublicKey
Addr string // host:port, IPv4
Group string // where it is hosted, such as a provider or region; used by Select
}Peer is a pinning peer: its kolo-id key, its IPv4 address and an optional group label.
type Report
type Report struct {
Results []Result // every peer tried, in order
Accepted int // how many peers accepted the log
SharedGroups bool // as in Selection
}Report is the result of AddMany.
func AddMany
func AddMany(ctx context.Context, id *koloid.Identity, peers []Peer, log kolosync.Source, capabilities [][]byte, n int) (Report, error)AddMany adds log to the n peers Select chooses, using the capability that names each one. An unreachable peer is replaced by the next peer in Select's order that was not chosen, at most MaxFallback times. It returns ErrNoPeer if none accepted, and ctx.Err() if ctx ended first; the Report always lists the tries made.
type Result
type Result struct {
Peer Peer
Outcome Outcome
Info Info // set when Accepted
Err error // set unless Accepted
}Result is one peer AddMany tried.
type Revocation
type Revocation struct {
Owner ed25519.PublicKey
Peer ed25519.PublicKey
Through time.Time
}Revocation revokes every capability an owner issued for one pinning peer at or before Through.
func ParseRevocation
func ParseRevocation(b []byte) (Revocation, error)ParseRevocation decodes a revoke note and checks its owner's signature; it never panics on untrusted input.
type Selection
type Selection struct {
Peers []Peer // the chosen peers, highest weight first
SharedGroups bool // there were fewer groups than peers to choose, so some chosen peers share a group
}Selection is the result of Select.
func Select
func Select(peers []Peer, logID kololog.LogID, n int) SelectionSelect returns the n peers with the highest rendezvous weight for logID, from different groups when it can. A peer without a group counts as its own group. Peers with an invalid ID are skipped, and of repeated IDs one is kept.
Good to know
How a pin is chosen
Rendezvous hashing: every device computes the same choice for a log, with no coordination. Weights here are shortened.
- For each pin, the client computes a weight: SHA-256 of the pin's ID and the log ID.
- The pins are sorted by weight, heaviest first.
- Select takes the heaviest pin of each group until it has enough copies, 2 by default. Pin 1 is skipped: its group already has pin 3. With too few groups, the next heaviest fill in.
- Add pin 5, in a new group. For this log it ranks second, so it takes the place of pin 4. A log where it does not rank stays where it is.
Capability and quota
A capability is the owner's signed permission for one pin: which logs, how many bytes, until when.
- The owner issues a capability for one pin: the pin's key, issue and expiry times, a byte quota and the logs it covers (none listed means all), signed by the owner.
- A device sends it in ADD. The capability, not the connection, says whose space the log uses.
- The pin checks the signature, that it names this pin and covers the log, that the owner is allowed, that it was issued no more than 60 seconds ahead of the pin's clock, and that it has not expired.
- On every write the pin adds the new bytes to the owner's total. Over the quota is ErrQuotaExceeded, over the pin's own limit ErrDiskFull. Deletes always pass; nothing is evicted.
- Every minute the pin drops the logs of owners whose capability has expired.
- REVOKE, signed by the owner, cancels every capability issued up to a time. The pin drops the owner's logs if the current one is cancelled.
Limits
- IPv4 only.
Peer.Addris an IPv4host:port. Thekolo-pincommand refuses an IPv6-listen,-healthor-metricsaddress, v4-mapped included, and exits with 2. Options.MaxBytes, the total for all owners:DefaultMaxBytes, 100 GB. A write past it isErrDiskFull.Capability.Quota, per owner, in bytes: 0 means only the peer's own limit. A write past it isErrQuotaExceeded.Options.MaxOwners(-max-owners): the owner records the peer keeps, one per owner key that ever had a capability or revoke note accepted,DefaultMaxOwners, 100,000. A note or capability from a new key past it isErrOwnerLimit(answer code 11), so throwaway keys cannot fill the disk or make every start read an unbounded list. Owners already kept are never refused, and one key's record never grows with its notes. The metricskolo_pin_owner_recordsandkolo_pin_owner_records_maxshow it.Options.AddTimeout(-add-timeout): the most time oneAddmay spend, pulling included. Past it theAddisErrBusyand the entries stored so far are kept. The default is none.- Connections come from kolo-transport's limits, set by
-max-conns(100 by default, from 1 to 10,000, handshaking or open) and-max-conns-per-ip(4 by default, at most-max-conns), or byKOLO_PIN_MAX_CONNSandKOLO_PIN_MAX_CONNS_PER_IP. Owners behind one NAT share one address: raise-max-conns-per-ipfor them. - Both limits are checked before anything is stored, and nothing is evicted to make room. Bytes are counted as stored keys and values, not disk blocks. Deletes always pass, so an owner over a lowered quota can still delete.
- If a delete fails (a disk error, or the owner's log open in the same process), the log stays on disk and still counts against the owner's quota,
MaxBytesandUseduntil it is deleted, so anAddcan answerErrQuotaExceededorErrDiskFullfor space the owner cannot see. A log the capability in use still covers stays served, so the nextAddpins it, andRemoveanswersErrBusywhile its delete fails. A log it does not cover is neither offered nor reported: the sweep tries the delete again after a minute, then after twice as long each time, up to 64 minutes, as does the next start. The command logsdelete failed, log keptonce, anddropped after retrywhen a later try works. - One entry must fit in one kolo-sync message,
MaxMessageSize(4 MiB), together with its record and encryption overhead, orAddreturnsErrTooLarge. An entry of exactly 4 MiB is already too large. Split large files into several entries. - A capability lists at most
MaxLogs(256) log IDs, or none for all of the owner's logs, and is at mostMaxCapabilitySize(8353) bytes. Times are whole Unix seconds up to 2^53-1, andExpiresmust be afterIssued. - The pinning peer accepts an issue time at most 60s in its own future, and checks for expired capabilities every minute.
- A request is at most
MaxRequestSize(8389) bytes. Requests on one connection are answered one at a time, each read and answered withinTimeout(60s). - Connecting to a pinning peer gives up after 10s, with
ErrUnreachable. - An open log is closed after a minute without use, and at most 64 are open at once. The least recently used ones are closed first, never one being pulled into.
SelectandAddManychooseDefaultCopies(2) peers when n is 0 or less.AddManyreplaces an unreachable peer at mostMaxFallback(2) times.Options.Dirmust be private. It is created with mode 0700, andidentity.keywith mode 0600. A directory that others can access is refused.- The
kolo-pincommand:-listenis127.0.0.1:7450,-healthis127.0.0.1:7451,-metricsis127.0.0.1:7452and-max-bytesis100GBby default. It exits with 0 after SIGINT or SIGTERM, 1 when it fails to start (such as a port already in use), and 2 on a bad flag, including an invalid-listen,-healthor-metricsaddress.
Security
- The pinning peer sees which owner keys and log IDs it hosts, their entry counts and sizes, the signed records, when each log grows, is deleted from or is pulled, and for every request the requester's key, IP address and timing. It cannot read entries. Anyone who knows an owner key and log ID can pull or fetch its ciphertext and records and learn the same.
- Everything it stores went through kolo-store's
Replica.UpdateorReplica.ApplyDelete, which check every signature, hash and proof first, so it never stores a forged entry.ErrInvalidmeans the log did not verify, and only what verified before is kept. - Capabilities and revoke notes are Ed25519ctx signatures by the owner, with the contexts
kolo-pin/capability/v0andkolo-pin/revoke/v0.ParseCapabilityandParseRevocationnever panic on untrusted input. - Per owner, the pinning peer keeps the newest capability. An older one, or another one with the same issue time, is
ErrRevoked; the same one again is fine. A newer one replaces it at once: logs it no longer covers are deleted, and its quota applies from then on. - When the capability expires, or a revoke note from
IssueRevocationcovers it, every log of that owner is deleted. A revoke note is signed, so anyone may deliver it withRevoke. Addworks with any identity, since the capability carries the authority.StatusandRemoveneed the connection's key to be the log's owner.- Deletion is best effort. The pinning peer applies an owner's new delete markers at the next
Add, as kolo-sync hands them over, and drops their ciphertexts in the same write. A peer that does not follow the protocol keeps what it has, and old disk blocks can still hold deleted bytes. See Deletion. - No error carries bytes a peer sent. The logs of the
kolo-pincommand hold public keys, log IDs, addresses and sizes, never a private key, seed or entry key. - The
kolo-pincommand serves Prometheus metrics on127.0.0.1:7452(-metrics,offto disable), andOptions.Metricsgives a library user the samekolo_pin_*metrics. They hold totals for the whole peer and fixed label values only, never an owner key, log ID, peer key, IP address or per-owner number. The endpoint has no authentication, so keep it on loopback. See Monitoring a pinning peer.
Known limits
AddManydoes not retry in the background. Call it again.- Clients reach a pinning peer directly, at the address it listens on, so it needs an address that all of them reach. Kolo Chat never reaches a pin through a relay.
- There is no request rate limit beyond the owner quota,
MaxBytes, and the connection, stream, size and time limits of kolo-transport and kolo-sync. - An open log keeps its tree hashes and a compact copy of each record in memory, about 250 bytes per entry, and reads ciphertexts from disk. The cap of 64 open logs counts logs, not entries, so that memory is bounded by the disk, not by RAM. In the worst case, 64 open logs that fill
MaxByteswith the smallest entries take about 0.42 timesMaxBytesin memory, about 40 GB for a full 100 GB peer, and any peer that knows their owner keys and log IDs can make the pinning peer open them. SetMaxBytes(-max-bytes) and the owner quotas for the RAM you have. - Opening a log reads and verifies all of it, so reading many idle logs costs the pinning peer CPU and disk reads.
- All tests run on 127.0.0.1. Real NAT traversal, a real VPS, stores of many gigabytes, a full filesystem (reported as
ErrBusy), clock skew beyond 60s and long-running memory use with many owners are not tested. - Experimental. The API may change before v1.0.0.
Errors
| Error | Message |
|---|---|
ErrBusy | pin: pinning peer busy |
ErrDiskFull | pin: pinning peer disk full |
ErrExpired | pin: capability expired |
ErrInvalid | pin: log data does not verify |
ErrInvalidArgument | pin: invalid argument |
ErrInvalidCapability | pin: invalid capability |
ErrNoPeer | pin: no pinning peer accepted the log |
ErrNotFound | pin: log not found |
ErrOwnerLimit | pin: owner limit reached |
ErrProtocol | pin: protocol error |
ErrQuotaExceeded | pin: owner quota exceeded |
ErrRevoked | pin: capability revoked or replaced |
ErrTooLarge | pin: too large |
ErrUnauthorized | pin: not authorized |
ErrUnreachable | pin: pinning peer unreachable |
Test errors with errors.Is. An answer from the pinning peer comes back as the error value itself, such as ErrQuotaExceeded. ErrUnreachable and ErrInvalidArgument from a failed dial wrap the kolo-secure error. When ctx ends first, the client functions return ctx.Err(), and so does AddMany instead of ErrNoPeer; each Result in its Report keeps its own Err.
References
- RFC 9162, Certificate Transparency Version 2.0 (Experimental, December 2021).
- Takes: the log verification that the pinning peer relies on, through kolo-log and kolo-sync.
- RFC 8032, Edwards-Curve Digital Signature Algorithm (EdDSA) (Informational, January 2017).
- Takes: Ed25519ctx, for the capability and revoke note signatures.
- RFC 4918, HTTP Extensions for WebDAV (Proposed Standard, June 2007), status 507, and RFC 9110, HTTP Semantics (Internet Standard, June 2022), status 413.
- Takes: storage limits refused before the write, with distinct answers.
- Not implemented: HTTP itself.
- Thaler and Ravishankar, Using name-based mappings to increase hit rates, IEEE/ACM Transactions on Networking, 1998.
- Takes: rendezvous hashing for
Select, with SHA-256 weights.
- Takes: rendezvous hashing for
- NIST FIPS 180-4 (2015).
- Takes: SHA-256, for the weights of
Select.
- Takes: SHA-256, for the weights of