One pinning peer can go offline or lose its disk, so an application
usually keeps each log on two or more of them, run by different people or
in different places. kolo-pin picks them with
Select and adds the log to them with AddMany.
Give each peer a group
The application keeps a list of pinning peers, each a pin.Peer with its
key, its IPv4 address and an optional Group, such as its provider or
region. Peers in one group can fail together, so Select takes chosen
peers from different groups when it can.
How Select picks
How a pin is chosen
Rendezvous hashing: every device computes the same choice for a log, with no coordination. Weights here are shortened.
- For each pin, the client computes a weight: SHA-256 of the pin's ID and the log ID.
- The pins are sorted by weight, heaviest first.
- Select takes the heaviest pin of each group until it has enough copies, 2 by default. Pin 1 is skipped: its group already has pin 3. With too few groups, the next heaviest fill in.
- Add pin 5, in a new group. For this log it ranks second, so it takes the place of pin 4. A log where it does not rank stays where it is.
Select(peers, logID, n) uses rendezvous hashing (Thaler and Ravishankar,
1998, the original paper on it). Every pair of a peer and a log gets a
weight, the SHA-256 hash of the peer's ID followed by the log ID. The peers
with the highest weights for that log win.
- Every device that has the same list picks the same peers for a log, with no coordination. The order of the list does not matter.
- When a peer joins or leaves, only the logs for which that peer is among the chosen ones move. Every other log stays where it is.
- Each log gets its own order, so the logs spread evenly over the peers.
Going down that order, Select takes the first peer of each group until it
has n. If there are fewer groups than n, the next heaviest peers fill in
and SharedGroups is true. A peer without a group counts as its own group,
peers with an invalid ID are skipped, and of repeated IDs one is kept. n of
0 or less means DefaultCopies, 2.
Add the log with AddMany
A capability names one pinning peer, so IssueMany signs one for each
peer, all with the same issue time.
AddMany(ctx, id, peers, log, capabilities, n) adds the log to the n peers
Select chooses, using the capability that names each one.
When a chosen peer is unreachable, AddMany tries the next peer in the
same order that was not chosen, at most MaxFallback (2) times. That next
peer can be in a group already used. A peer that answers with an error is
Refused and is not replaced. The Report lists every try with its
Outcome (Accepted, Refused or Unreachable) and counts how many
accepted. AddMany returns ErrNoPeer only if none did, and there is no
retry in the background: call it again later.
This program runs three pinning peers on loopback in two groups, with their data in a temporary directory, and pins one log to two of them:
package main
import (
"context"
"fmt"
"log"
"os"
"path/filepath"
"sync"
"time"
koloid "github.com/ciranda-dev/kolo-id"
"github.com/ciranda-dev/kolo-pin"
kolosecure "github.com/ciranda-dev/kolo-secure"
kolostore "github.com/ciranda-dev/kolo-store"
)
func main() {
dir, err := os.MkdirTemp("", "kolo-pin-choose")
if err != nil {
log.Fatal(err)
}
defer os.RemoveAll(dir)
var wg sync.WaitGroup
defer wg.Wait()
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// Three pinning peers in two groups; an application reads this list from its configuration.
var peers []pin.Peer
for i, group := range []string{"provider-a", "provider-a", "provider-b"} {
d := filepath.Join(dir, fmt.Sprint("pin", i))
id, err := pin.LoadIdentity(d)
if err != nil {
log.Fatal(err)
}
ln, err := kolosecure.Listen(ctx, "127.0.0.1:0", id, nil)
if err != nil {
log.Fatal(err)
}
defer ln.Close()
wg.Go(func() { pin.Serve(ctx, ln, pin.Options{Dir: d}) })
peers = append(peers, pin.Peer{ID: id.PublicKey(), Addr: ln.Addr().String(), Group: group})
}
alice, err := koloid.Generate()
if err != nil {
log.Fatal(err)
}
st := kolostore.New(kolostore.NewMemory())
defer st.Close()
l, err := st.Create(alice)
if err != nil {
log.Fatal(err)
}
if _, err := l.Append([]byte("chat history")); err != nil {
log.Fatal(err)
}
sel := pin.Select(peers, l.ID(), 2)
fmt.Printf("chosen: %s and %s, shared group: %v\n", sel.Peers[0].Group, sel.Peers[1].Group, sel.SharedGroups)
// One capability per peer, so a fallback peer has one too.
caps, err := pin.IssueMany(alice, peers, pin.Capability{Expires: time.Now().Add(30 * 24 * time.Hour)})
if err != nil {
log.Fatal(err)
}
report, err := pin.AddMany(ctx, alice, peers, l, caps, 2)
if err != nil {
log.Fatal(err)
}
fmt.Printf("accepted by %d peers\n", report.Accepted)
for _, r := range report.Results {
fmt.Printf(" %s: %v, %d entries\n", r.Peer.Group, r.Outcome, r.Info.Entries)
}
}It prints something like this. The two groups can come in the other order, since the keys are new each run:
chosen: provider-a and provider-b, shared group: false
accepted by 2 peers
provider-a: accepted, 1 entries
provider-b: accepted, 1 entriesThe program signs a capability for every peer in the list, not only the
chosen ones. A fallback peer without a capability that names it is
Refused.
Next steps
- Run a pinning peer of your own: Run a pinning peer on a VPS.
- Read the kolo-pin page for every limit and error.