In this guide two people write one document together, each in a terminal window on your computer. Each line someone types is signed by its writer, and both terminals show the same lines in the same order. Then you see what "settled" and "tentative" mean, restart a terminal, and remove a writer.
The program uses kolo-braid, which merges the
signed logs of several members into one order that every peer computes the
same way. Everything stays on your computer, on the address 127.0.0.1. You
start from the
kolo-braid-starter
repository, which also holds the finished code.
1Check what you need
You need Go 1.26 or newer, git and two terminal windows. Build a chat in your terminal shows how to install Go. Here Terminal 1 is Ana, who starts the document, and Terminal 2 is Bruno, who joins it.
2Get the starter
In Terminal 1, clone the starter and go into it:
git clone https://github.com/ciranda-dev/kolo-braid-starter.git && cd kolo-braid-startermain.go is done: it reads the flags, keeps your key in the folder you give
with -dir, and passes each line you type to doc.go. doc.go only declares
what main.go calls. (The finished version is in finished/, and its main.go
is the same file as this one.) Make a temporary folder for Ana:
ANA="$(mktemp -d)"$ana = (New-Item -ItemType Directory -Path (Join-Path $env:TEMP "kolo-ana-$(Get-Random)")).FullNameANA="$(mktemp -d)"Run the starter as it is:
go run . -dir "$ANA" -listen 127.0.0.1:7450go run . -dir $ana -listen 127.0.0.1:7450go run . -dir "$ANA" -listen 127.0.0.1:7450It prints Ana's key. Type show and press Enter:
07:05:06 me 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87
07:05:06 type: say <text>, show, add <key>, remove <key>
07:05:07 not written yet: follow the guidePress Ctrl-C to stop it. The next step fills in doc.go.
3Write doc.go
Replace the content of doc.go with the five pieces below, one after the
other.
The first piece is the rule that turns the order into the document. kolo-braid
calls apply for each entry, in the same order on every peer. Each line is
kept under its position, with the first 4 bytes of its writer's key. A line
that is empty or over 500 bytes is refused with Reject, the same way on every
peer. apply must not read the clock, draw random numbers or do I/O, so every
peer builds the same document. say refuses such a line on the typing side too,
with the error a line is 1 to 500 bytes, so a line that no peer would show is
never written.
package main
import (
"bytes"
"context"
"crypto/ed25519"
"encoding/binary"
"encoding/hex"
"errors"
"fmt"
"io"
"log"
"os"
"path/filepath"
"slices"
"sync"
"time"
kolobraid "github.com/ciranda-dev/kolo-braid"
koloid "github.com/ciranda-dev/kolo-id"
kolosecure "github.com/ciranda-dev/kolo-secure"
kolostore "github.com/ciranda-dev/kolo-store"
)
const (
app = "doc/v1"
maxLine = 500
)
var errKey = errors.New("a key is 64 hex digits")
// apply turns each data entry into one line of the document, kept under its position in the braid's order with the
// first 4 bytes of its writer's key.
func apply(tx *kolobraid.Tx, e *kolobraid.Entry) error {
if e.Kind != kolobraid.KindData {
return nil
}
if len(e.Body) == 0 || len(e.Body) > maxLine {
return kolobraid.Reject("a line is 1 to 500 bytes")
}
key := binary.BigEndian.AppendUint64([]byte("line/"), e.Position)
return tx.Put(key, append(e.Writer[:4:4], e.Body...))
}The second piece opens the document stored in the folder, or starts a new one
(-listen), or joins one (-peer). The folder holds a
kolo-store Pebble database, so the document is
still there after a restart. It prints listening on IP:PORT when it listens.
Close waits for the goroutines (the accept loop, the connection handlers and
follow) before it closes the braid and the store.
// Doc is one person's copy of the shared document.
type Doc struct {
braid *kolobraid.Braid
db kolostore.Backend
l *kolosecure.Listener
stop context.CancelFunc
wg sync.WaitGroup
allowed sync.Map // keys the owner typed add for, each let in once
}
// openDoc opens the document stored in dir, or starts one (listen) or joins one (peer), and keeps it in sync.
func openDoc(ctx context.Context, id *koloid.Identity, dir, listen, peer string, owner ed25519.PublicKey) (*Doc, error) {
db, err := kolostore.OpenPebble(filepath.Join(dir, "doc"))
if err != nil {
return nil, err
}
opts := kolobraid.Options{Backend: db, App: app, Apply: apply}
br, err := kolobraid.Open(ctx, id, opts)
switch {
case errors.Is(err, kolobraid.ErrNotFound) && listen != "":
log.Print("started a new document")
br, err = kolobraid.Create(ctx, id, kolobraid.GroupConfig{App: app}, opts)
case errors.Is(err, kolobraid.ErrNotFound):
br, err = join(ctx, id, opts, peer, owner)
}
if err != nil {
db.Close()
return nil, err
}
ctx, stop := context.WithCancel(ctx)
d := &Doc{braid: br, db: db, stop: stop}
if listen == "" {
d.wg.Go(func() { d.follow(ctx, id, peer, owner) })
return d, nil
}
if d.l, err = kolosecure.Listen(ctx, listen, id, nil); err != nil {
d.Close()
return nil, err
}
log.Print("listening on ", listen)
d.wg.Go(func() { d.accept(ctx) })
return d, nil
}
// Close stops the document.
func (d *Doc) Close() {
d.stop()
if d.l != nil {
d.l.Close()
}
d.wg.Wait()
d.braid.Close()
d.db.Close()
}The third piece runs the commands. say appends one entry to your own strand,
your signed kolo-log. show reads two snapshots:
View (settled and tentative lines) and Settled (settled lines only), and
marks each line with the one it is in. add and remove take a key of 64 hex
digits and refuse anything else with a key is 64 hex digits.
// do runs one command typed in the terminal.
func (d *Doc) do(ctx context.Context, cmd, arg string) error {
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
key, _ := hex.DecodeString(arg)
switch cmd {
case "say":
if len(arg) == 0 || len(arg) > maxLine {
return errors.New("a line is 1 to 500 bytes")
}
_, err := d.braid.Append(ctx, []byte(arg))
return err
case "show":
d.show(os.Stdout)
return nil
case "add":
if len(key) != ed25519.PublicKeySize {
return errKey
}
d.allowed.Store(string(key), true)
log.Printf("%.8s can join now", arg)
return nil
case "remove":
if len(key) != ed25519.PublicKeySize {
return errKey
}
_, err := d.braid.RemoveMember(ctx, key)
return err
}
return errors.New("commands: say <text>, show, add <key>, remove <key>")
}
// show prints every line in the braid's order with its writer, and whether its place is settled or still tentative.
func (d *Doc) show(w io.Writer) {
view, settled := d.braid.View(), d.braid.Settled()
defer view.Close()
defer settled.Close()
n := 0
for k, v := range view.Scan([]byte("line/"), []byte("line0")) {
state := "tentative"
if s, err := settled.Get(k); err == nil && bytes.Equal(s, v) {
state = "settled"
}
n++
fmt.Fprintf(w, "%d. %x: %s (%s)\n", n, v[:4], v[4:], state)
}
if n == 0 {
fmt.Fprintln(w, "the document is empty")
}
}The fourth piece runs on Ana's side. Each connection is a
kolo-secure connection, so Ana knows the key of
the person at the other end. A member is synced with ServeConn. A key Ana
typed add for may send one join request: Admit adds it as a writer and
returns the welcome. The permission is used up only after Admit succeeded, so
a connection that sends no request does not burn it, and Ana does not have to
type add again. Every other connection is closed.
// accept syncs with members, lets in a key the owner typed add for, and closes every other connection.
func (d *Doc) accept(ctx context.Context) {
for {
c, err := d.l.Accept(ctx)
if err != nil {
return
}
d.wg.Go(func() {
defer c.Close()
key := c.RemotePublicKey()
v := d.braid.View()
member := slices.ContainsFunc(v.Members(), func(m kolobraid.Member) bool { return bytes.Equal(m.Key, key) })
v.Close()
if member {
d.braid.ServeConn(ctx, c)
} else if _, ok := d.allowed.Load(string(key)); ok {
if err := d.admit(ctx, c); err != nil {
log.Print("add: ", err)
} else {
d.allowed.Delete(string(key))
}
}
})
}
}
// admit reads a join request on c, adds its sender as a writer and sends the welcome back.
func (d *Doc) admit(ctx context.Context, c *kolosecure.Conn) error {
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
s, err := c.AcceptStream(ctx)
if err != nil {
return err
}
req, err := io.ReadAll(io.LimitReader(s, kolobraid.MaxJoinRequest))
if err != nil {
return err
}
writer := kolobraid.Role(0) // not an admin and not a settler
welcome, err := d.braid.Admit(ctx, c.RemotePublicKey(), req, &kolobraid.AdmitOptions{Roles: &writer})
if err != nil {
return err
}
if _, err := s.Write(welcome); err != nil {
d.braid.RemoveMember(ctx, c.RemotePublicKey())
return err
}
s.Close()
select { // the joiner hangs up once it has the welcome
case <-c.Context().Done():
case <-ctx.Done():
}
log.Printf("added %x", c.RemotePublicKey()[:4])
return nil
}The last piece runs on Bruno's side. NewJoin makes his strand and his join
request. He sends it to Ana once a second until she lets him in, then
Complete opens the document with her welcome. After that he stays connected
to Ana and syncs with ServeConn, and connects again if she restarts.
// join asks the owner to let this key in, once a second until the owner typed add for it, and opens the document.
func join(ctx context.Context, id *koloid.Identity, opts kolobraid.Options, peer string, owner ed25519.PublicKey) (*kolobraid.Braid, error) {
j, err := kolobraid.NewJoin(ctx, id, opts)
if err != nil {
return nil, err
}
req, err := j.Request(nil)
if err != nil {
return nil, err
}
log.Print("waiting for the owner to type add and my key")
for {
if welcome, err := ask(ctx, id, peer, owner, req); err == nil {
log.Print("joined the document")
return j.Complete(ctx, welcome)
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(time.Second):
}
}
}
// ask sends the join request to the owner and returns the welcome.
func ask(ctx context.Context, id *koloid.Identity, peer string, owner ed25519.PublicKey, req []byte) ([]byte, error) {
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
c, err := kolosecure.Dial(ctx, peer, id, owner, nil)
if err != nil {
return nil, err
}
defer c.Close()
s, err := c.OpenStream(ctx)
if err != nil {
return nil, err
}
s.SetDeadline(time.Now().Add(10 * time.Second))
if _, err := s.Write(req); err != nil {
return nil, err
}
s.Close()
welcome, err := io.ReadAll(io.LimitReader(s, kolobraid.MaxWelcome))
if err == nil && len(welcome) == 0 {
err = errors.New("not added yet")
}
return welcome, err
}
// follow keeps a connection to the owner and syncs the document over it, again after the owner restarts.
func (d *Doc) follow(ctx context.Context, id *koloid.Identity, peer string, owner ed25519.PublicKey) {
for ctx.Err() == nil {
if c, err := kolosecure.Dial(ctx, peer, id, owner, nil); err == nil {
d.braid.ServeConn(ctx, c)
c.Close()
}
select {
case <-ctx.Done():
case <-time.After(time.Second):
}
}
}Compare your file with the finished one. This prints nothing when they are the same:
git diff --no-index doc.go finished/doc.go4Start the document in Terminal 1
In Terminal 1, run the program again with Ana's folder:
go run . -dir "$ANA" -listen 127.0.0.1:7450go run . -dir $ana -listen 127.0.0.1:7450go run . -dir "$ANA" -listen 127.0.0.1:7450Ana keeps her key and starts a new document. Your times and keys will differ:
07:05:08 me 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87
07:05:08 started a new document
07:05:08 type: say <text>, show, add <key>, remove <key>5Join it from Terminal 2
In Terminal 2, go to the same kolo-braid-starter folder (use the path
where you cloned it):
cd kolo-braid-starterMake a temporary folder for Bruno:
BRUNO="$(mktemp -d)"$bruno = (New-Item -ItemType Directory -Path (Join-Path $env:TEMP "kolo-bruno-$(Get-Random)")).FullNameBRUNO="$(mktemp -d)"Then join with Ana's key, the long one on her me line (paste your own):
go run . -dir "$BRUNO" -peer 127.0.0.1:7450 -key 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87go run . -dir $bruno -peer 127.0.0.1:7450 -key 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87go run . -dir "$BRUNO" -peer 127.0.0.1:7450 -key 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87Bruno waits for Ana:
07:05:10 me 3fa349c6c47a2177b99ebfb71af5ab9db93e411e8448e7a0ce725225469e9abf
07:05:10 waiting for the owner to type add and my keyIn Terminal 1, type add and Bruno's key (paste your own):
add 3fa349c6c47a2177b99ebfb71af5ab9db93e411e8448e7a0ce725225469e9abfAna lets him in:
07:05:13 3fa349c6 can join now
07:05:13 added 3fa349c6In Terminal 2, Bruno is in:
07:05:13 joined the document
07:05:13 type: say <text>, show, add <key>, remove <key>6Write together
In Terminal 1, type:
say Kolo meeting notesIn Terminal 2, type:
say We meet on Friday at 10Type show in both terminals. Both print the same document, in the same
order, with the short key of each writer:
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)7See a tentative line
A line is tentative while its place in the order can still change, for example when someone else wrote a line at the same time. It is settled once a settler has fixed its place, and then it never moves again. Here Ana is the only settler, so a line settles only while Ana is online.
Stop Ana with Ctrl-C in Terminal 1. In Terminal 2, type
say Ana brings the slides, then show:
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)
3. 3fa349c6: Ana brings the slides (tentative)Bruno's new line is in his document, but its place is not fixed yet. In Terminal 1, start Ana again with the same folder:
go run . -dir "$ANA" -listen 127.0.0.1:7450go run . -dir $ana -listen 127.0.0.1:7450go run . -dir "$ANA" -listen 127.0.0.1:7450Her key and her document are still there, so she does not start a new one:
07:05:23 me 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87
07:05:23 type: say <text>, show, add <key>, remove <key>Bruno connects again within a few seconds and sends her his new line, which
she settles. Type show in both terminals:
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)
3. 3fa349c6: Ana brings the slides (settled)8Remove a writer
In Terminal 1, remove Bruno with his key (paste your own):
remove 3fa349c6c47a2177b99ebfb71af5ab9db93e411e8448e7a0ce725225469e9abfThen, in Terminal 1, type say Bruno left the project, and in
Terminal 2, type say Bruno writes after the removal. Wait a few seconds
and type show in both terminals. Ana's document has her new line, not
Bruno's:
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)
3. 3fa349c6: Ana brings the slides (settled)
4. 16dbfe82: Bruno left the project (settled)Bruno's document does not get Ana's new line, and his own line stays tentative:
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)
3. 3fa349c6: Ana brings the slides (settled)
4. 3fa349c6: Bruno writes after the removal (tentative)Ana stopped sending Bruno anything, and every entry Bruno writes after his removal is void everywhere: no other member shows it. On his side it stays tentative, because nothing settles for him anymore. Bruno keeps the lines he received before.
9Stop everything
Press Ctrl-C in Terminal 2, then in Terminal 1.
What just happened
- Each person wrote to a strand of their own: a kolo-log in which every entry is signed by its writer, stored with kolo-store.
- kolo-braid merged the strands into one order. Every peer that holds the same
entries computes the same order, byte for byte, and calls
applyin it, so both documents were the same. - Ana created the document, so she was its admin and its only settler. Bruno was added as a writer: he can write, not add or remove anyone.
- Membership is ordered like the lines. Bruno's lines before his removal kept their place; the one after it was void.
- The two terminals synced over one kolo-secure
connection on
127.0.0.1, withServeConn.
Between two computers
The two terminals connect directly: -listen opens the port and -peer dials
it. They do not use the DHT or the swarm, so there is no bootstrap to configure.
-listenand-peermust be an IPv4IP:PORTthat is not 0.0.0.0 and has a port other than 0. Anything else exits with"..." is not an IPv4 IP:PORT such as 127.0.0.1:7450.- On one computer, use
127.0.0.1, as in the steps above. - On the same local network, Terminal 1 passes
-listenwith the IPv4 address of its network card, such as-listen 192.168.1.20:7450, and Terminal 2 passes-peer 192.168.1.20:7450with the key. - Between two different networks it works only when the first computer has a public IPv4 address that the second can reach, with the UDP port open in the firewall.
- Two computers that are both behind home routers will not connect: this program does not punch through NAT and uses no relay.
For that case use the Kolo examples (chat, share, meet, board) or the kolo-swarm library, which find peers through the public entry nodes by default.
Before you build on it
- kolo-braid is beta and experimental, like every Kolo library at v0.x: the API may change before v1.0.0.
- It is tested with unit, property and fuzz tests, bounded model checks of
settlement and membership, which cover only the sizes they enumerate and are
not a proof for any size, and scenarios with several processes on one Mac,
on
127.0.0.1only. It has not been tested over a real network or through NATs. - With one settler, the document depends on that device: while Ana is
offline, nothing settles.
Admitwith no options admits a plain member, as this program does for Bruno;AdmitOptions.GrowSettlersmakes the first joiners admins and settlers, up to 4 seats, and is for people you trust that far. - A line can never be erased: every peer keeps every entry, history is never pruned, and each entry costs about 600 bytes of disk on every peer. Keep data that may need erasure outside the braid.
- The order among lines written at the same time is not fair, and a settler can try to choose it, so do not build a "first one wins" rule on it.
- Settlement keeps a small round mark per settler for each round, until the settler config changes. For 4 settlers that settle all day at the default pace, that is up to about 22 MB of memory and 55 MB of disk a day. They are what lets a pulse that arrives late get its exact round, and a settler config change removes them.
- The join here is the smallest one that works: Ana lets in a key she typed, on a direct connection. A real app carries the join request and the welcome with kolo-invite, as in Invite someone to a group, in a terminal program and on the kolo-braid page.
ServeConnworks over direct and relayed connections. To reach someone behind a NAT, see Run your own infrastructure.
What was not tested
Every run behind this guide was on one Mac, on 127.0.0.1. The Windows and
Linux commands were not run.
Next steps
- Read the kolo-braid page: the apply function,
settlement, adding a writer with kolo-invite and the
kvhelper. - Find peers by topic instead of by address: Build a chat in your terminal.
- Read Logs and Peers.