kolo docsbeta

Build a shared document in two terminals

A step-by-step guide. Two people write one document with kolo-braid, see the same lines in the same order, find out what settled and tentative mean, and remove a writer.

On this page

In this guide two people write one document together, each in a terminal window on your computer. Each line someone types is signed by its writer, and both terminals show the same lines in the same order. Then you see what "settled" and "tentative" mean, restart a terminal, and remove a writer.

The program uses kolo-braid, which merges the signed logs of several members into one order that every peer computes the same way. Everything stays on your computer, on the address 127.0.0.1. You start from the kolo-braid-starter repository, which also holds the finished code.

1Check what you need

You need Go 1.26 or newer, git and two terminal windows. Build a chat in your terminal shows how to install Go. Here Terminal 1 is Ana, who starts the document, and Terminal 2 is Bruno, who joins it.

2Get the starter

In Terminal 1, clone the starter and go into it:

Terminal
git clone https://github.com/ciranda-dev/kolo-braid-starter.git && cd kolo-braid-starter

main.go is done: it reads the flags, keeps your key in the folder you give with -dir, and passes each line you type to doc.go. doc.go only declares what main.go calls. (The finished version is in finished/, and its main.go is the same file as this one.) Make a temporary folder for Ana:

bash
ANA="$(mktemp -d)"

Run the starter as it is:

bash
go run . -dir "$ANA" -listen 127.0.0.1:7450

It prints Ana's key. Type show and press Enter:

Text
07:05:06 me 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87
07:05:06 type: say <text>, show, add <key>, remove <key>
07:05:07 not written yet: follow the guide

Press Ctrl-C to stop it. The next step fills in doc.go.

3Write doc.go

Replace the content of doc.go with the five pieces below, one after the other.

The first piece is the rule that turns the order into the document. kolo-braid calls apply for each entry, in the same order on every peer. Each line is kept under its position, with the first 4 bytes of its writer's key. A line that is empty or over 500 bytes is refused with Reject, the same way on every peer. apply must not read the clock, draw random numbers or do I/O, so every peer builds the same document. say refuses such a line on the typing side too, with the error a line is 1 to 500 bytes, so a line that no peer would show is never written.

doc.go
package main

import (
	"bytes"
	"context"
	"crypto/ed25519"
	"encoding/binary"
	"encoding/hex"
	"errors"
	"fmt"
	"io"
	"log"
	"os"
	"path/filepath"
	"slices"
	"sync"
	"time"

	kolobraid "github.com/ciranda-dev/kolo-braid"
	koloid "github.com/ciranda-dev/kolo-id"
	kolosecure "github.com/ciranda-dev/kolo-secure"
	kolostore "github.com/ciranda-dev/kolo-store"
)

const (
	app     = "doc/v1"
	maxLine = 500
)

var errKey = errors.New("a key is 64 hex digits")

// apply turns each data entry into one line of the document, kept under its position in the braid's order with the
// first 4 bytes of its writer's key.
func apply(tx *kolobraid.Tx, e *kolobraid.Entry) error {
	if e.Kind != kolobraid.KindData {
		return nil
	}
	if len(e.Body) == 0 || len(e.Body) > maxLine {
		return kolobraid.Reject("a line is 1 to 500 bytes")
	}
	key := binary.BigEndian.AppendUint64([]byte("line/"), e.Position)
	return tx.Put(key, append(e.Writer[:4:4], e.Body...))
}

The second piece opens the document stored in the folder, or starts a new one (-listen), or joins one (-peer). The folder holds a kolo-store Pebble database, so the document is still there after a restart. It prints listening on IP:PORT when it listens. Close waits for the goroutines (the accept loop, the connection handlers and follow) before it closes the braid and the store.

doc.go
// Doc is one person's copy of the shared document.
type Doc struct {
	braid   *kolobraid.Braid
	db      kolostore.Backend
	l       *kolosecure.Listener
	stop    context.CancelFunc
	wg      sync.WaitGroup
	allowed sync.Map // keys the owner typed add for, each let in once
}

// openDoc opens the document stored in dir, or starts one (listen) or joins one (peer), and keeps it in sync.
func openDoc(ctx context.Context, id *koloid.Identity, dir, listen, peer string, owner ed25519.PublicKey) (*Doc, error) {
	db, err := kolostore.OpenPebble(filepath.Join(dir, "doc"))
	if err != nil {
		return nil, err
	}
	opts := kolobraid.Options{Backend: db, App: app, Apply: apply}
	br, err := kolobraid.Open(ctx, id, opts)
	switch {
	case errors.Is(err, kolobraid.ErrNotFound) && listen != "":
		log.Print("started a new document")
		br, err = kolobraid.Create(ctx, id, kolobraid.GroupConfig{App: app}, opts)
	case errors.Is(err, kolobraid.ErrNotFound):
		br, err = join(ctx, id, opts, peer, owner)
	}
	if err != nil {
		db.Close()
		return nil, err
	}
	ctx, stop := context.WithCancel(ctx)
	d := &Doc{braid: br, db: db, stop: stop}
	if listen == "" {
		d.wg.Go(func() { d.follow(ctx, id, peer, owner) })
		return d, nil
	}
	if d.l, err = kolosecure.Listen(ctx, listen, id, nil); err != nil {
		d.Close()
		return nil, err
	}
	log.Print("listening on ", listen)
	d.wg.Go(func() { d.accept(ctx) })
	return d, nil
}

// Close stops the document.
func (d *Doc) Close() {
	d.stop()
	if d.l != nil {
		d.l.Close()
	}
	d.wg.Wait()
	d.braid.Close()
	d.db.Close()
}

The third piece runs the commands. say appends one entry to your own strand, your signed kolo-log. show reads two snapshots: View (settled and tentative lines) and Settled (settled lines only), and marks each line with the one it is in. add and remove take a key of 64 hex digits and refuse anything else with a key is 64 hex digits.

doc.go
// do runs one command typed in the terminal.
func (d *Doc) do(ctx context.Context, cmd, arg string) error {
	ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
	defer cancel()
	key, _ := hex.DecodeString(arg)
	switch cmd {
	case "say":
		if len(arg) == 0 || len(arg) > maxLine {
			return errors.New("a line is 1 to 500 bytes")
		}
		_, err := d.braid.Append(ctx, []byte(arg))
		return err
	case "show":
		d.show(os.Stdout)
		return nil
	case "add":
		if len(key) != ed25519.PublicKeySize {
			return errKey
		}
		d.allowed.Store(string(key), true)
		log.Printf("%.8s can join now", arg)
		return nil
	case "remove":
		if len(key) != ed25519.PublicKeySize {
			return errKey
		}
		_, err := d.braid.RemoveMember(ctx, key)
		return err
	}
	return errors.New("commands: say <text>, show, add <key>, remove <key>")
}

// show prints every line in the braid's order with its writer, and whether its place is settled or still tentative.
func (d *Doc) show(w io.Writer) {
	view, settled := d.braid.View(), d.braid.Settled()
	defer view.Close()
	defer settled.Close()
	n := 0
	for k, v := range view.Scan([]byte("line/"), []byte("line0")) {
		state := "tentative"
		if s, err := settled.Get(k); err == nil && bytes.Equal(s, v) {
			state = "settled"
		}
		n++
		fmt.Fprintf(w, "%d. %x: %s (%s)\n", n, v[:4], v[4:], state)
	}
	if n == 0 {
		fmt.Fprintln(w, "the document is empty")
	}
}

The fourth piece runs on Ana's side. Each connection is a kolo-secure connection, so Ana knows the key of the person at the other end. A member is synced with ServeConn. A key Ana typed add for may send one join request: Admit adds it as a writer and returns the welcome. The permission is used up only after Admit succeeded, so a connection that sends no request does not burn it, and Ana does not have to type add again. Every other connection is closed.

doc.go
// accept syncs with members, lets in a key the owner typed add for, and closes every other connection.
func (d *Doc) accept(ctx context.Context) {
	for {
		c, err := d.l.Accept(ctx)
		if err != nil {
			return
		}
		d.wg.Go(func() {
			defer c.Close()
			key := c.RemotePublicKey()
			v := d.braid.View()
			member := slices.ContainsFunc(v.Members(), func(m kolobraid.Member) bool { return bytes.Equal(m.Key, key) })
			v.Close()
			if member {
				d.braid.ServeConn(ctx, c)
			} else if _, ok := d.allowed.Load(string(key)); ok {
				if err := d.admit(ctx, c); err != nil {
					log.Print("add: ", err)
				} else {
					d.allowed.Delete(string(key))
				}
			}
		})
	}
}

// admit reads a join request on c, adds its sender as a writer and sends the welcome back.
func (d *Doc) admit(ctx context.Context, c *kolosecure.Conn) error {
	ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
	defer cancel()
	s, err := c.AcceptStream(ctx)
	if err != nil {
		return err
	}
	req, err := io.ReadAll(io.LimitReader(s, kolobraid.MaxJoinRequest))
	if err != nil {
		return err
	}
	writer := kolobraid.Role(0) // not an admin and not a settler
	welcome, err := d.braid.Admit(ctx, c.RemotePublicKey(), req, &kolobraid.AdmitOptions{Roles: &writer})
	if err != nil {
		return err
	}
	if _, err := s.Write(welcome); err != nil {
		d.braid.RemoveMember(ctx, c.RemotePublicKey())
		return err
	}
	s.Close()
	select { // the joiner hangs up once it has the welcome
	case <-c.Context().Done():
	case <-ctx.Done():
	}
	log.Printf("added %x", c.RemotePublicKey()[:4])
	return nil
}

The last piece runs on Bruno's side. NewJoin makes his strand and his join request. He sends it to Ana once a second until she lets him in, then Complete opens the document with her welcome. After that he stays connected to Ana and syncs with ServeConn, and connects again if she restarts.

doc.go
// join asks the owner to let this key in, once a second until the owner typed add for it, and opens the document.
func join(ctx context.Context, id *koloid.Identity, opts kolobraid.Options, peer string, owner ed25519.PublicKey) (*kolobraid.Braid, error) {
	j, err := kolobraid.NewJoin(ctx, id, opts)
	if err != nil {
		return nil, err
	}
	req, err := j.Request(nil)
	if err != nil {
		return nil, err
	}
	log.Print("waiting for the owner to type add and my key")
	for {
		if welcome, err := ask(ctx, id, peer, owner, req); err == nil {
			log.Print("joined the document")
			return j.Complete(ctx, welcome)
		}
		select {
		case <-ctx.Done():
			return nil, ctx.Err()
		case <-time.After(time.Second):
		}
	}
}

// ask sends the join request to the owner and returns the welcome.
func ask(ctx context.Context, id *koloid.Identity, peer string, owner ed25519.PublicKey, req []byte) ([]byte, error) {
	ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
	defer cancel()
	c, err := kolosecure.Dial(ctx, peer, id, owner, nil)
	if err != nil {
		return nil, err
	}
	defer c.Close()
	s, err := c.OpenStream(ctx)
	if err != nil {
		return nil, err
	}
	s.SetDeadline(time.Now().Add(10 * time.Second))
	if _, err := s.Write(req); err != nil {
		return nil, err
	}
	s.Close()
	welcome, err := io.ReadAll(io.LimitReader(s, kolobraid.MaxWelcome))
	if err == nil && len(welcome) == 0 {
		err = errors.New("not added yet")
	}
	return welcome, err
}

// follow keeps a connection to the owner and syncs the document over it, again after the owner restarts.
func (d *Doc) follow(ctx context.Context, id *koloid.Identity, peer string, owner ed25519.PublicKey) {
	for ctx.Err() == nil {
		if c, err := kolosecure.Dial(ctx, peer, id, owner, nil); err == nil {
			d.braid.ServeConn(ctx, c)
			c.Close()
		}
		select {
		case <-ctx.Done():
		case <-time.After(time.Second):
		}
	}
}

Compare your file with the finished one. This prints nothing when they are the same:

Terminal
git diff --no-index doc.go finished/doc.go

4Start the document in Terminal 1

In Terminal 1, run the program again with Ana's folder:

bash
go run . -dir "$ANA" -listen 127.0.0.1:7450

Ana keeps her key and starts a new document. Your times and keys will differ:

Text
07:05:08 me 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87
07:05:08 started a new document
07:05:08 type: say <text>, show, add <key>, remove <key>

5Join it from Terminal 2

In Terminal 2, go to the same kolo-braid-starter folder (use the path where you cloned it):

Terminal
cd kolo-braid-starter

Make a temporary folder for Bruno:

bash
BRUNO="$(mktemp -d)"

Then join with Ana's key, the long one on her me line (paste your own):

bash
go run . -dir "$BRUNO" -peer 127.0.0.1:7450 -key 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87

Bruno waits for Ana:

Text
07:05:10 me 3fa349c6c47a2177b99ebfb71af5ab9db93e411e8448e7a0ce725225469e9abf
07:05:10 waiting for the owner to type add and my key

In Terminal 1, type add and Bruno's key (paste your own):

Text
add 3fa349c6c47a2177b99ebfb71af5ab9db93e411e8448e7a0ce725225469e9abf

Ana lets him in:

Text
07:05:13 3fa349c6 can join now
07:05:13 added 3fa349c6

In Terminal 2, Bruno is in:

Text
07:05:13 joined the document
07:05:13 type: say <text>, show, add <key>, remove <key>

6Write together

In Terminal 1, type:

Text
say Kolo meeting notes

In Terminal 2, type:

Text
say We meet on Friday at 10

Type show in both terminals. Both print the same document, in the same order, with the short key of each writer:

Text
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)

7See a tentative line

A line is tentative while its place in the order can still change, for example when someone else wrote a line at the same time. It is settled once a settler has fixed its place, and then it never moves again. Here Ana is the only settler, so a line settles only while Ana is online.

Stop Ana with Ctrl-C in Terminal 1. In Terminal 2, type say Ana brings the slides, then show:

Text
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)
3. 3fa349c6: Ana brings the slides (tentative)

Bruno's new line is in his document, but its place is not fixed yet. In Terminal 1, start Ana again with the same folder:

bash
go run . -dir "$ANA" -listen 127.0.0.1:7450

Her key and her document are still there, so she does not start a new one:

Text
07:05:23 me 16dbfe82b405e09e5d28706ab7ff5974d8c52c9c9f88a8f14f20ba25cb13da87
07:05:23 type: say <text>, show, add <key>, remove <key>

Bruno connects again within a few seconds and sends her his new line, which she settles. Type show in both terminals:

Text
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)
3. 3fa349c6: Ana brings the slides (settled)

8Remove a writer

In Terminal 1, remove Bruno with his key (paste your own):

Text
remove 3fa349c6c47a2177b99ebfb71af5ab9db93e411e8448e7a0ce725225469e9abf

Then, in Terminal 1, type say Bruno left the project, and in Terminal 2, type say Bruno writes after the removal. Wait a few seconds and type show in both terminals. Ana's document has her new line, not Bruno's:

Text
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)
3. 3fa349c6: Ana brings the slides (settled)
4. 16dbfe82: Bruno left the project (settled)

Bruno's document does not get Ana's new line, and his own line stays tentative:

Text
1. 16dbfe82: Kolo meeting notes (settled)
2. 3fa349c6: We meet on Friday at 10 (settled)
3. 3fa349c6: Ana brings the slides (settled)
4. 3fa349c6: Bruno writes after the removal (tentative)

Ana stopped sending Bruno anything, and every entry Bruno writes after his removal is void everywhere: no other member shows it. On his side it stays tentative, because nothing settles for him anymore. Bruno keeps the lines he received before.

9Stop everything

Press Ctrl-C in Terminal 2, then in Terminal 1.

What just happened

  • Each person wrote to a strand of their own: a kolo-log in which every entry is signed by its writer, stored with kolo-store.
  • kolo-braid merged the strands into one order. Every peer that holds the same entries computes the same order, byte for byte, and calls apply in it, so both documents were the same.
  • Ana created the document, so she was its admin and its only settler. Bruno was added as a writer: he can write, not add or remove anyone.
  • Membership is ordered like the lines. Bruno's lines before his removal kept their place; the one after it was void.
  • The two terminals synced over one kolo-secure connection on 127.0.0.1, with ServeConn.

Between two computers

The two terminals connect directly: -listen opens the port and -peer dials it. They do not use the DHT or the swarm, so there is no bootstrap to configure.

  • -listen and -peer must be an IPv4 IP:PORT that is not 0.0.0.0 and has a port other than 0. Anything else exits with "..." is not an IPv4 IP:PORT such as 127.0.0.1:7450.
  • On one computer, use 127.0.0.1, as in the steps above.
  • On the same local network, Terminal 1 passes -listen with the IPv4 address of its network card, such as -listen 192.168.1.20:7450, and Terminal 2 passes -peer 192.168.1.20:7450 with the key.
  • Between two different networks it works only when the first computer has a public IPv4 address that the second can reach, with the UDP port open in the firewall.
  • Two computers that are both behind home routers will not connect: this program does not punch through NAT and uses no relay.

For that case use the Kolo examples (chat, share, meet, board) or the kolo-swarm library, which find peers through the public entry nodes by default.

Before you build on it

  • kolo-braid is beta and experimental, like every Kolo library at v0.x: the API may change before v1.0.0.
  • It is tested with unit, property and fuzz tests, bounded model checks of settlement and membership, which cover only the sizes they enumerate and are not a proof for any size, and scenarios with several processes on one Mac, on 127.0.0.1 only. It has not been tested over a real network or through NATs.
  • With one settler, the document depends on that device: while Ana is offline, nothing settles. Admit with no options admits a plain member, as this program does for Bruno; AdmitOptions.GrowSettlers makes the first joiners admins and settlers, up to 4 seats, and is for people you trust that far.
  • A line can never be erased: every peer keeps every entry, history is never pruned, and each entry costs about 600 bytes of disk on every peer. Keep data that may need erasure outside the braid.
  • The order among lines written at the same time is not fair, and a settler can try to choose it, so do not build a "first one wins" rule on it.
  • Settlement keeps a small round mark per settler for each round, until the settler config changes. For 4 settlers that settle all day at the default pace, that is up to about 22 MB of memory and 55 MB of disk a day. They are what lets a pulse that arrives late get its exact round, and a settler config change removes them.
  • The join here is the smallest one that works: Ana lets in a key she typed, on a direct connection. A real app carries the join request and the welcome with kolo-invite, as in Invite someone to a group, in a terminal program and on the kolo-braid page.
  • ServeConn works over direct and relayed connections. To reach someone behind a NAT, see Run your own infrastructure.

What was not tested

Every run behind this guide was on one Mac, on 127.0.0.1. The Windows and Linux commands were not run.

Next steps