kolo docsbeta

kolo-swarm

Join a topic and keep connections to the peers found on it.

On this page

kolo-swarm joins topics and keeps at most one kolo-secure connection per remote key to the peers found on them. A Swarm runs its own kolo-dht node: it announces and looks up the topics it joined, dials the peers it finds with backoff and jitter, and accepts the peers that dial it. With relays listed in Config.Relays, a peer it cannot reach directly is dialed through a kolo-relay that both peers list. A connection the application opened another way enters through Attach. It keeps direct paths first and uses a relay last, as ICE (RFC 8445) does, and it bounds connections per address against the eclipse attacks described by Heilman et al..

All steps

Joining a topic

You name a topic; kolo-swarm finds the peers on it and hands you connections, one per peer.

  1. Join announces your signed record on the topic at up to 20 DHT servers close to it, and looks the topic up.
  2. Each record found is a Found event. By default kolo-swarm dials up to 3 peers at a time and keeps at most 64 peers, connected or being dialed.
  3. Connect asks a server that stored the peer's record to pass the request on. Then both sides send packets to open a direct path (hole punching).
  4. kolo-secure proves both peers' keys on the new connection.
  5. On a new stream, the peer with the larger key sends hello with the topics it found the other on. The smaller key decides, and answers with a generation number.
  6. One connection per peer is kept; a duplicate closes. The app gets Connected, and Lost when it ends. Reconnects back off from 1 second to 10 minutes.

Install

go get github.com/ciranda-dev/kolo-swarm

Builds without cgo.

Quick start

A kolo-dht server and two swarms on loopback join one topic, connect, and echo a message.

main.go
package main

import (
	"context"
	"crypto/sha256"
	"fmt"
	"io"
	"log"
	"net"
	"net/netip"
	"time"

	kolodht "github.com/ciranda-dev/kolo-dht"
	koloid "github.com/ciranda-dev/kolo-id"
	koloswarm "github.com/ciranda-dev/kolo-swarm"
)

func swarm(ctx context.Context, server netip.AddrPort) *koloswarm.Swarm {
	id, err := koloid.Generate()
	if err != nil {
		log.Fatal(err)
	}
	s, err := koloswarm.New(ctx, "127.0.0.1:0", id, nil)
	if err != nil {
		log.Fatal(err)
	}
	if err := s.Bootstrap(ctx, []netip.AddrPort{server}); err != nil {
		log.Fatal(err)
	}
	return s
}

func connected(ctx context.Context, s *koloswarm.Swarm) *koloswarm.Peer {
	for {
		ev, err := s.Next(ctx)
		if err != nil {
			log.Fatal(err)
		}
		if ev.Kind == koloswarm.Connected {
			return ev.Peer
		}
	}
}

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	sid, err := koloid.Generate()
	if err != nil {
		log.Fatal(err)
	}
	server, err := kolodht.New(ctx, "127.0.0.1:0", sid, &kolodht.Config{Server: true})
	if err != nil {
		log.Fatal(err)
	}
	defer server.Close()
	saddr := server.Addr().(*net.UDPAddr).AddrPort()

	alice, bob := swarm(ctx, saddr), swarm(ctx, saddr)
	defer alice.Close()
	defer bob.Close()

	topic := koloswarm.Topic(sha256.Sum256([]byte("kolo example topic")))
	if err := bob.Join(ctx, topic, &koloswarm.JoinOptions{Announce: true}); err != nil {
		log.Fatal(err)
	}
	if err := alice.Join(ctx, topic, &koloswarm.JoinOptions{Lookup: true}); err != nil {
		log.Fatal(err)
	}
	pa, pb := connected(ctx, alice), connected(ctx, bob)
	fmt.Printf("alice connected to %x (direct %v)\n", pa.PublicKey()[:8], pa.Direct())

	go func() {
		st, err := pb.Conn().AcceptStream(ctx)
		if err != nil {
			return
		}
		io.Copy(st, st)
		st.Close()
	}()
	st, err := pa.Conn().OpenStream(ctx)
	if err != nil {
		log.Fatal(err)
	}
	if _, err := st.Write([]byte("hello kolo")); err != nil {
		log.Fatal(err)
	}
	st.Close()
	reply, err := io.ReadAll(st)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s from %x\n", reply, pa.PublicKey()[:8])
}

It prints the first 8 bytes of bob's key with direct true (the connection came from kolo-dht), then hello kolo echoed back. With no list, Bootstrap(ctx, nil) joins the public network through the entry nodes built into kolo-dht. For a private network, bootstrap from a few kolo-dht servers of your own, such as kolo-node; see Public and private networks.

API reference

The API reference is generated from the Go source and is in English.

Package koloswarm · github.com/ciranda-dev/kolo-swarm · v0.1.0

Package koloswarm joins Kolo topics and keeps connections to the peers found on them.

A Swarm owns a kolo-dht node. It announces and looks up the topics it joined, dials the peers it finds with backoff, accepts the peers that dial it, and keeps at most one kolo-secure connection per remote key, within fixed limits on peers, inbound peers, dials and handshakes. It uses relays only when Config.Relays lists some: then a peer it cannot reach directly is dialed through a kolo-relay that both peers list. Connect dials a known key at a known address, and a connection the application opened another way enters through Attach.

Bootstrap with no servers and no Config.BootstrapNodes joins the public network: Ciranda's entry nodes learn the node's IP address and public key, and an announced topic hash and IP address become visible there. Pass an explicit list to stay private.

v0.1.x is IPv4 only, and IPv6 input, IPv4-mapped included, is refused with ErrInvalidArgument.

Experimental: the API may change before v1.0.0.

Variables

var (
	// ErrInvalidConfig means a Config field is out of range, here or in kolo-dht (whose error is wrapped).
	ErrInvalidConfig = errors.New("koloswarm: invalid config")
	// ErrInvalidArgument means a nil identity or Conn, an IPv6 address (v0.1.x is IPv4 only), invalid JoinOptions, a
	// topic already joined, MaxTopics reached, or a Connect key, topic or address that cannot be used.
	ErrInvalidArgument = errors.New("koloswarm: invalid argument")
	// ErrClosed means the swarm is closed.
	ErrClosed = errors.New("koloswarm: closed")
	// ErrRefused means a connection was refused by Allow, a limit, the lack of a shared topic, or the peer; for
	// Connect also the key's, host's or address's backoff, the dial rate, or Leave of the topic.
	ErrRefused = errors.New("koloswarm: connection refused")
	// ErrWait wraps ErrRefused: Connect was refused for now by a backoff (of the key, the address, the key at the
	// address, or the host), the dial rate, a dial to the host in progress, or MaxConnecting dials running.
	ErrWait = fmt.Errorf("%w: wait", ErrRefused)
	// ErrDuplicate means the pair of keys already has a connection that this one does not replace, or, for Connect, a
	// lookup dial, a proved dial or a hello of the key, or a Connect dial of it at the same address.
	ErrDuplicate = errors.New("koloswarm: duplicate connection")
	// ErrProtocol means the peer sent a malformed hello, or none in time.
	ErrProtocol = errors.New("koloswarm: protocol error")
	// ErrLost means a connection ended for another reason: the peer closed it, or it timed out or failed; or a Connect
	// dial failed.
	ErrLost = errors.New("koloswarm: connection lost")
)

Functions

func LoadNodes

func LoadNodes(path string) ([]string, error)

LoadNodes reads a file of bootstrap nodes: one host:port per line, as in ParseNodes, where "#" starts a comment and blank lines are ignored. It reads only a regular file of at most 64 KiB; a bad line is ErrInvalidArgument.

func ParseNodes

func ParseNodes(s string) ([]string, error)

ParseNodes parses a comma-separated list of at most 16 bootstrap nodes, each host:port with an IPv4 address or a host name and a port from 1 to 65535. Spaces around entries and empty entries are ignored and duplicates dropped; IPv6 in any form, port 0, more entries or anything else invalid is ErrInvalidArgument.

Types

type Config

type Config struct {
	// DHT is passed to kolodht.New. A zero MaxAnnouncements becomes MaxTopics, a lower one is refused, and zero
	// kolo-transport MaxDialed, MaxConns and MaxMemory are raised to cover MaxPeers, and MaxMemory also the growth
	// WindowGrowth allows. With Allow set, WindowGrowth lets the connections Allow admitted grow their windows.
	DHT *kolodht.Config
	// MaxPeers caps connected peers plus dials in progress, inbound and outbound. Default 64, from 1 to 1024.
	MaxPeers int
	// MaxInbound caps the peers that connected to us. Default MaxPeers/2 rounded up, from 1 to MaxPeers.
	MaxInbound int
	// MaxConnectPeers caps the places Swarm.Connect holds. Default MaxPeers/4, from 1 to MaxPeers - MaxInbound; when
	// MaxInbound equals MaxPeers it must be unset, and Connect is always refused.
	MaxConnectPeers int
	// MaxTopics caps the topics joined at once. Default 16, from 1 to 32.
	MaxTopics int
	// MaxConnecting caps kolo-dht Connect calls in progress, and separately relay dials and Swarm.Connect dials, and
	// is also the Swarm.Connect dial starts per RetryMin. Default 3, from 1 to 16.
	MaxConnecting int
	// MaxHandshakes caps hellos in progress. From 4 up, a quarter is kept for Attach and another for known keys (an
	// outbound slot, or an outbound connection that lasted 15s). Default 16, from 1 to 256.
	MaxHandshakes int
	// MaxHandshakesPerIP caps hellos in progress from one source host, or through one relay. Default 4 (or
	// MaxHandshakes if lower), from 1 to MaxHandshakes.
	MaxHandshakesPerIP int
	// MaxCandidates caps the keys kept from lookups, split equally between joined topics; the backoff of up to twice
	// as many dropped keys, and keys Swarm.Connect dialed, is kept. Default 1024, from 16 to 32768.
	MaxCandidates int
	// MaxEvents caps the queued Found events, and separately the queued Unreachable events; more are dropped.
	// Default 64, from 16 to 4096.
	MaxEvents int
	// HelloTimeout bounds the hello on a new connection. Default 10s, from 1s to 60s.
	HelloTimeout time.Duration
	// LookupInterval is the time between lookups of a topic, plus up to a fifth of it at random. Default 10m, from
	// 1s to 1h.
	LookupInterval time.Duration
	// RetryMin is the shortest wait before dialing a key again, and the period in which Swarm.Connect starts at most
	// MaxConnecting dials, so raising it slows Connect. Default 1s, from 100ms to 1m.
	RetryMin time.Duration
	// RetryMax is the longest wait before dialing a key again. Default 10m, from RetryMin to 1h.
	RetryMax time.Duration
	// Allow, if set, is asked before every dial and every new connection; false or a panic refuses. addr is the
	// address to dial or the connection's remote address, zero when the Conn has none.
	Allow func(key ed25519.PublicKey, addr netip.AddrPort) bool
	// BootstrapNodes are the entry nodes Bootstrap uses when it is given none: at most 16 host:port entries, each an
	// IPv4 address or a host name resolved to IPv4, as from ParseNodes or LoadNodes. The default is empty, and Bootstrap
	// then falls back to kolo-dht's public entry nodes, which then receive this node's IP address and public key; set
	// it to stay off the public network.
	BootstrapNodes []string
	// Relays, if not empty, connect peers that cannot connect directly: at most 16, in order of preference. While a
	// topic is joined with Announce the swarm keeps a reservation at each and admits the peers that come through
	// them, and a key whose kolo-dht Connect failed is dialed through them in turn, the one that carried it last
	// first, with the swarm's own DHT.Secure settings. Two peers meet through any relay both list. Empty never uses
	// a relay.
	Relays []Relay
}

Config sets the limits of a Swarm. A zero field takes its default; out of range is ErrInvalidConfig.

type Conn

type Conn interface {
	RemotePublicKey() ed25519.PublicKey
	OpenStream(ctx context.Context) (*kolotransport.Stream, error)
	AcceptStream(ctx context.Context) (*kolotransport.Stream, error)
	OpenUniStream(ctx context.Context) (*kolotransport.SendStream, error)
	AcceptUniStream(ctx context.Context) (*kolotransport.ReceiveStream, error)
	Context() context.Context
	Close() error
	CloseWithError(code uint64, msg string) error
}

Conn is an authenticated connection to one peer; *kolosecure.Conn and *kolorelay.Conn both satisfy it. RemotePublicKey must be a key the connection has proved, as those two do; the swarm trusts it without a check.

type Event

type Event struct {
	Kind      EventKind
	Topic     Topic             // set for Found and Unreachable
	PublicKey ed25519.PublicKey // the remote key, or the relay's key for RelayChanged
	Addr      netip.AddrPort    // the remote address; zero for RelayChanged
	Peer      *Peer             // set for Connected and Lost
	Err       error             // set for Unreachable and Lost, and for RelayChanged when the reservation is not held
}

Event is one thing that happened in the swarm; which fields are set depends on Kind.

type EventKind

type EventKind int

EventKind says what an Event reports.

const (
	// Found means a lookup of Topic returned PublicKey at Addr, for the first time since it was last absent.
	Found EventKind = iota + 1
	// Connected means Peer is connected; its Conn is ready for streams.
	Connected
	// Unreachable means kolo-dht could not connect to PublicKey, found on Topic; Err is kolodht.ErrUnreachable. With
	// Config.Relays set, it comes only after the relays failed too.
	Unreachable
	// Lost means the connection of a Peer returned by an earlier Connected ended. Err says why, with nothing the peer
	// chose: ErrDuplicate (replaced, or closed as a duplicate), ErrClosed (closed here), ErrRefused or ErrProtocol (the
	// peer refused it), or ErrLost (anything else).
	Lost
	// RelayChanged means the reservation at the relay whose key is PublicKey is held (Err nil) or not held; Err is
	// then kolorelay.ErrNotAllowed, kolorelay.ErrLimit, kolorelay.ErrReservationEnded, kolosecure.ErrUnexpectedPeer,
	// ErrInvalidConfig (that relay's key cannot be used; the swarm stops using it), ErrClosed (no announced topic is
	// joined) or ErrLost (anything else). Topic, Addr and Peer are zero. Only the newest one per relay waits in the
	// queue.
	RelayChanged
)

The event kinds.

func (EventKind) String
func (k EventKind) String() string

String returns the kind's name.

type JoinOptions

type JoinOptions struct {
	// Announce publishes our record on the topic.
	Announce bool
	// Lookup finds the other peers of the topic and dials them.
	Lookup bool
	// MaxPeers caps the peers dialed or admitted for this topic. Default and maximum: Config.MaxPeers.
	MaxPeers int
}

JoinOptions says how to join a topic. Announce publishes our record; Lookup finds and dials others.

type Peer

type Peer struct {
	// contains filtered or unexported fields
}

Peer is a connected peer: one connection to one remote key.

func (Peer) Addr
func (p *Peer) Addr() netip.AddrPort

Addr returns the peer's UDP address, or the zero AddrPort if its Conn has no RemoteAddr method.

func (Peer) Conn
func (p *Peer) Conn() Conn

Conn returns the connection; the peer is lost when its Context ends.

func (Peer) Direct
func (p *Peer) Direct() bool

Direct reports whether the connection is direct, from kolo-dht or Connect, rather than through one of Config.Relays or from Attach.

func (Peer) Outbound
func (p *Peer) Outbound() bool

Outbound reports whether the peer holds one of our outbound places (not capped by MaxInbound), whichever side opened the kept connection.

func (Peer) PublicKey
func (p *Peer) PublicKey() ed25519.PublicKey

PublicKey returns a copy of the peer's kolo-id public key.

func (Peer) Topics
func (p *Peer) Topics() []Topic

Topics returns the topics this peer currently shares with us, as far as the peer claimed them: a topic hash is not secret, so any peer can claim it. Config.Allow, which sees the key, is where to admit only known members.

type Relay

type Relay struct {
	// Addr is the relay's host:port, an IPv4 address or a host name resolved to IPv4, in the form of a
	// BootstrapNodes entry. A name must point to one relay process.
	Addr string
	// PublicKey is the relay's kolo-id public key.
	PublicKey ed25519.PublicKey
}

Relay is a kolo-relay that peers of the network may list, used when a direct connection fails.

func LoadRelays
func LoadRelays(path string) ([]Relay, error)

LoadRelays reads a file of relays, one host:port@key per line as in ParseRelays, where "#" starts a comment and blank lines are ignored. It reads only a regular file of at most 64 KiB; a bad line is ErrInvalidArgument.

func ParseRelays
func ParseRelays(s string) ([]Relay, error)

ParseRelays parses a comma-separated list of at most 16 relays, each host:port@key with the key as 64 hex characters. Spaces and empty entries are ignored and duplicates dropped; one key with two addresses, one address with two keys, IPv6 in any form, port 0 or anything else invalid is ErrInvalidArgument.

type Swarm

type Swarm struct {
	// contains filtered or unexported fields
}

Swarm keeps connections to the peers of the topics it joined, on its own kolo-dht node.

func New
func New(ctx context.Context, addr string, id *koloid.Identity, cfg *Config) (*Swarm, error)

New opens a kolo-dht node on addr, such as "127.0.0.1:0" or ":4000", as id, and starts the swarm on it. v0.1.x is IPv4 only: an IPv6 addr, IPv4-mapped included, is ErrInvalidArgument.

func (Swarm) Addr
func (s *Swarm) Addr() net.Addr

Addr returns the local address of the node's UDP socket.

func (Swarm) Attach
func (s *Swarm) Attach(ctx context.Context, c Conn, outbound bool) error

Attach runs the hello on a connection the application opened (outbound if we did) and returns nil once the peer is admitted. The connection must authenticate RemotePublicKey itself, as kolo-secure and kolo-relay do: the hello carries no proof, so a key the peer merely claims lets it impersonate any key. The swarm owns c from the call on; both ends must call it at once (see the README). With Config.Relays, it counts as relayed: a direct connection to the key replaces it.

func (Swarm) Bootstrap
func (s *Swarm) Bootstrap(ctx context.Context, servers []netip.AddrPort) error

Bootstrap passes servers to kolo-dht Bootstrap; its ErrNoNodes passes through, and its ErrInvalidArgument is wrapped in ours. An IPv6 server address, IPv4-mapped included, is ErrInvalidArgument. With no servers it resolves Config.BootstrapNodes instead (ErrNoNodes if none resolves), each name within 5s and ctx.

With neither, a nil servers list joins the public network: Ciranda's entry nodes built into kolo-dht learn this node's IP address and public key, and Join with Announce publishes the topic hash and IP address there. A nil slice from a flag or an empty parse counts as nil. To stay private, pass your own non-empty list or set Config.BootstrapNodes; an empty list that is not nil is ErrInvalidArgument and sends nothing.

func (Swarm) Close
func (s *Swarm) Close() error

Close closes every peer connection and the node, and waits for the swarm's goroutines. It is safe to call more than once.

func (Swarm) Connect
func (s *Swarm) Connect(ctx context.Context, topic Topic, pub ed25519.PublicKey, addr netip.AddrPort, from netip.Addr) error

Connect dials pub at addr, which the peer at from supplied (zero: unknown, so global addresses only), and runs the hello for topic; it returns nil once this dial's connection is admitted. Errors: ErrInvalidArgument for unusable arguments, ErrRefused for Allow, a limit or the peer (ErrWait when a backoff, the dial rate or dials in progress refuse it for now), ErrDuplicate when the key already has a place or a hello, or a Connect dial at addr, and ErrLost when the dial fails (wrapping kolosecure.ErrUnexpectedPeer when another key answered). See the guide for the rest.

func (Swarm) Join
func (s *Swarm) Join(ctx context.Context, topic Topic, opts *JoinOptions) error

Join announces topic, looks it up, or both (nil opts), and connects to the peers found. With Announce it returns once kolo-dht stored our record, and its errors pass through; the topic is then not joined. Lookups run every LookupInterval in the background. If Leave(topic) runs during Join, Join returns context.Canceled. With Announce and Config.Relays, it first waits for the first reservation attempt at each relay, in parallel, except relays known down.

func (Swarm) Leave
func (s *Swarm) Leave(topic Topic)

Leave stops announcing and looking up topic, and closes the peers that share no other topic with us; with Config.Relays, those closes may finish after it returns.

func (Swarm) Next
func (s *Swarm) Next(ctx context.Context) (Event, error)

Next returns the next event. Connected and Lost are never dropped; Found and Unreachable are dropped past MaxEvents. A peer can be replaced by a newer connection to the same key: its Lost, with ErrDuplicate, comes before the new Connected.

func (Swarm) PublicKey
func (s *Swarm) PublicKey() ed25519.PublicKey

PublicKey returns a copy of the swarm's kolo-id public key.

type Topic

type Topic = kolodht.Topic

Topic is a 32-byte topic, the same type as kolo-dht's.

Good to know

Public and private networks

A network is the set of kolo-dht servers that know each other, and a node is in the network of the entry nodes it bootstraps from. With no list at all, Bootstrap(ctx, nil) joins the public network through the entry nodes built into kolo-dht (kolodht.PublicServers), which Ciranda runs for everyone, free, with no uptime guarantee while Kolo is a beta. Config.BootstrapNodes is empty by default; when it is set, or when you pass servers, those are used instead. An empty list that is not nil is ErrInvalidArgument and sends nothing, so an application that must stay private never joins the public network by mistake.

Who learns what on the public network: with no list, the two entry nodes learn the node's IP address and public key with its first packet, and every public-network node it queries, or that queries it, sees them too. With Join and Announce, the topic hash and the node's IP address are stored in the public network, and anyone who looks up that topic can read them; Lookup alone shows the topic hash to the nodes it asks. Nothing hides the address. To stay private, never rely on the default: pass your own list to Config.BootstrapNodes or to Bootstrap. A nil list means the public network also when it comes from a flag, a config file or an empty ParseNodes result, so check that it is not empty before the call; an empty list that is not nil is ErrInvalidArgument and works as a guard, for example servers := []netip.AddrPort{} before you add your own entries.

  • The public network needs no setup. Another public network, whose operators publish their entry nodes, is reached by putting that list in Config.BootstrapNodes (from ParseNodes or LoadNodes), or by passing it to Bootstrap.
  • A private network runs its own entry nodes. Its first entry node starts with no list (kolo-node run -first, or a kolo-dht server that never calls Bootstrap), more entry nodes bootstrap from it, and every member lists only these entry nodes. Run at least two on hosts that stay up, with their UDP port open.

ParseNodes parses a comma-separated list and LoadNodes reads a file, one entry per line, with # comments and blank lines ignored; a UTF-8 BOM and CRLF are accepted, and only a regular file of at most 64 KiB is read. An entry is host:port with an IPv4 address or a host name and a port from 1 to 65535. IPv6 in any form, port 0, more than 16 entries or a bad line is ErrInvalidArgument.

Host names in the list are resolved to IPv4 at each Bootstrap, with the system resolver, in parallel, each name within 5s and within ctx's deadline: at most 8 addresses per name and 64 in all, literal addresses first. A name's address is kept only if it is global or in the scope of the swarm's own address (loopback, link-local or private), so a DNS answer cannot point the swarm at a local network it was not given; on an unspecified address such as 0.0.0.0 only global answers count. Literal addresses are kept as configured. Names that do not resolve are skipped; if nothing resolves, Bootstrap returns ErrNoNodes.

What it is: separation by topology. A kolo-dht server learns other servers only from the servers it asks and from servers that send it requests, and a lookup only asks servers learned that way, so two networks with different entry nodes never see each other's servers, records or peers (a test checks this with two networks on one host).

What it is not: access control. The list is not a secret. Anyone who learns one entry address can bootstrap from it, look up topics and announce on them, and an entry node that also knows another network's servers joins the two. Decide who may connect with Config.Allow and kolo-secure keys, admit members with kolo-invite, and trust only signed records.

This program starts a private network on loopback: a first entry node, a second one that joins it, and two swarms that bootstrap only from them and connect.

main.go
// Command private starts a private network on loopback: two entry nodes of its own and two swarms that bootstrap
// only from them.
package main

import (
	"context"
	"crypto/sha256"
	"fmt"
	"log"
	"net"
	"net/netip"
	"time"

	kolodht "github.com/ciranda-dev/kolo-dht"
	koloid "github.com/ciranda-dev/kolo-id"
	koloswarm "github.com/ciranda-dev/kolo-swarm"
)

func identity() *koloid.Identity {
	id, err := koloid.Generate()
	if err != nil {
		log.Fatal(err)
	}
	return id
}

// entryNode starts a kolo-dht server; with no servers it is the first node of a new network.
func entryNode(ctx context.Context, servers []netip.AddrPort) *kolodht.Node {
	n, err := kolodht.New(ctx, "127.0.0.1:0", identity(), &kolodht.Config{Server: true})
	if err != nil {
		log.Fatal(err)
	}
	if len(servers) > 0 {
		if err := n.Bootstrap(ctx, servers); err != nil {
			log.Fatal(err)
		}
	}
	return n
}

func addr(n *kolodht.Node) netip.AddrPort { return n.Addr().(*net.UDPAddr).AddrPort() }

func connected(ctx context.Context, s *koloswarm.Swarm) *koloswarm.Peer {
	for {
		ev, err := s.Next(ctx)
		if err != nil {
			log.Fatal(err)
		}
		if ev.Kind == koloswarm.Connected {
			return ev.Peer
		}
	}
}

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	first := entryNode(ctx, nil)
	defer first.Close()
	second := entryNode(ctx, []netip.AddrPort{addr(first)})
	defer second.Close()

	// The list an application would read from its own config, a flag or an environment variable.
	nodes, err := koloswarm.ParseNodes(addr(first).String() + "," + addr(second).String())
	if err != nil {
		log.Fatal(err)
	}
	topic := koloswarm.Topic(sha256.Sum256([]byte("kolo private example")))
	var swarms []*koloswarm.Swarm
	for range 2 {
		s, err := koloswarm.New(ctx, "127.0.0.1:0", identity(), &koloswarm.Config{BootstrapNodes: nodes})
		if err != nil {
			log.Fatal(err)
		}
		defer s.Close()
		if err := s.Bootstrap(ctx, nil); err != nil {
			log.Fatal(err)
		}
		if err := s.Join(ctx, topic, nil); err != nil {
			log.Fatal(err)
		}
		swarms = append(swarms, s)
	}
	p := connected(ctx, swarms[0])
	connected(ctx, swarms[1])
	fmt.Printf("private network: %d entry nodes, connected to %x\n", len(nodes), p.PublicKey()[:8])
}

It prints private network: 2 entry nodes, connected to and 16 hex characters. The same network with kolo-node, on loopback, with the keys in a new temporary directory:

Terminal
dir=$(mktemp -d)
kolo-node run -listen 127.0.0.1:7447 -identity "$dir/a/node.key" -first
kolo-node run -listen 127.0.0.1:7448 -identity "$dir/b/node.key" -bootstrap 127.0.0.1:7447

Members then list both entry nodes, for example in a file read with LoadNodes:

Text
# entry nodes of our network
127.0.0.1:7447
127.0.0.1:7448   # second entry node

Relay fallback

Two peers whose direct connection fails, for example two computers behind firewalls that let only outgoing traffic through, can meet through a kolo-relay. Each side runs its own kolo-secure connection through it, so the relay copies encrypted bytes, and the target proves its key as on a direct connection. kolo-sync takes such a connection like a direct one.

Relays are a network setting, like the entry nodes: Config.Relays is an ordered list of at most 16 Relay values (Addr, PublicKey), empty by default, and with an empty list the swarm runs no relay code at all. ParseRelays and LoadRelays read the list as host:port@key, the key as 64 hex characters, with the rules of ParseNodes and LoadNodes:

Text
# relays of our network
203.0.113.20:7448@<64 hex characters>
relay.example.org:7448@<64 hex characters>   # resolved to IPv4 at each use

New refuses, with ErrInvalidConfig, an IPv6 address in any form, port 0, a bad name, a key that is not 32 bytes or is our own, 17 entries, and one key at two addresses or one address with two keys. An exact duplicate is dropped.

  • A peer that announces a topic keeps a reservation at every listed relay, taken before Join announces. A swarm that joins topics with Lookup only holds none. After the last announced Leave, the swarm closes its reservations and the inbound relayed peers that came through them.
  • Two peers meet through any relay that both list, identified by its key; the lists need not share an order. Lists that share no relay give no relayed connection.
  • On first contact, the relays are tried only after the direct Connect failed, not when it answered try later: the relay that carried that key last first, then list order, skipping relays known down. A key that needed a relay before is dialed directly and through its relays at once, and a direct connection that succeeds replaces the relayed one (Lost with ErrDuplicate, then Connected with Direct() true).
  • First contact can take up to 3 × kolo-dht PunchTimeout (135s at its default) when punches start but never connect, then one relay dial of at most 60s per relay tried.
  • A relayed Peer has Direct() false and a zero Addr(). Its Conn() is a *kolorelay.Conn whose RelayPublicKey() names the relay. Relayed connections use the swarm's own DHT.Secure settings, and Allow is asked for a relayed peer with a zero address before the end-to-end handshake.
  • RelayChanged reports each listed relay's reservation: Err nil when it is held, or kolorelay.ErrNotAllowed, kolorelay.ErrLimit, kolorelay.ErrReservationEnded, kolosecure.ErrUnexpectedPeer (another key answered), ErrInvalidConfig (the key cannot be used; that relay is no longer used), ErrClosed or ErrLost.

What a relay operator learns is on the kolo-relay page.

Limits

  • IPv4 only. An IPv6 address, IPv4-mapped included, given to New, Bootstrap or Attach is ErrInvalidArgument. A lookup record or kolo-dht connection with one is dropped before Allow sees it.
  • A zero Config field takes its default. A value out of range is ErrInvalidConfig.
  • MaxPeers: 64 connected peers plus dials in progress, inbound and outbound (1 to 1024).
  • MaxInbound: MaxPeers/2 rounded up, 32 at the default (1 to MaxPeers). Each topic has the same split: inbound peers take at most ceil(topic MaxPeers x MaxInbound / MaxPeers) of its places, 32 of 64 at the defaults. A connection with no shared topic that has room is closed with code 3.
  • JoinOptions.MaxPeers: default and maximum Config.MaxPeers. Out of range is ErrInvalidArgument.
  • MaxTopics: 16 topics joined at once (1 to 32). Past it, Join returns ErrInvalidArgument. A hello carries at most 32 topics.
  • MaxConnecting: 3 kolo-dht Connect calls at once (1 to 16), and separately as many relay dials.
  • Config.Relays: at most 16 relays, empty by default.
  • MaxHandshakes: 16 hellos in progress (1 to 256). From 4 up, a quarter is kept for Attach and another quarter for known keys.
  • MaxHandshakesPerIP: 4 hellos from one IPv4 address or through one relay, or MaxHandshakes if lower (1 to MaxHandshakes). At most 2 hellos run at once per key.
  • MaxCandidates: 1024 keys from lookups over all topics, split equally between joined topics (16 to 32768). The backoff state of up to twice as many dropped keys is kept.
  • MaxEvents: 64 queued Found events, and separately 64 Unreachable (16 to 4096); newer ones are dropped. At most 2 x MaxPeers + 2 x MaxEvents events wait.
  • HelloTimeout: 10s for a whole hello (1s to 60s). Each step of the hello has 2s, or HelloTimeout if shorter. A hello that misses its window closes the connection with code 5.
  • LookupInterval: 10m between lookups of a topic, plus up to a fifth of it at random (1s to 1h). Lookups are skipped while the topic is at its limit.
  • RetryMin: 1s (100ms to 1m). RetryMax: 10m (RetryMin to 1h). The wait before redialing a key after n failures is RetryMin + uniform[0, min(RetryMax, RetryMin·2^n) - RetryMin).
  • A connection that lasts 15s resets its key's failure count; one that ends sooner counts as a failure.
  • kolo-dht's ErrTryLater is not a failure, but each one grows the key's wait, from kolo-dht's RandomPunchInterval (20s by default) up to RetryMax, or that interval if longer.
  • A host takes at most two dials at once.
  • A HELLO is at most 1026 bytes and a REPLY at most 1044.
  • DHT: a zero MaxAnnouncements becomes MaxTopics, and a lower one is ErrInvalidConfig. Zero kolo-transport limits are raised: MaxDialed to MaxPeers + MaxHandshakes + kolo-secure MaxPending, at least 32 (144 at the defaults); MaxConns to MaxInbound + MaxHandshakes + MaxPending, at least 100 (112 at the defaults); MaxMemory to (MaxConns + MaxDialed) x MaxConnReceiveWindow, at least 1 GiB. Size these to the memory the process may use, and set GOMEMLIMIT a little below it.
  • BootstrapNodes: empty by default, at most 16 entries. Each name is a DNS query at every Bootstrap, and 16 entry nodes are more than a network needs to stay reachable; the cap equals kolo-node's, so one list works in both.
  • Fixed: at most 8 addresses per bootstrap name and 64 in all, 5s per name resolved in parallel within ctx, and a LoadNodes file of at most 64 KiB.
  • On loopback every peer is one host for the per-address limits of kolo-dht and kolo-transport. Tests with many local peers must raise them.

Faster transfers for admitted peers

A kolo-transport stream moves at most one window per round trip, and the windows start small for every connection because a hostile peer can make an application hold memory for unread data. kolo-transport can grow them for a connection the application marks as trusted, and it never learns who a peer is. The swarm marks a connection once Allow has admitted its peer, so growth is for the peers you chose:

Go
cfg := &koloswarm.Config{
	DHT: &kolodht.Config{Secure: &kolosecure.Config{Transport: &kolotransport.Config{
		WindowGrowth:           true,
		MaxStreamReceiveWindow: 256 << 10,
		MaxConnReceiveWindow:   256 << 10, // equal, so a connection nobody marked keeps exactly the old windows
	}}},
	Allow: func(key ed25519.PublicKey, addr netip.AddrPort) bool { return paired[string(key)] },
}
  • Growth is granted to every connection Allow admitted, on the accepted path, the dialed path and Attach, once the swarm has made it a Peer. For an application whose Allow is a pairing list, that is the paired peer only.
  • A connection Allow refused, or that the swarm did not admit for another reason (no shared topic, full), is never marked. With no Allow there is no trust decision, so the swarm marks nothing and no connection grows, even with WindowGrowth on. A relayed connection is never marked either.
  • WindowGrowth is off by default and then no window grows. When the swarm derives MaxMemory (a zero value) it adds the growth MaxGrownMemory allows (64 MiB by default), so kolo-transport's memory check still passes at any MaxPeers.

Security

  • Attach and Conn trust the key a connection reports: the connection must authenticate RemotePublicKey itself, as kolo-secure and kolo-relay do, because the hello carries no proof.
  • Peer.Topics is what the peer claimed. Use Config.Allow to admit only keys you know.
  • MaxInbound keeps part of the slots, overall and per topic, for peers chosen from lookups, against eclipse attacks.
  • Half of each topic's candidate share is kept for known keys. The rest is a uniform random sample of the lookup's other records; record order, reported age and claimed address never count.
  • A connection from Attach never replaces one from kolo-dht (ErrDuplicate). One from kolo-dht replaces an attached one.
  • The Err of a Lost event is a bare sentinel with nothing the peer chose.
  • Peer.Conn().Close() ends a current connection, for example after a key is banned.
  • Experimental. The API may change before v1.0.0.

Known limits

  • Two peers meet through a relay only when their Config.Relays share one. A dialer cannot learn where the other side reserved, so it tries its relays one at a time.
  • Not guaranteed: finding or reaching every peer (kolo-dht's limits), true topic claims, resistance to many keys from one or many addresses (a flood can fill the sampled part of every share), and fairness between topics beyond the per-topic limits.
  • In a simultaneous open, each side may see Connected, Lost, Connected for one key.
  • A peer whose punched connections are full answers try later for as long as they last, so its key can wait with no Unreachable event.

Errors

ErrorMessage
ErrClosedkoloswarm: closed
ErrDuplicatekoloswarm: duplicate connection
ErrInvalidArgumentkoloswarm: invalid argument
ErrInvalidConfigkoloswarm: invalid config
ErrLostkoloswarm: connection lost
ErrProtocolkoloswarm: protocol error
ErrRefusedkoloswarm: connection refused
ErrWait%w: wait

Test with errors.Is. ErrInvalidConfig wraps a kolo-dht config error, and Bootstrap wraps kolo-dht's ErrInvalidArgument in ours, so both sentinels match; kolo-dht's ErrNoNodes, and the ErrNotStored or ErrNoNodes of Join with Announce, pass through unchanged. The Err of a Lost event is ErrDuplicate (close code 2), ErrRefused (3 or 4), ErrProtocol (5), ErrClosed (closed here) or ErrLost.

References