In about 30 to 45 minutes you write the Kolo part of Kolo Meet, a desktop example for video calls. The window, built with Wails 3 and React, is already done, and so is most of the Go service behind it: the call's states, the network, the notch and the tray. You write the parts that use Kolo: the cameras and microphones, the invite, the relay that the person who starts the call runs, your own broadcast, watching the others, and the group keys.
A call is camera, microphone and nothing else: there is no chat, no text
message and no file sharing. Everything here stays on your computer, on the
address 127.0.0.1, and you end with three copies of the app in one call.
1Check what you need and run the window
You need Go 1.26 or newer, Node.js with npm, Git, and three terminal windows. If Go is not installed yet, follow the first step of Build a chat in your terminal. Then install the Wails 3 command line tool, which builds the app:
go install github.com/wailsapp/wails/v3/cmd/wails3@v3.0.0On Linux, run wails3 doctor: it lists the system packages the window needs.
Only the window needs cgo. The audio, video and network libraries are Go
only: you install no codec, no sound library and no compiler for them.
Get the starter:
git clone https://github.com/ciranda-dev/kolo-meet-wails3-starter.git && cd kolo-meet-wails3-starterIts go.mod already lists every library you use:
kolo-video for cameras and VP8,
kolo-audio for microphones, speakers and Opus,
kolo-invite for the invite,
kolo-moq for the relay and the keys, and
kolo-media v0.1.0 for the broadcasts, with
kolo-swarm, kolo-dht
and kolo-secure underneath.
Build it. The first build takes a few minutes, because the video codec is a large Go package; later builds use Go's cache. On macOS you may see linker warnings about the macOS version; they are harmless.
wails3 buildRun it with a data folder of its own. -first makes this window run the
first entry node of a small network on 127.0.0.1:4000, the address the
other windows ask first to find each other. It is for local testing only: the
finished Kolo Meet uses the public network by default, and -bootstrap points
it at an entry node of your own.
./bin/kolo-meet -data "$(mktemp -d)/ana" -first 127.0.0.1:4000$dir = New-Item -ItemType Directory -Path (Join-Path $env:TEMP "kolo-meet-$(Get-Random)")
.\bin\kolo-meet.exe -data (Join-Path $dir.FullName "ana") -first 127.0.0.1:4000./bin/kolo-meet -data "$(mktemp -d)/ana" -first 127.0.0.1:4000Once the network is ready, the terminal shows the app's key among the lines Wails prints. Your time and key will differ:
2026/11/16 09:53:02 kolo-id c7865ee7c063ed613ff06f2f24a706bc192e07e560dce797a7a5aef01d63ada7The window asks for a name: type one and click Continue. Click Start a call:
the lobby shows the camera preview, the microphone meter and the device
pickers. The devices are made up by the starter, in starter.go: turn the
camera on and you see moving grey stripes, turn the microphone on and the
meter moves every half second. Nothing real is opened yet. Click Start call:
the call ends at once with "Could not start the call: not implemented in the
starter (step 3 of the guide)." Every part you write says so until you write
it.
Here is what you write, and where. Each function is already in the starter,
as a stub that returns ErrNotInStarter, so the app always builds:
| Step | File | What you write |
|---|---|---|
| 2 | hw.go | listing and opening cameras, microphones and speakers, and their errors |
| 3 | host.go, member.go, publisher.go | the relay and the invite, entering the call, your audio track |
| 4 | guest.go, member.go, remote.go | joining with an invite, watching the others, decoding what they send |
| 5 | member.go | the group keys: who gets yours, and how you get theirs |
Steps 7 and 8 add no code: they show two parts that the starter already has, the call quality report and the display rate.
2Devices and the lobby preview
The service reaches the machine only through the hardware interface in
hw.go. The app's version is kolo-video and kolo-audio; the tests use made-up
devices, so a test never opens a real camera, microphone or speaker. Listing
never opens a device either, so it never asks for permission: only turning a
device on does, and turning it off closes it, so the system's "in use" light
goes out.
In hw.go, replace the import block with:
import (
"context"
"errors"
"io/fs"
koloaudio "github.com/ciranda-dev/kolo-audio"
kolovideo "github.com/ciranda-dev/kolo-video"
)Then replace everything from // realHardware is to the end of the file with
the three blocks below, pasted one after the other.
// realHardware is kolo-video and kolo-audio on the machine.
type realHardware struct{}
func list[T any](ds []T, err error, conv func(T) Device) ([]Device, error) {
out := []Device{}
for _, d := range ds {
out = append(out, conv(d))
}
if errors.Is(err, kolovideo.ErrNoDevice) || errors.Is(err, kolovideo.ErrUnsupported) ||
errors.Is(err, koloaudio.ErrNoDevice) || errors.Is(err, koloaudio.ErrUnsupported) {
return out, nil
}
return out, err
}
func (realHardware) Cameras() ([]Device, error) {
ds, err := kolovideo.Devices()
return list(ds, err, func(d kolovideo.Device) Device { return Device{ID: d.ID, Name: d.Name, Default: d.Default} })
}
func audioDevices(kind koloaudio.Kind) ([]Device, error) {
ds, err := koloaudio.Devices(kind)
return list(ds, err, func(d koloaudio.Device) Device { return Device{ID: d.ID, Name: d.Name, Default: d.Default} })
}
func (realHardware) Microphones() ([]Device, error) { return audioDevices(koloaudio.Capture) }
func (realHardware) Speakers() ([]Device, error) { return audioDevices(koloaudio.Playback) }kolovideo.Devices and koloaudio.Devices list what is plugged in.
ErrNoDevice and ErrUnsupported are not errors here: a computer without a
camera gets an empty list and can still join a call to listen.
func (realHardware) MicrophoneAccess() (string, error) {
a, err := koloaudio.AccessStatus()
if err != nil {
return "", err
}
switch a {
case koloaudio.AccessNotDetermined:
return micNotDetermined, nil
case koloaudio.AccessRestricted:
return micRestricted, nil
case koloaudio.AccessDenied:
return micDenied, nil
case koloaudio.AccessAuthorized:
return micAuthorized, nil
}
return "", koloaudio.ErrBackend
}
func (realHardware) RequestMicrophone(ctx context.Context) error { return koloaudio.RequestAccess(ctx) }
func (realHardware) RequestCamera(ctx context.Context) error { return kolovideo.RequestAccess(ctx) }
func (realHardware) OpenCamera(id string, w, h int, fps float64) (cameraStream, error) {
c, err := kolovideo.OpenCapture(kolovideo.CaptureConfig{DeviceID: id, Width: w, Height: h, FrameRate: fps})
if err != nil {
return nil, err
}
return c, nil
}
func (realHardware) OpenMicrophone(id string) (micStream, error) {
in, err := koloaudio.OpenInput(koloaudio.Config{DeviceID: id, Format: koloaudio.Format{SampleRate: sampleRate, Channels: audioChannels}})
if err != nil {
return nil, err
}
return in, nil
}
func (realHardware) OpenSpeaker(id string) (speakerStream, error) {
out, err := koloaudio.OpenOutput(koloaudio.Config{DeviceID: id, Format: koloaudio.Format{SampleRate: sampleRate, Channels: audioChannels}})
if err != nil {
return nil, err
}
return out, nil
}OpenCapture and OpenInput never ask for permission. RequestCamera and
RequestMicrophone do: on macOS, kolovideo.RequestAccess and
koloaudio.RequestAccess show the system prompt the first time, and on Linux
and Windows they return at once. MicrophoneAccess only reads the microphone
permission (koloaudio.AccessStatus), so the lobby knows about a refusal
before any device opens. Audio runs at 48 kHz, mono, in 20 ms frames,
kolo-audio's defaults.
// deviceError turns an error of kolo-video or kolo-audio into the MeetError of a device kind ("camera",
// "microphone" or "speaker").
func deviceError(kind string, err error) *MeetError {
var me *MeetError
switch {
case err == nil:
return nil
case errors.As(err, &me):
return me
case errors.Is(err, kolovideo.ErrPermission), errors.Is(err, koloaudio.ErrPermission), errors.Is(err, fs.ErrPermission):
return meetErr(kind+"-denied", err, "")
case errors.Is(err, kolovideo.ErrNoDevice), errors.Is(err, koloaudio.ErrNoDevice),
errors.Is(err, kolovideo.ErrUnsupported), errors.Is(err, koloaudio.ErrUnsupported):
return meetErr(kind+"-none", err, "")
case errors.Is(err, kolovideo.ErrDeviceLost), errors.Is(err, koloaudio.ErrDeviceLost):
return meetErr(kind+"-lost", err, "")
}
return meetErr(kind+"-failed", err, "")
}deviceError gives each failure its own sentence: a refusal is
kolovideo.ErrPermission for the camera, koloaudio.ErrPermission for the
microphone, or an error that matches fs.ErrPermission, so the window can say
where to allow it. A missing device and a lost one get their own sentences too. In case a
system gives silence instead of an error, the app also says so when the
microphone sends nothing but zeros for 4 seconds.
The starter already loads the video codec when it starts (loadCodecs in
meetservice.go): kolo-video's VP8 is libvpx, either as a library embedded in
the module or as the same code translated to Go, and it keeps the library in
the data folder.
Check that the tests still use made-up devices only, then rebuild:
go test ./...
wails3 buildRun the window again, as in step 1. The pickers now list your real devices. On macOS, the first time you turn the camera or the microphone on, the system asks for permission. If it is refused, the window says so, and you can go on with what is left: a call without a camera, or without both, is still a call.
3Start a call: the invite, the relay and your broadcast
The person who starts a call is the host. The host's app runs the call's relay: a kolo-moq relay with an identity of its own, on a network node of its own. Everyone's media goes through it. The invite is served by a second network node, under a second identity: anyone with the invite can reach that node, while the relay's node lets in only the people the host let in. So a connection that asks to join is never one that carries media.
In host.go, replace the import block with:
import (
"bytes"
"crypto/ed25519"
"crypto/rand"
"encoding/hex"
"errors"
"net/netip"
"strconv"
kolodht "github.com/ciranda-dev/kolo-dht"
koloid "github.com/ciranda-dev/kolo-id"
invite "github.com/ciranda-dev/kolo-invite"
kolomoq "github.com/ciranda-dev/kolo-moq"
kolosecure "github.com/ciranda-dev/kolo-secure"
kolostore "github.com/ciranda-dev/kolo-store"
koloswarm "github.com/ciranda-dev/kolo-swarm"
kolotransport "github.com/ciranda-dev/kolo-transport"
)Replace startHost with the four blocks below, pasted one after the other.
func (c *call) startHost() error {
s := c.svc
sw := s.swarm()
if sw == nil {
return meetErr("network-not-ready", ErrNotReady, "")
}
relID, err := koloid.Generate()
if err != nil {
return err
}
roomID, seed := make([]byte, 16), make([]byte, 32)
rand.Read(roomID)
rand.Read(seed)
c.mu.Lock()
c.relID = relID.PublicKey()
c.room = &room{id: hex.EncodeToString(roomID), topic: roomTopic(seed), relay: relID.PublicKey(), host: c.me,
hostName: s.displayName(), max: s.maxParticipants()}
c.mu.Unlock()
set := s.settingsCopy()
payload, err := encodeGrant(c.room.id, seed, c.room.relay, c.me, c.room.hostName, c.room.max, relayKeys(set.relays))
if err != nil {
return err
}A new identity for the relay, a random room ID and a seed. The grant is what each guest receives once the host lets it in: the room, the seed, the relay's key, the host, and the keys of the host's connection relays, so a guest that lists none of them is warned. The seed never travels in the clear: the network sees only the topic made from it.
if c.relay, err = kolomoq.NewRelay(&kolomoq.RelayConfig{Session: kolomoq.Config{Authorize: c.relayAuthorize}, Resume: reconnectWindow}); err != nil {
return err
}
if c.ln, err = kolosecure.Listen(c.ctx, "127.0.0.1:0", relID, &kolosecure.Config{Transport: transportConfig()}); err != nil {
return err
}
c.spawn(func() {
for {
conn, err := c.ln.Accept(c.ctx)
if err != nil {
return
}
if bytes.Equal(conn.RemotePublicKey(), c.me) {
c.spawn(func() { c.relay.NewServer(c.ctx, conn) })
continue
}
c.spawn(func() { c.serveConn(conn, conn.RemotePublicKey()) })
}
})relayAuthorize, already in the starter, is the relay's rule: kolo-moq's
RoomPolicy (only a member may announce a namespace under the room, and only
its own), and a guest without a place in the call is refused at once.
isMember says who the host let in while the call has room. The relay also listens on a
loopback address for the host's own session. The relay keeps the place of a guest
whose connection was lost for reconnectWindow, 60 seconds
(RelayConfig.Resume), so nobody sees the guest leave while it reconnects. The
host's own connection goes straight to the relay; any other goes to
serveConn, which keeps the guest's session under its key, so the host can
remove that guest later.
// The invite is served under a key of its own on a swarm any peer may reach; the call's relay only to the people
// let in, so that a stranger never holds a connection with the stream budget of a person in the call.
invID, err := koloid.Generate()
if err != nil {
return err
}
if c.rsw, err = c.openSwarm(set, swarmCallRelay, relID, c.relayAllow); err != nil {
return err
}
if err := c.rsw.Join(c.ctx, c.room.topic, &koloswarm.JoinOptions{Announce: true}); err != nil {
return meetErr("call-unavailable", err, "")
}
c.spawn(c.relayLoop)The comment says why there are two nodes. A person in the call may open up to 1000 streams on one connection, because every subscription and every group of objects takes a stream, and the transport sets that limit when the connection starts. A stranger who asks to join needs one stream, so the invite's node keeps kolo-transport's default of 16, and a stranger never holds a connection with the budget of a person in the call.
openSwarm, already in the starter, starts a network node under an identity
and connects it to the network: the entry nodes, the connection relays from
-relay, so a guest that cannot reach the node directly reaches it through a
relay both list, and the transport setting from swarmTransport.
relayAllow lets into the relay's node only the people the host let in, and
the ones it barred, so they can be told; never the host's own app. The
relay's node announces the room's topic, so guests find it, and relayLoop
gives the relay every guest that connects.
if c.isw, err = c.openSwarm(set, swarmCallInvite, invID, c.inviteAllow); err != nil {
return err
}
db := kolostore.NewMemory()
c.invDB = db
if c.inv, err = invite.Open(db, invID, nil); err != nil {
return err
}
c.spawn(func() {
if err := c.inv.Announce(c.ctx, c.isw); err != nil && c.ctx.Err() == nil && !errors.Is(err, invite.ErrClosed) {
c.logf("invite announce: %v", err)
}
})
c.spawn(c.inviteSwarmLoop)
c.grant = payload
if _, err := c.newInvite(); err != nil {
return err
}
c.spawn(c.inviteLoop)
if err := sw.Join(c.ctx, c.room.topic, nil); err != nil {
return meetErr("call-unavailable", err, "")
}
conn, err := kolosecure.Dial(c.ctx, c.ln.Addr().String(), s.id, relID.PublicKey(), &kolosecure.Config{Transport: transportConfig()})
if err != nil {
return meetErr("relay-unreachable", err, "")
}
return c.enter(conn)
}The second node runs under the invite's identity and takes anyone but the
host's own app (inviteAllow). kolo-invite keeps its records in kolo-store,
here in memory only, and is announced on that node; inviteSwarmLoop
answers each request to join there. Then the host enters its own call: it
dials the relay's loopback listener and calls enter, like any other
participant.
Replace newInvite with:
// newInvite makes an invite for the call and keeps its code as the one the window shows. The earlier invites keep
// working until they expire or the call ends.
func (c *call) newInvite() (string, error) {
if c.inv == nil {
return "", ErrNoCall
}
infos, err := c.inv.List()
if err != nil {
return "", err
}
active := 0
for _, i := range infos {
if i.State == invite.Active {
active++
}
}
if active >= maxCallInvites {
return "", meetErr("too-many-invites", ErrBusy, strconv.Itoa(maxCallInvites))
}
code, _, err := c.inv.Create(&invite.CreateOptions{TTL: inviteTTL, MaxUses: maxInviteUses, Manual: true})
if err != nil {
return "", err
}
c.mu.Lock()
c.inviteStr = code
c.mu.Unlock()
c.changed()
return code, nil
}An invite lasts 24 hours and is answered by the host's app (Manual), which
checks that the call has room. The invite is a secret: whoever has it can ask
to join. It never goes to the history, the logs, the tray or the notch.
Now the broadcast. In member.go, replace the import block with:
import (
"bytes"
"context"
"crypto/ed25519"
"encoding/hex"
"errors"
"fmt"
"slices"
"sync"
"time"
kolomedia "github.com/ciranda-dev/kolo-media"
kolomoq "github.com/ciranda-dev/kolo-moq"
"github.com/ciranda-dev/kolo-moq/secobj"
kolotransport "github.com/ciranda-dev/kolo-transport"
)Replace enter with:
// enter makes this person a participant: its protected broadcast, its keys, its session to the relay of the call,
// the roster and the media. conn is a connection to the relay.
func (c *call) enter(conn kolomoq.Conn) error {
s := c.svc
var err error
if c.origin, err = kolomoq.NewOrigin(nil); err != nil {
return err
}
ns := c.room.memberNS(c.me)
if c.sender, err = secobj.NewSender(c.origin, ns, nil); err != nil {
return err
}
if c.bc, err = kolomedia.NewSecureBroadcast(c.origin, c.sender); err != nil {
return err
}
if c.keys, err = secobj.NewKeyring(secobj.KeyringConfig{Authorize: c.keyOwner, Relays: c.roomRelays()}); err != nil {
return err
}
c.mu.Lock()
c.pub = newPublisher(c.bc, c.sender, s.vid, &c.others)
c.mu.Unlock()
profile, err := c.origin.CreateTrack(ns, profileTrack, &kolomoq.TrackOptions{Priority: kolomoq.DefaultPriority, CacheGroups: 1})
if err != nil {
return err
}
c.profile = profile
if err := c.attach(conn); err != nil {
return err
}
c.mix = newMixer(nil)
c.spawn(c.superviseRelay)
c.spawn(c.rosterLoop)
c.spawn(c.publishProfiles)
c.spawn(c.expirePendingLoop)
c.spawn(c.runSpeaker)
c.spawn(c.adaptLoop)
c.spawn(c.watchdogLoop)
c.spawn(c.qualityLoop)
mic, _ := s.media.isOpen()
if err := c.pub.setAudio(mic); err != nil {
return err
}
s.media.setSink(&sinks{audio: c.pub.onAudio, video: c.pub.onVideo})
return nil
}enter is where a person becomes a participant, host or guest. The Origin
holds what you publish. The Sender owns your key for your namespace, and
NewSecureBroadcast makes a kolo-media broadcast whose every object, the
catalog included, is sealed with it. The Keyring will hold the keys of the
others (step 5). Your session to the relay lets only the relay read your
broadcast, and PublishNamespace announces it there: once that announcement
comes back, the call is active.
attach is the part of enter that makes the session and announces the
broadcast, a function of its own so the call can run it again. superviseRelay
starts a new session when the old one ends, trying for up to 60 seconds with
waits that double from 0.5 s to 4 s, and keeps your people, devices and mute
state. watchdogLoop does the same when a person who sends delivers nothing for
6 seconds, a path that carries small datagrams and drops large ones. adaptLoop
moves the camera along a quality ladder, 360p, 720p and the 1080p ceiling,
from what the connection to the relay carries: down at once when the queue is
old, up after 4 seconds of a clear path. qualityLoop adds what you send to
the call quality report every 5 seconds (step 7). These helpers are already in
the starter, in reconnect.go, watchdog.go, adapt.go, ladder.go and
quality.go, so you do not write them.
The video track is already in the starter (onVideo in publisher.go). Write
the audio track: in publisher.go, replace setAudio with:
// setAudio adds or removes the microphone track.
func (p *publisher) setAudio(on bool) error {
p.amu.Lock()
defer p.amu.Unlock()
if p.closed || (on == (p.audio != nil)) {
return nil
}
if !on {
err := p.audio.Close()
p.audio = nil
p.aenc.Close()
p.aenc = nil
return err
}
enc, err := opus.NewEncoder(audioEncoderConfig())
if err != nil {
return meetErr("codec-unavailable", err, "")
}
if audioComplexity >= 0 {
enc.SetComplexity(audioComplexity)
}
tp, err := p.b.AddTrack(audioTrackDef(), kolomedia.PackagerOptions{})
if err != nil {
enc.Close()
return err
}
if p.lossPct > 0 {
setLoss(enc, p.lossPct)
}
ts, _ := p.clock.At(time.Now(), sampleRate)
p.audio, p.aenc, p.abase, p.asamples = tp, enc, ts, 0
return nil
}A track exists only while its device is on, so muting removes it, and the
others see "microphone off" from your sealed catalog. The audio is Opus at a
constant 32 kbit/s, so the size of the packets does not tell when you speak.
The microphone's encoder has a lock of its own (amu), so it never waits for a
camera frame being encoded, and the app turns Opus's forward error correction
on (setLoss) while the path loses groups.
Replace onAudio with a constant and the function:
// audioResync is how far the timeline of the microphone's frames may fall behind the clock before it jumps forward to
// it: the playout delay a receiver needs at least (kolo-media's PlayoutConfig.MinDelay), so ordinary jitter of the
// microphone's thread stays inside it.
const audioResync = 100 * time.Millisecond
// onAudio encodes one 20 ms frame of the microphone. Timestamps count samples, so they never jitter, and the timeline
// follows the clock of the video: a microphone that delivers fewer frames than the clock counts (its thread starved, or
// the device overran) moves it forward by the frames lost, a gap the receiver makes up, instead of letting it fall behind
// without end, which the receiver's playout, measuring arrival against these timestamps with the video's, takes for a
// lateness that no delay covers: it drops every audio frame from then on.
func (p *publisher) onAudio(pcm []int16) {
now := time.Now()
p.amu.Lock()
defer p.amu.Unlock()
if p.closed || p.audio == nil {
return
}
out := make([]byte, opus.PacketBufferSize)
n, err := p.aenc.Encode(pcm, out)
if err != nil || n == 0 {
return
}
ts := p.abase + p.asamples
if wall, err := p.clock.At(now, sampleRate); err == nil && wall > ts+uint64(len(pcm))+uint64(audioResync.Seconds()*sampleRate) {
p.abase += (wall - ts - uint64(len(pcm))) / uint64(len(pcm)) * uint64(len(pcm))
ts = p.abase + p.asamples
}
p.asamples += uint64(len(pcm))
if err := p.audio.WriteFrame(kolomedia.Frame{Timestamp: ts, Keyframe: true, Data: out[:n]}); err != nil && !errors.Is(err, kolomedia.ErrClosed) {
return
}
}Each 20 ms frame of the microphone becomes one Opus packet and one frame of
the track. Its timestamp counts samples, so it never jitters. When the
microphone delivers fewer frames than the clock counts, because its thread was
starved or the device overran, the timeline jumps forward to the clock once it
is more than 100 ms behind (audioResync). What was lost is then a gap that the
receiver makes up, and not audio that is dropped for good.
Rebuild and run one window as in step 1. Start a call: the call view says
"1 in call, end-to-end encrypted, through relay run by you", and the host's
view has the invite, which starts with koloinv0. Nobody can join yet.
4Join a call and watch the others
A guest pastes the invite. Its app looks for the host on a topic made from the invite's secret, asks to join, gets the grant sealed for it alone, and then connects to the relay.
In guest.go, replace the import block with:
import (
"context"
"errors"
"time"
invite "github.com/ciranda-dev/kolo-invite"
kolomoq "github.com/ciranda-dev/kolo-moq"
koloswarm "github.com/ciranda-dev/kolo-swarm"
)Replace joinHost with:
func (c *call) joinHost() error {
s := c.svc
sw := s.swarm()
if sw == nil {
return meetErr("network-not-ready", ErrNotReady, "")
}
if err := sw.Join(c.ctx, c.code.Topic(), &koloswarm.JoinOptions{Lookup: true}); err != nil {
return meetErr("lookup-failed", err, "")
}
var peer swarmPeer
select {
case peer = <-c.joinPeer:
case <-time.After(s.set.FindTimeout):
sw.Leave(c.code.Topic())
c.mu.Lock()
unreachable := c.noInviter
c.mu.Unlock()
if unreachable {
return hostUnreachable(len(s.set.relays) > 0)
}
return ErrNotFound
case <-c.ctx.Done():
sw.Leave(c.code.Topic())
return c.ctx.Err()
}
c.setPhase(phaseWaitingHost)
actx, cancel := context.WithTimeout(c.ctx, joinAnswerTimeout)
defer cancel()
payload, err := invite.JoinConn(actx, peer.Conn(), s.id, c.code, nil)
sw.Leave(c.code.Topic()) // the invite-only connection closes; the call gets its own
if err != nil {
return err
}
r, err := parseGrant(payload)
if err != nil {
return err
}
mine := s.settingsCopy().relays
c.mu.Lock()
c.room = r
c.relayShare = shareRelay(mine, r.relayKeys)
c.warning = relayMismatch(mine, r.relayKeys)
c.mu.Unlock()
if !c.move(callConnecting, phaseRelay) {
return context.Canceled
}
if err := sw.Join(c.ctx, r.topic, nil); err != nil {
return meetErr("lookup-failed", err, "")
}
select {
case rp := <-c.relayPeer:
kind, via := s.linkOf(rp.Conn())
c.mu.Lock()
c.relayLink, c.relayVia = kind, via
c.mu.Unlock()
conn := rp.Conn()
if h := relayHook; h != nil {
conn = h(c, conn)
}
return c.enter(conn)
case <-time.After(s.set.FindTimeout):
c.mu.Lock()
defer c.mu.Unlock()
return relayWaitReason(c.noRelay, c.relayShare)
case <-c.ctx.Done():
return c.ctx.Err()
}
}The invite carries a fingerprint of the invite's key and a secret, never an
address. JoinConn asks to join on the host's invite node and waits for the
answer. parseGrant, already in the starter, checks that the answer is a
grant and refuses anything malformed. The invite's key is never the relay's,
so the guest learns the relay's key only from the grant, which was sealed for
it alone. The other helpers are in the starter too:
hostUnreachable says when the host was found but could not be reached,
directly or through the connection relays, and relayMismatch warns a guest
whose connection relays share none with the host's.
In member.go, replace watchLoop with:
// watchLoop plays p's broadcast through the relay. Watch returns once a catalog that opens under p's key arrives,
// so until the key is here the person is "waiting for keys" and nothing of it plays.
func (c *call) watchLoop(p *participant) {
caps := watchCapabilities()
for p.ctx.Err() == nil {
w, err := kolomedia.Watch(p.ctx, c.session(), p.ns, kolomedia.WatchOptions{Capabilities: caps, Keys: c.keys, RequestKeyframes: !noKeyRequests})
if err != nil {
c.mu.Lock()
p.watchErr = err
c.mu.Unlock()
if !sleep(p.ctx, retryEvery) {
return
}
continue
}
c.setPState(p, partLive)
c.playWatcher(p, w)
w.Close()
if p.ctx.Err() != nil {
return
}
c.mu.Lock()
p.audioOn, p.videoOn = false, false
c.mu.Unlock()
c.setPState(p, partStopped)
if !sleep(p.ctx, retryEvery) {
return
}
c.setPState(p, partWaitingKeys)
}
}kolomedia.Watch with Keys returns only once a catalog arrives that opens
under the sender's key. Until then the person is "waiting for keys" and
nothing of them plays. It runs on c.session(), the current session to the
relay, so after a reconnect the watch starts again on the new one.
RequestKeyframes makes a watch ask the sender for a new group, which starts
with a key frame, when its video track waits for one, at most once a second,
instead of waiting for the next group (up to 2 s). The sender serves it with
TrackPublisher.KeyframeRequests.
In remote.go, replace the import block with:
import (
"context"
"fmt"
"math"
"sync"
"time"
"github.com/ciranda-dev/kolo-audio/opus"
kolomedia "github.com/ciranda-dev/kolo-media"
"github.com/ciranda-dev/kolo-video/vpx"
)Replace playAudio and playVideo with:
func (c *call) playAudio(ctx context.Context, p *participant, w *kolomedia.Watcher) error {
dec, err := opus.NewDecoder(sampleRate, audioChannels)
if err != nil {
return err
}
defer dec.Close()
pcm := make([]int16, int(opus.MaxPacketDuration.Seconds()*sampleRate)*audioChannels)
var speak speakTracker
var g gaps
for {
f, _, err := w.Next(ctx, kolomedia.Audio)
if err != nil {
return err
}
if miss := g.missing(f.Timestamp); miss > 0 && miss <= maxGapFrames {
p.missed.Add(uint64(miss))
if !noConceal {
c.concealGap(p, dec, f.Data, pcm, miss)
}
}
n, err := dec.Decode(f.Data, pcm)
if err != nil || n == 0 {
continue
}
p.audioBytes.Add(uint64(len(f.Data)))
db := pcmLevel(pcm[:n])
now := time.Now()
p.level.Store(math.Float64bits(db))
p.speaking.Store(speak.update(db, now))
p.lastAudio.Store(now.UnixNano())
p.audioIn.Add(1)
c.mix.push(p.id, pcm[:n])
}
}func (c *call) playVideo(ctx context.Context, p *participant, w *kolomedia.Watcher) error {
dec, err := vpx.NewDecoder(remoteDecoderConfig())
if err != nil {
return err
}
defer dec.Close()
tile := tileRenderer{b: &c.svc.tiles, id: p.id}
defer c.svc.tiles.release(p.id)
for {
f, _, err := w.Next(ctx, kolomedia.Video)
if err != nil {
return err
}
began := time.Now()
img, err := dec.Decode(vpx.Packet{Data: f.Data, Keyframe: f.Keyframe})
if err != nil || img == nil {
continue
}
now := time.Now()
p.noteDecode(now.Sub(began))
p.videoIn.Add(1)
p.videoBytes.Add(uint64(len(f.Data)))
p.videoSize.Store(uint64(img.Width)<<32 | uint64(img.Height))
if ev, ok := tile.render(img, now); ok {
ev.ID = p.id
c.svc.emit("meet:frame", ev)
p.videoShown.Add(1)
p.freeze.picture(now)
}
}
}Audio goes to the mixer, which plays everyone on your speaker; the loudness
drives the speaking ring. Video is decoded and drawn into a small JPEG for
the window's tile. A frame that does not decode is skipped until the next key
frame. Frames that never arrived are counted (gaps), and up to 3 in a row are
made up: Opus loss concealment, and for the last one the forward error
correction in the next packet. Each frame also adds to the counters of the
call quality report (step 7): the bytes received, the size of the picture,
the time it took to decode, the pictures shown, and freeze, which notices
when pictures stop.
Rebuild. In Terminal 1, start the host as in step 1. In Terminal 2, start a second window that joins the network of the first:
./bin/kolo-meet -data "$(mktemp -d)/bruno" -bootstrap 127.0.0.1:4000$dir = New-Item -ItemType Directory -Path (Join-Path $env:TEMP "kolo-meet-$(Get-Random)")
.\bin\kolo-meet.exe -data (Join-Path $dir.FullName "bruno") -bootstrap 127.0.0.1:4000./bin/kolo-meet -data "$(mktemp -d)/bruno" -bootstrap 127.0.0.1:4000Start a call in the first window and copy the invite. In the second, click Join with an invite, paste it, click Continue, then Join call. Each window now has a tile for the other person, the tile says "Waiting for keys", and the header counts "1 waiting for keys": no picture, no sound. Even the names are missing: the tiles show "Guest" and the start of the other person's key, because the names travel sealed too. That is by design. The media reaches the relay sealed, and nobody has given anyone a key yet.
5Group keys
Each participant seals its broadcast with its own key and gives that key to each other participant over the connection between the two, direct or through a connection relay both list, never through the call's relay. When someone joins or leaves, the sender's key changes, so a person who left reads nothing sealed after they left.
In member.go, replace keyOwner with:
// keyOwner is the Keyring's rule: a peer gives the key of its own namespace, and only while it is in the call.
func (c *call) keyOwner(peer ed25519.PublicKey, ns kolomoq.Namespace) error {
if !c.room.owns(peer, ns) {
return errors.New("kolo-meet: not the owner of that namespace")
}
c.mu.Lock()
defer c.mu.Unlock()
if c.people[hex.EncodeToString(peer)] == nil {
return errors.New("kolo-meet: not in the call")
}
return nil
}This is the Keyring's rule: a peer may give the key of its own namespace,
and only while it is in the call.
Replace updateMembers with:
// updateMembers gives our key to everyone in the call, rotating it when the set changed.
func (c *call) updateMembers() {
if c.sender == nil {
return
}
c.memberMu.Lock()
defer c.memberMu.Unlock()
c.mu.Lock()
keys := make([]ed25519.PublicKey, 0, len(c.people))
for _, p := range c.people {
keys = append(keys, p.key)
}
c.mu.Unlock()
c.others.Store(int32(len(keys)))
if err := c.sender.SetMembers(keys); err != nil && c.ctx.Err() == nil {
c.logf("set members: %v", err)
}
}SetMembers sets who receives your key. When the set changes, the Sender
makes a new key, sends it to every member's key track before it uses it, and
closes the track of someone who left. The window then shows "NAME no longer
receives your media".
Replace startLink with:
// startLink makes the media session between us and p on their own connection, and fetches p's key on it.
func (c *call) startLink(p *participant, peer swarmPeer) {
c.spawn(func() {
lctx, cancel := context.WithCancel(p.ctx)
cfg := &kolomoq.Config{Origin: c.origin, Authorize: c.pairAuthorize, SetupTimeout: pendingTTL}
var sess *kolomoq.Session
var err error
if bytes.Compare(c.me, p.key) < 0 {
sess, err = kolomoq.NewClient(lctx, peer.Conn(), cfg)
} else {
sess, err = kolomoq.NewServer(lctx, peer.Conn(), cfg)
}
if err != nil {
cancel()
return
}
l := &link{peer: peer, sess: sess, cancel: cancel}
l.kind, l.via = c.svc.linkOf(peer.Conn())
if old := c.setLink(p, l, nil); old != nil {
old.stop()
}
if c.svc.holdKeys != nil && c.svc.holdKeys(p.key) {
return
}
for {
err := c.keys.Receive(lctx, sess, c.me, p.ns)
c.mu.Lock()
p.keyErr, p.keyTries = err, p.keyTries+1
c.mu.Unlock()
select {
case <-lctx.Done():
return
case <-sess.Done():
return
case <-time.After(retryEvery):
}
}
})
}The connection between two participants carries one kolo-moq session.
pairAuthorize, already in the starter, refuses every request on it except
the key track made for the asker, so no media is served there. Receive
installs the other person's keys as they come; holdKeys is a hook the tests
use to play someone whose key never arrives, and it is nil in the app.
Rebuild and run the two windows of step 4 again. Within about a second the tiles change from "Waiting for keys" to the other person's name, picture and sound, and the header reads "2 in call, end-to-end encrypted, through relay run by you" in the host's window.
What this protects, as the window says it when you open the header: the relay cannot see audio, video or names, not even how long a name is. The relay sees who is in the call, who watches whom, when each person sends and how much, when microphones and cameras are off, and the room ID; it can drop or delay media. Anyone in the call can read everyone's media. Anyone in the call could, with the relay's help, make audio, video or names that seem to come from someone else, and the host, who runs the relay, could do it alone: every participant holds every sender's key, and objects are not signed per sender. There is no protection for the current key period if a device is compromised, and no recovery after a compromise.
A participant whose key has not arrived plays nothing: there is no playback
without keys. The keys travel only over the connection between two
participants, never through the call's relay. That connection is direct, or
goes through a connection relay that both list with -relay. A pair that
cannot connect either way keeps showing "Waiting for keys". Go's AES-GCM
is constant time only on amd64 and arm64 processors with AES instructions,
ppc64x and s390x.
6Run three copies on one computer
Every stub is written now, so the starter's own files are not needed. Delete
starter.go and starter_test.go: nothing uses them any more. Then check
your code with the finished example's own tests. They start two to four copies of
the service in one process, with made-up devices, on 127.0.0.1, and check
that everyone sees and hears everyone, mute, camera off, leaving, a full call
and a participant whose key never arrives. They take a minute or two:
go vet -tags finished .
go test -tags finished .
wails3 buildStart three windows. Terminal 1 is the host and runs the first entry node; Terminal 2 and Terminal 3 join its network:
./bin/kolo-meet -data "$(mktemp -d)/ana" -first 127.0.0.1:4000
./bin/kolo-meet -data "$(mktemp -d)/bruno" -bootstrap 127.0.0.1:4000
./bin/kolo-meet -data "$(mktemp -d)/carla" -bootstrap 127.0.0.1:4000$dir = New-Item -ItemType Directory -Path (Join-Path $env:TEMP "kolo-meet-$(Get-Random)")
.\bin\kolo-meet.exe -data (Join-Path $dir.FullName "ana") -first 127.0.0.1:4000
.\bin\kolo-meet.exe -data (Join-Path $dir.FullName "bruno") -bootstrap 127.0.0.1:4000
.\bin\kolo-meet.exe -data (Join-Path $dir.FullName "carla") -bootstrap 127.0.0.1:4000./bin/kolo-meet -data "$(mktemp -d)/ana" -first 127.0.0.1:4000
./bin/kolo-meet -data "$(mktemp -d)/bruno" -bootstrap 127.0.0.1:4000
./bin/kolo-meet -data "$(mktemp -d)/carla" -bootstrap 127.0.0.1:4000Run each line in its own terminal (on Windows, run the first line in each
terminal before its own kolo-meet line). Start a call in Ana's window and
send the invite to the other two: the same invite works for both. The grid
shows three people, with names and a ring around whoever speaks. All three
copies share your one camera and microphone, so use headphones or mute two of
them, or the speakers feed back into the microphone.
Try mute and camera off: the tile changes for the others. When Bruno clicks Leave, the others see "Bruno no longer receives your media", and the keys change. When Ana, the host, ends the call, it ends for everyone and says why: the relay was hers.
7See how the call is doing
While a call runs, each window writes a call quality report: every 5
seconds, one JSON line for each person you watch ("kind":"remote") and one
for what you send ("kind":"local"). The file is kolo-meet-quality.jsonl
in the data folder, readable only by you (mode 0600); at 2 MiB it becomes
kolo-meet-quality.jsonl.1 and a new one starts. It holds counts, sizes and
times only, never a name, key, invite, address or media. When a window
starts, its terminal shows where the report goes, in a line that begins with
call quality report is on. -quality-report off turns it off, and
-quality-log also prints each line to the terminal, after quality:.
You write nothing in this step: the report is in quality.go, and you added
its counters in steps 3 and 4. Here is where its numbers come from. For each
person you watch, logMedia, already in the starter, calls reportRemote
every 5 seconds:
// reportRemote writes the quality record of p for the interval since q0 and returns the counters it read.
func (c *call) reportRemote(p *participant, st kolomedia.WatchStats, q0 peerCounters, secs float64, now time.Time) peerCounters {
q := p.counters(st)
if !c.svc.qualityOn() {
return q
}
r := remoteRecord(q0, q, secs)
r.At, r.T, r.Call, r.Person = now.Unix(), round1(now.Sub(c.began).Seconds()), c.id, p.person
size := p.videoSize.Load()
r.VideoW, r.VideoH = int(size>>32), int(size&0xffffffff)
n, longest := p.freeze.take(now)
r.Freezes, r.FreezeLongestMs = n, longest.Milliseconds()
r.PlayoutDelayMs, r.PlayoutJitterMs = st.Playout.Delay.Milliseconds(), st.Playout.Jitter.Milliseconds()
r.DecodeMs = ms(time.Duration(p.decodeNs.Load()))
rate, cost := c.svc.tiles.display(p.id)
r.DisplayFPS, r.TileMs = rate, ms(cost)
c.svc.reportQuality(r)
return q
}
// counters reads the totals of p's watch for the quality record: the playout's drops come from st.
func (p *participant) counters(st kolomedia.WatchStats) peerCounters {
dropped := func(t kolomedia.TrackStats) uint64 { return t.Late + t.Skipped + t.Overflow }
return peerCounters{
audioIn: p.audioIn.Load(), missed: p.missed.Load(), concealed: p.concealed.Load(), audioDropped: dropped(st.Playout.Tracks["audio"]),
videoIn: p.videoIn.Load(), shown: p.videoShown.Load(), videoDropped: dropped(st.Playout.Tracks["video"]),
audioBytes: p.audioBytes.Load(), videoBytes: p.videoBytes.Load(),
}
}Watcher.Stats of kolo-media tells what the watch of one person saw: the
delay of the jitter buffer and how late frames arrive (Playout.Delay and
Playout.Jitter), and for each track the frames it dropped because they came
too late, were skipped, or did not fit in the buffer. The counters you added
in playAudio and playVideo give the rest. A freeze is a stop of more than
500 ms between two pictures while the person's camera is on.
For what you send, qualityLoop reads Stats of each of your sessions to the
relay (a host has one for each guest) and adds them up:
// sentStats is what the sessions of the sender showed at one reading, by session: counters restart with a session.
type sentStats map[*kolomoq.Session]kolomoq.SessionStats
// sinceLast adds up what each session carried since prev, and the oldest queue among them.
func (cur sentStats) sinceLast(prev sentStats) (sent, reset, expired uint64, queue time.Duration) {
for s, st := range cur {
p := prev[s]
sent += sub(st.SentBytes, p.SentBytes)
reset += sub(st.ResetGroups, p.ResetGroups)
expired += sub(st.ExpiredGroups, p.ExpiredGroups)
queue = max(queue, st.QueueDelay)
}
return sent, reset, expired, queue
}
// net is the transport's view of the sessions that report it, since prev; nil when none does. A session of a connection
// without statistics, such as one through a connection relay, has an empty controller name and is left out.
func (cur sentStats) net(prev sentStats) *qualityNet {
var worst *kolomoq.SessionStats
var sent, lost uint64
for s, st := range cur {
if st.Transport.CongestionController == "" {
continue
}
p := prev[s].Transport
sent += sub(st.Transport.PacketsSent, p.PacketsSent)
lost += sub(st.Transport.PacketsLost, p.PacketsLost)
if worst == nil || st.Transport.SmoothedRTT > worst.Transport.SmoothedRTT {
worst = &st
}
}
if worst == nil {
return nil
}
t := worst.Transport
n := &qualityNet{
RTTMs: ms(t.SmoothedRTT), MinRTTMs: ms(t.MinRTT), RTTVarMs: ms(t.RTTVariation),
CwndBytes: t.CongestionWindow, InFlightByte: t.BytesInFlight, Controller: t.CongestionController,
BwKbps: round1(float64(t.BandwidthEstimate) * 8 / 1000), PathChanges: t.PathChanges,
}
if sent > 0 {
n.LossPct = round1(float64(min(lost, sent)) * 100 / float64(sent))
}
return n
}kolo-moq's SessionStats has the bytes sent, the groups of objects given up
(reset because the queue was full, or expired because they were too old), and
how long the oldest unsent object waits: over 250 ms is a full path. Its
Transport part is what kolo-transport measured on the connection: the
round-trip time, the packets lost, the congestion window and the name of the
congestion controller. The net object of the local line keeps the session
with the highest round-trip time. A session through a connection relay has
no such numbers, so it is left out.
The controller is reno unless you start the app with -congestion cubic or
-congestion bbr. A call sends less than the path carries, and bbr keeps a
queue of 25 to 150 ms, so reno is the default; the other two are there to
compare on a lossy path, with this report.
8Choose how many pictures per second
The window does not decode video: the Go service decodes each person's frames
and draws each picture into a small JPEG for its tile (tile.render in
playVideo). Drawing takes processor time, so the app chooses how many
pictures a second each tile draws. -display-fps sets it: auto, the
default, 30 or 15. You write nothing in this step either: the rule is in
tiles.go.
// The display rates, in pictures a second of one tile, and the rule that picks one.
const (
// tileRateHigh is the rate while at most tileFewRemote other people show video: kolo-video's default capture rate,
// so every picture sent is shown.
tileRateHigh = kolovideo.DefaultFrameRate
// tileRateLow is the rate with more people, and while the guard finds the drawing too costly: half of it.
tileRateLow = kolovideo.DefaultFrameRate / 2
tileFewRemote = 4
// tileCPUMax is the share of one core the JPEG work of all tiles may take at tileRateHigh before the rate falls to
// tileRateLow; it goes back up when that work is under tileCPUResume, so a load near the limit does not flip the rate.
tileCPUMax = 0.5
tileCPUResume = 0.4
// tileCostWeight is the weight of each new picture in a tile's mean cost, and tileCostWarm the pictures a tile
// needs before its cost counts.
tileCostWeight = 0.1
tileCostWarm = 5
// tileEarly is how long before its turn a frame is still taken: camera frames are not evenly spaced, and at
// tileRateHigh a frame that came a millisecond early would cost a whole interval.
tileEarly = 8 * time.Millisecond
// tileByteRateLow bounds the bytes a second of picture events over all tiles at tileRateLow. Wails parks an event
// over 8 KiB until the webview fetches it, at most 64 MiB in all, each for 30 s plus up to a sweep of 7.5 s, and
// sends what does not fit inline, which keeps memory on macOS (wails v3.0.0 pkg/application/event_payload_store.go:33-57,
// webview_window.go:1459-1475). At this rate even a webview that fetches nothing never fills the store; the budget
// grows with the rate (tileByteRate), so at tileRateHigh such a webview fills it in half the time.
tileByteRateLow = (64 << 20) * 4 / (30 * 5)
// selfTile is the ID of the tile of our own camera.
selfTile = "self"
)
// -display-fps values. Auto is the rule above; the other two fix the rate and switch the guard off.
const (
displayAuto = "auto"
display30 = "30"
display15 = "15"
)// rateLocked is the rate tiles draw at now: the fixed one of -display-fps 30 or 15, else tileRateHigh with at most
// tileFewRemote other people showing video, unless the guard holds it at tileRateLow, else tileRateLow.
func (b *tileBudget) rateLocked() int {
switch b.mode {
case display30:
return tileRateHigh
case display15:
return tileRateLow
}
remote := len(b.live)
if _, ok := b.live[selfTile]; ok {
remote--
}
if remote > tileFewRemote {
return tileRateLow
}
if noTileGuard {
return tileRateHigh
}
var work float64
warm := true
for _, t := range b.live {
work += t.cost.Seconds() * tileRateHigh
warm = warm && t.n >= tileCostWarm
}
switch {
case !warm:
case b.guarded && work < tileCPUResume:
b.guarded = false
case !b.guarded && work > tileCPUMax:
b.guarded = true
}
if b.guarded {
return tileRateLow
}
return tileRateHigh
}With auto, each tile draws 30 pictures a second, kolo-video's default
capture rate, so every picture sent is shown, while at most 4 other people
show video, and 15 with more. A guard keeps a running mean of how long one
picture takes to draw and compress, for each tile. When drawing all tiles at
30 would take more than half of one processor core, they draw at 15 until it
falls under 40%, so a load near the limit does not make the rate jump back
and forth. 30 and 15 fix the rate and turn the guard off.
To try it, start one more window as in step 6, with a data folder of its own
and -display-fps 15 at the end of the line, and join Ana's call with the
same invite. Every tile of that window, its own camera's too, draws 15
pictures a second, and its report says so: display_mode is 15 in the local lines, and the remote
lines have display_fps 15 next to tile_ms, the time one picture takes to
draw.
What just happened
- Each window found the others through the entry node that the first one ran. The host's app ran the call's relay, and every broadcast went through it. The host's upload carries every participant's media to every other one.
- The invite led the guests to the host's invite node, and the host's app let them in while the call had room. Only then did they learn the relay's identity, from the grant.
- Every object of every broadcast was sealed with its sender's key, and the keys went over the direct connection between each pair, never through the relay. See kolo-media and kolo-moq.
Next steps
- Read the kolo-media page: broadcasts, watching and protected media, with their limits.
- Read how an invite works in Invite someone to a group, in a terminal program.
- The finished code of this example is in the kolo-meet-wails3-example repository
of the ciranda-dev organization. Without
starter.go, your Go files are the same.