The kolo-pin command serves metrics in the Prometheus text format. They
tell you how full the pinning peer is, how busy it is and why it refuses
requests. This page starts from the peer of
Run a pinning peer on a VPS.
The metrics endpoint
kolo-pin run answers GET /metrics on 127.0.0.1:7452 by default. The
flag is -metrics and its environment variable is KOLO_PIN_METRICS; a
flag wins over its variable. Set it to off to turn the endpoint off. It
is a separate listener from the health endpoint, GET /healthz on
127.0.0.1:7451.
The address is a TCP address, IPv4 only. An IPv6 address is refused and kolo-pin exits with code 2. At start, kolo-pin logs where it listens:
time=2026-11-12T08:16:33.990+07:00 level=INFO msg=metrics addr=127.0.0.1:7452On the server, ask it for the main numbers:
curl -s http://127.0.0.1:7452/metrics | grep -E '^kolo_pin_(bytes|disk|owners|logs|entries|requests)'This is what it printed right after one owner pinned a log of four entries:
kolo_pin_bytes_max 1e+11
kolo_pin_bytes_received_total 2040
kolo_pin_bytes_served_total 0
kolo_pin_bytes_used 3081
kolo_pin_disk_free_bytes 1.73485133824e+11
kolo_pin_entries 4
kolo_pin_logs 1
kolo_pin_owners 1
kolo_pin_requests_total{op="add"} 1
kolo_pin_requests_total{op="remove"} 0
kolo_pin_requests_total{op="revoke"} 0
kolo_pin_requests_total{op="status"} 1
kolo_pin_requests_total{op="sync_pull"} 0The storage and request metrics appear once the store is open, around the
serving log line. Counters start again at 0 each time kolo-pin starts.
A Go program that runs the library itself gets the same kolo_pin_*
metrics by passing a prometheus.Registerer in Options.Metrics, and
serves them as it likes.
The metrics
kolo_pin_bytes_used(gauge): bytes stored for all owners, keys and values.kolo_pin_bytes_max(gauge): the-max-byteslimit.kolo_pin_disk_free_bytes(gauge): free bytes on the filesystem of the data folder, on Linux, macOS and Windows only.kolo_pin_owners(gauge): owners with a capability in use or a hosted log.kolo_pin_logs(gauge): hosted logs.kolo_pin_entries(gauge): entries of all hosted logs.kolo_pin_requests_total(counter, labelop:add,remove,status,revoke,sync_pull): requests answered;sync_pullcounts the kolo-sync streams other peers open on a hosted log: a new reader opens two (header, then pull), andFetchHeadandFetchopen one each.kolo_pin_refused_total(counter, labelreason:bad_token,expired_token,replayed_token,revoked,quota_exceeded,disk_full,oversized_entry,bad_signature,bad_proof,other): requests refused.kolo_pin_delete_markers_applied_total(counter): owner delete markers applied, each dropping one ciphertext the peer held. An entry that arrives already deleted is not counted.kolo_pin_bytes_received_total(counter): bytes of new entries (records and ciphertexts) pulled from owners and stored.kolo_pin_bytes_served_total(counter): bytes kolo-sync wrote to peers pulling hosted logs, every frame included.kolo_pin_active_connections(gauge): open kolo-secure connections.kolo_pin_active_sync_streams(gauge): kolo-sync streams running now, those served to pulling peers plus the pulls from owners.kolo_pin_active_pulls(gauge): of those, the pulls from owners running now.kolo_pin_sync_pull_duration_seconds(histogram): how long each pull from an owner takes.kolo_pin_request_duration_seconds(histogram, labelop:add,remove,status,revoke): how long each pin request takes. kolo-sync streams (sync_pull) are not timed, so that label is not here.kolo_pin_build_info(gauge, labelsversion,go_version): always 1; the labels name the build.kolo_pin_uptime_seconds(gauge): seconds since the program started.
Both histograms use buckets of 0.01, 0.05, 0.1, 0.5, 1, 5, 10, 30, 60 and
300 seconds. Every op and reason value is there from the start, at 0,
so an alert sees the first refusal. The command also exports the standard
go_* runtime metrics, and process_* metrics on Linux and Windows.
What the metrics reveal
They show totals for the whole peer: bytes, owners, logs, entries, request and refusal counts by kind, durations, and the build.
They never hold an owner key, a log ID, a peer key, an IP address or any number for one owner. Labels take only the fixed values listed above, so nothing a peer sends ever becomes a label.
Totals still say something. Anyone who reads the endpoint learns how many
owners and logs you host, how much you store, and how often and why
requests are refused. Keep it private. The logs of kolo-pin hold more
(public keys, log IDs, addresses), as
Run a pinning peer on a VPS explains.
Scrape it with Prometheus
If Prometheus runs on the VPS, point it at the loopback address. This
prometheus.yml scrapes kolo-pin every 30 seconds and loads the alert
rules further down:
rule_files:
- kolo-pin-rules.yml
scrape_configs:
- job_name: kolo-pin
scrape_interval: 30s
static_configs:
- targets: ["127.0.0.1:7452"]If Prometheus runs on your own computer, open an SSH tunnel to the VPS instead of opening a port:
ssh -L 7452:127.0.0.1:7452 user@203.0.113.10While the tunnel is open, the same config works on your computer, since
127.0.0.1:7452 there now leads to the VPS. When the tunnel closes, the
scrapes fail and the last alert below fires.
Set it in systemd
The default already listens on 127.0.0.1:7452. To make it explicit, or
to change it, add this line to the [Service] section of the unit from
the VPS guide, next to the other Environment= lines:
[Service]
Environment=KOLO_PIN_METRICS=127.0.0.1:7452Use off as the value to turn the endpoint off. Then reload and restart:
sudo systemctl daemon-reload
sudo systemctl restart kolo-pinjournalctl -u kolo-pin shows the metrics line with the address.
Alert rules
Three rules in Prometheus format, for the file kolo-pin-rules.yml that
the config above loads:
groups:
- name: kolo-pin
rules:
- alert: KoloPinStorageAlmostFull
expr: kolo_pin_bytes_used / kolo_pin_bytes_max > 0.9
for: 15m
labels:
severity: warning
annotations:
summary: "kolo-pin on {{ $labels.instance }} uses over 90% of -max-bytes"
- alert: KoloPinRefusingDiskFull
expr: increase(kolo_pin_refused_total{reason="disk_full"}[10m]) > 0
labels:
severity: critical
annotations:
summary: "kolo-pin on {{ $labels.instance }} refuses new data: disk full"
- alert: KoloPinNoMetrics
expr: up{job="kolo-pin"} == 0 or absent(kolo_pin_bytes_used{job="kolo-pin"})
for: 5m
labels:
severity: critical
annotations:
summary: "No kolo-pin metrics for 5 minutes"- The first fires when the peer has used 90% of
-max-bytesfor 15 minutes. Raise-max-bytes(if the disk has room), or list your owners. - The second fires when
Addreturnedpin: pinning peer disk full(ErrDiskFull) at least once in the last 10 minutes: owners can no longer add new data. - The third fires when no scrape has worked for 5 minutes.
up == 0covers a stopped peer or a closed tunnel, andabsent(...)covers a target that is missing or answers without the kolo-pin metrics.
-max-bytes counts stored keys and values, not disk blocks, so the real
disk can fill first. Watch kolo_pin_disk_free_bytes as well.
Keep it off the public address
kolo-pin never listens for metrics on a public address unless you give it one. The endpoint has no authentication and no TLS.
What was not tested
Every run behind this page was on one computer, on 127.0.0.1: the build,
kolo-pin run with the default addresses, curl of /healthz and
/metrics before and after the owner program of the VPS guide pinned a
log, -metrics off and the IPv6 refusal. Prometheus, the scrape config
and the alert rules (only checked to be valid YAML), the SSH tunnel, the
systemd unit, a reverse proxy and a Linux server were not tested.
Next steps
- Set up the peer itself: Run a pinning peer on a VPS.
- Read the kolo-pin page: its API, its limits and its errors.
- See every default in Limits.