API keys and tokens

Last updated 2 November 2026

Give your program a Tonbo API key before it reaches a database or a stream, and narrow it to what the program uses. A request carries the key itself, or a short-lived access token exchanged from it.

In Node.js, open() from @tonbo/sqlite-ursula/tonbo takes the key, looks the database up by name and keeps a fresh token in the background. A host that does not run it, or that holds its token in a file, makes the exchange itself. In Python, sqlite_ursula.open() takes the database’s URL and a key that acts as you as its token, sent directly; a key that acts as an Agent is exchanged first, and the token goes in its place (see Connect from Python). A database’s URL is on its region’s database origin (see Regions and URLs), such as https://db.tonbo.dev. This page describes where open() finds the key, sending a key directly, the exchange, tokens for a web page, what a key can reach, and a reference refresher.

Opening by name

open("notes") takes the key from the first of these that is set, and tries no other:

1. the apiKey option;
2. TONBO_API_KEY;
3. the login tonbo login saves in ${XDG_CONFIG_HOME:-~/.config}/tonbo/credentials.json. The file must be a regular file with mode 0600. A symlink, or a file open to group or others, is refused and never read.

An explicit token option, or URSULA_VFS_TOKEN_FILE, serves the database requests instead of the key. Use one key per process: the process holds one key for its access tokens, so every database it opens is reached as that key’s account, and an open by name with a second key fails.

resolveDatabase("notes"), from the same entry, finds the key in the same order and fails with the same errors, without opening the database. It returns { url, token } for another opener, such as Pi’s openUrsulaPiStorage: the extension keeps the key’s access tokens fresh, and token is undefined while they serve, so pass it on as it is.

A failed open rejects with TonboError. Its kind says why, and its message ends with the fix:

A database still being created is waited for. When the Management API is rate limited, down or unreachable, the open retries until its signal fires, then rejects with UrsulaUnavailableError and the kind rate_limited, unavailable or unreachable. Every open by name looks the name up through the Management API, so while that API does not answer, open the database by its URL (see Open by URL), which still works.

Sending the key directly

A key that acts as you is accepted as Authorization: Bearer at the Streams and database origins of every region, with no exchange. It reaches exactly what a token exchanged from it reaches, narrowing included (see What a key can reach).

• A key that acts as an Agent is never accepted directly. It answers 401 with application/problem+json whose code is exchange_required and whose token_endpoint names where to exchange it (see below).
• Revoking the key stops it within 60 seconds, usually within a few seconds.
• When Tonbo cannot check a key for a moment, the request answers 503 with Retry-After; send it again after that long. A token needs no such check.

The token exchange

Exchange the key when the credential your program sends should expire on its own, such as one a host keeps in a file, or when the key acts as an Agent. Send the key as a bearer token to POST https://tonbo.dev/api/iam/token, with the Streams audience, the name of the API the token is for (see below):

curl -sS -X POST "https://tonbo.dev/api/iam/token" \
  -H "Authorization: Bearer $TONBO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"audience":"https://streams.tonbo.dev"}'

The key may instead be sent as api_key in the JSON body. A successful answer is JSON:

The audience is required. Without it the exchange returns a token for another Tonbo API, which Streams refuses. An unknown, revoked or expired key answers 401 with {"error":"Invalid API key."}.

The exchange counts 600 tokens a minute for each key, exchanges and tokens for a web page together, and 3000 requests a minute from each address. Past either, it answers 429 with the code rate_limited and Retry-After: 60. A program that sends its key directly makes no exchange at all.

The audience is https://streams.tonbo.dev, the name of the Tonbo Streams API, and not the origin of your database URL. A token for that audience works at the database origin of every region: https://db.tonbo.dev, https://streams.ap-southeast-1.tonbo.dev. In us-east-1, Streams also answers at https://streams.tonbo.dev beside https://db.tonbo.dev. Both names serve the same data and accept the same tokens, and the audience stays https://streams.tonbo.dev on both, so a host that moves a database from one name to the other keeps its refresher and its key.

Tokens for a web page

Your server can mint a token narrower and shorter-lived than its own key or token, to hand to one client such as a web page, which then calls the stream directly (see A token for a web page). Send JSON to the same POST https://tonbo.dev/api/iam/token, with no Authorization header:

PAGE_TOKEN=$(jq -n --arg key "$TONBO_API_KEY" '{
    grant_type: "urn:ietf:params:oauth:grant-type:token-exchange",
    subject_token: $key,
    subject_token_type: "urn:tonbo:params:oauth:token-type:api_key",
    streams: ["events"],
    read_only: true,
    expires_in: 300
  }' | curl -sS -X POST "https://tonbo.dev/api/iam/token" \
  -H "Content-Type: application/json" -d @- | jq -r .access_token)

Any other field is refused. The answer has the fields of an exchange’s, with the subject’s sub and client_id, and scope listing what the token reaches. The token is never wider than its subject and lives at most 600 seconds; revoking the key it came from stops it with the key’s other tokens. A refusal is {"error": …, "code": …}:

A token minted from a key reaches the named streams only in the key’s account, the account_id of the answer. Each request counts toward the exchange’s limits (see above).

What a key can reach

A key belongs to one account and acts as one of its members, with that member’s current role: you, or an Agent of the account, an identity with its own keys and the role you give it, created on Members. Its tokens reach databases and streams as follows:

• A key that is not narrowed reads and writes every database and stream its member can reach in the key’s account. A key reaches only the account you created it in, even when you also belong to others: a database or stream of another account answers as if it did not exist, so a program that works in two accounts holds a key of each.
• A key narrowed to databases, which you choose under Databases and Name prefixes when you create it (--database and --database-prefix in the CLI), reads and writes only the databases it names and those whose name starts with one of its prefixes, for its whole life.
• A key narrowed to streams, which you choose under Streams and Stream prefixes (--stream and --stream-prefix), reaches only the streams it names and those whose name starts with one of its prefixes, by the names tonbo stream list shows, for its whole life. A key narrowed to both databases and streams reaches both; a key narrowed to one reaches nothing of the other.
• A narrowed key neither manages nor calls Machines: a Machine’s private port takes a key that is not narrowed (see Private ports).
• A narrowed key can also be read-only (--read-only): it then reads and tails, and never writes or appends. Through the Management API a narrowed key only reads: the account, and the databases and streams it reaches, which is how an application opens a database by name with it. Apart from revoking itself, it never creates, changes or deletes anything.
• A key that acts as you and is not narrowed can also call the Tonbo Management API at https://api.tonbo.dev with whatever your role allows in the account, such as creating and deleting databases.
• Revoking a key stops new exchanges at once, and the key itself, sent directly, within 60 seconds. Streams in every region learns of the revocation shortly after and refuses the tokens already exchanged from it, which in any case stop working when they expire: up to 10 minutes, plus the minute Streams allows for clock skew.

Give each application a key of its own, narrowed to its databases or streams, so a leaked key exposes nothing else. For example, a key that only reads one stream:

tonbo api-key create events-reader --read-only --stream events --expires-in 15_days

To keep it independent of any person, create an Agent on Members and a key that acts as it. Keep the key in a secret store and give it an expiry. To rotate a key, create the next one, give it to the application (or the refresher) and restart it, then revoke the old key.

Why the refresher is a separate process

open() and resolveDatabase() from @tonbo/sqlite-ursula/tonbo need no refresher. The refresher is only for hosts that do not run the package and for anyone who wants the token in a file (URSULA_VFS_TOKEN_FILE).

node:sqlite is synchronous. While a commit waits for Tonbo, the thread that would call setToken is blocked, so a token that expires during a retried commit cannot be replaced from inside the application. The extension instead reads the file named by URSULA_VFS_TOKEN_FILE again whenever it changes and after every 401, including between the retries of one commit. A refresher in another process can therefore replace the token at any moment.

The refresher

Save this as refresh-token.mjs. It needs Node.js 22 or later and has no dependencies.

#!/usr/bin/env node
// Keeps the file named by URSULA_VFS_TOKEN_FILE holding a fresh Tonbo Streams
// access token for @tonbo/sqlite-ursula. It exchanges the Tonbo API key in
// TONBO_API_KEY for a token with the Streams audience (600 s in production)
// and replaces the file atomically: a temporary file next to it, then a rename.
//
// By default it keeps running and refreshes after about a third of each
// token's lifetime, with jitter. A failed refresh is logged and retried, and
// once the token in the file has expired the process exits 1, so a supervisor
// restarts it and alerts. With --once it refreshes once, quietly, and exits 0,
// or 1 on failure (for cron or a systemd timer).
import { rename, rm, writeFile } from "node:fs/promises";
import { setTimeout as sleep } from "node:timers/promises";

const AUDIENCE = "https://streams.tonbo.dev";
const tokenUrl = process.env.TONBO_TOKEN_URL || "https://tonbo.dev/api/iam/token";
const url = URL.canParse(tokenUrl) ? new URL(tokenUrl) : null;
const LOOPBACK = ["127.0.0.1", "localhost", "[::1]"];
const { TONBO_API_KEY: key, URSULA_VFS_TOKEN_FILE: file } = process.env;
const once = process.argv.includes("--once");
const log = (level, event, detail = "") =>
  console.error(`streams-token-refresher level=${level} event=${event}${detail}`);

/** Exchanges the key for a Streams token, writes it, and returns its lifetime in ms. */
async function refresh() {
  const response = await fetch(url, {
    method: "POST",
    headers: { authorization: `Bearer ${key}`, "content-type": "application/json" },
    body: JSON.stringify({ audience: AUDIENCE }),
    redirect: "error",
    signal: AbortSignal.timeout(15_000),
  });
  const body = await response.json().catch(() => null);
  if (!response.ok || typeof body?.access_token !== "string" || !(body.expires_in > 0)) {
    throw new Error(`token endpoint answered ${response.status}: ${body?.error ?? "no token"}`);
  }
  const temp = `${file}.${process.pid}.tmp`;
  // Created afresh ("wx"): never written through a link, or into a file with a
  // wider mode, that a crashed run or anyone else left at this name.
  await rm(temp, { force: true });
  await writeFile(temp, `${body.access_token}\n`, { flag: "wx", mode: 0o600 });
  await rename(temp, file);
  return body.expires_in * 1000;
}

async function run() {
  let lifetime = 0;
  let expiresAt = 0;
  for (;;) {
    try {
      lifetime = await refresh();
      expiresAt = Date.now() + lifetime;
      if (once) return 0;
      log("info", "refreshed", ` expires_in_s=${lifetime / 1000}`);
      await sleep((lifetime / 3) * (0.8 + 0.4 * Math.random()));
    } catch (error) {
      const cause = error.cause?.code ?? error.cause?.message;
      const reason = cause ? `${error.message} (${cause})` : error.message;
      log("error", "refresh_failed", ` error=${JSON.stringify(reason)}`);
      if (once || Date.now() >= expiresAt) return 1;
      await sleep(lifetime / 20);
    }
  }
}

if (!key?.startsWith("tbo_") || !file) {
  log("error", "misconfigured", ' detail="set TONBO_API_KEY (tbo_...) and URSULA_VFS_TOKEN_FILE"');
  process.exitCode = 2;
} else if (
  url?.protocol !== "https:" &&
  !(url?.protocol === "http:" && LOOPBACK.includes(url.hostname))
) {
  log("error", "misconfigured", ' detail="TONBO_TOKEN_URL must be an https URL"');
  process.exitCode = 2;
} else {
  process.exitCode = await run();
}

Run it as the application’s user, because it writes the file with mode 0600. Each refresh writes a temporary file next to it and renames it over the file, so the application never reads a partial token. The application must have URSULA_VFS_TOKEN_FILE in its environment when it starts, because the extension reads the variable once, and the file must hold a token before the first open.

Timing and failures

• It refreshes after a third of each token’s lifetime, with 20% jitter: every 160 to 240 seconds for a 600 second token. The token in the file still has at least 360 seconds left when a refresh starts, which covers a short outage of the exchange.
• A failed refresh is logged as streams-token-refresher level=error event=refresh_failed and retried every 30 seconds (a twentieth of the lifetime).
• Once the token in the file has expired, it exits with status 1, and it exits 1 at once if the first exchange fails. Run it under a supervisor that restarts it, and alert on the restarts.
• With --once it refreshes once and exits: 0 without output on success, 1 with the error on failure. Use it from cron or a timer, and before the application starts.

Running the refresher

systemd

A service that keeps /run/tonbo/streams-token fresh. Its comments list the lines the application’s unit needs to wait for the first token.

# /etc/systemd/system/streams-token-refresher.service
#
# Keeps /run/tonbo/streams-token fresh. Put TONBO_API_KEY=tbo_... in
# /etc/tonbo/streams-token-refresher.env (mode 0600). In the application's unit add:
#
#   Wants=streams-token-refresher.service
#   After=streams-token-refresher.service
#   Environment=URSULA_VFS_TOKEN_FILE=/run/tonbo/streams-token
#   ExecStartPre=/bin/sh -c 'until [ -s /run/tonbo/streams-token ]; do sleep 1; done'
#
# Alert on this unit's restarts (NRestarts) or on level=error lines in its journal.
[Unit]
Description=Tonbo Streams token refresher
Wants=network-online.target
After=network-online.target

[Service]
# The application's user: the token file is readable by its owner only.
User=app
EnvironmentFile=/etc/tonbo/streams-token-refresher.env
Environment=URSULA_VFS_TOKEN_FILE=/run/tonbo/streams-token
RuntimeDirectory=tonbo
RuntimeDirectoryPreserve=yes
ExecStart=/usr/bin/node /opt/streams-token-refresher/refresh-token.mjs
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

cron

# /etc/cron.d/streams-token-refresher
#
# Refreshes every 3 minutes, a third of the 600 s token lifetime, so two more
# runs can fail before the token in the file expires. The env file holds
# TONBO_API_KEY=tbo_... and URSULA_VFS_TOKEN_FILE=/var/lib/tonbo/streams-token.
# The command runs as app and reads the env file itself, so app must own it
# (mode 0600), and /var/lib/tonbo must exist and be writable by app. Cron's
# PATH is /usr/bin:/bin, so node is named by its full path. A successful run
# prints nothing, and cron mails a failed run's output to MAILTO. Run the same
# command once before the application starts, so the file exists.
MAILTO=ops@example.com
*/3 * * * * app sh -c 'set -a && . /etc/tonbo/streams-token-refresher.env && exec /usr/bin/node /opt/streams-token-refresher/refresh-token.mjs --once'

Kubernetes

The refresher as a native sidecar (Kubernetes 1.29 or later): it starts before the application, which starts once the first token is written. The Deployment uses the Recreate strategy because a database has one writer at a time. A StatefulSet works too.

# An application using @tonbo/sqlite-ursula, with the refresher as a native
# sidecar (Kubernetes 1.29 or later). Create its inputs first:
#
#   kubectl create secret generic tonbo-streams --from-literal=api-key=tbo_...
#   kubectl create configmap streams-token-refresher --from-file=refresh-token.mjs
#
# The sidecar starts before the application, which starts once the first token
# is written. A sidecar that cannot refresh exits 1 when its token expires and
# is restarted; alert on its restarts.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sqlite-app
spec:
  replicas: 1
  # One writer per database: the old pod stops before the new one attaches.
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: sqlite-app
  template:
    metadata:
      labels:
        app: sqlite-app
    spec:
      # One user for both containers: the token file is readable by its owner only.
      securityContext:
        runAsUser: 10001
        runAsGroup: 10001
        fsGroup: 10001
      initContainers:
        - name: streams-token-refresher
          image: node:22-alpine
          restartPolicy: Always
          command: ["node", "/opt/streams-token-refresher/refresh-token.mjs"]
          env:
            - name: TONBO_API_KEY
              valueFrom:
                secretKeyRef:
                  name: tonbo-streams
                  key: api-key
            - name: URSULA_VFS_TOKEN_FILE
              value: /run/tonbo/streams-token
          startupProbe:
            exec:
              command: ["test", "-s", "/run/tonbo/streams-token"]
            periodSeconds: 1
            failureThreshold: 60
          volumeMounts:
            - name: token
              mountPath: /run/tonbo
            - name: refresher
              mountPath: /opt/streams-token-refresher
              readOnly: true
      containers:
        - name: app
          image: registry.example.com/sqlite-app:1.0.0
          env:
            - name: URSULA_VFS_TOKEN_FILE
              value: /run/tonbo/streams-token
          volumeMounts:
            - name: token
              mountPath: /run/tonbo
              readOnly: true
            - name: data
              mountPath: /var/lib/sqlite-app
      volumes:
        - name: token
          emptyDir:
            medium: Memory
            sizeLimit: 1Mi
        - name: refresher
          configMap:
            name: streams-token-refresher
        # The database files. A new pod rebuilds them from the stream.
        - name: data
          emptyDir: {}