Snapshots and S3

Last updated 29 October 2026

Read this when you need to know what Tonbo keeps of a database, and where: how the process that writes it keeps its stream short enough to rebuild from, and what Ursula keeps in S3.

When the owner publishes a snapshot

Ursula stores snapshots but does not make them: the process that owns the database does. A snapshot is due once the commits acknowledged since the latest one take more than the database’s size or 8 MiB, whichever is larger. A thread of the owning process then:

Checkpoints the WAL and copies the database as of the last logged commit.
Waits until the stream acknowledges that commit, at an offset W.
Compresses the copy with Zstandard behind a 56-byte header that names W, and publishes it at W, keyed by W and the body’s digest. A body larger than one request goes up in parts.
Moves the stream’s retention to the snapshot before this one, so the history it keeps readable is always one snapshot behind.

  offset 0                                                    tail
  |-- older log --|== snapshot at W1 ==|-- log --|== snapshot at W2 ==|-- log --|
                   retention: readable                  latest:  a cold open reads
                   history starts here                  this snapshot, then the
                   (one snapshot behind)                log after W2

A publish that fails waits from 100 ms, doubling, up to five minutes before the next try. db.status().snapshot shows the log written since the latest snapshot (log_bytes), the threshold (due_bytes) and the last error. Tonbo supports databases whose snapshot is up to 256 MiB compressed (see Limits).

A cold open

A host without a copy of the database builds one from the latest snapshot and the log after it, in a directory it renames into place only once the copy is complete, and then claims the database:

  open("notes") on a host with no copy
        |
        |  HEAD                     -> the latest snapshot's offset, W
        |  GET snapshot/W, by range -> gen-<id>.tmp/db
        |  GET the stream from W    -> apply the commits after it
        |  rename gen-<id>.tmp      -> gen-<id>, the copy is ready
        v
  claim: this process owns the database

Every read goes to Ursula, which streams the snapshot’s body and the older log from S3 when they are there. Your host never reads S3. Cold starts says how long a build takes.

What Ursula keeps in S3

A write lives first in memory and in the Raft logs of the voters. Ursula’s cold tier then moves to S3, in the background:

What	When
A stream’s committed log, in chunks	A flush runs every second. It moves a group’s committed bytes once the group holds 8 MiB of them, a stream’s whole tail once it is 5 minutes old, and small backlogs once the groups a voter leads hold 128 MiB together. Small chunks are merged later
A snapshot’s body	On publish, for a body of 1 MiB or more. A replaced body is deleted 5 minutes after the snapshot that replaced it
Ursula’s own snapshots of each Raft group	As each group’s Raft log grows, compressed with Zstandard and written with a conditional write, so no snapshot replaces another

After an upload succeeds, Ursula records the object’s reference through Raft. A read then takes recent bytes from memory and older ones with ranged reads of S3, and any voter that applied the reference can serve it. Ursula writes S3 through Apache OpenDAL.

Compression happens before data reaches Ursula, and once inside it. The extension compresses every commit’s pages and every snapshot with Zstandard on your host (see The SQLite extension), and Ursula stores the stream’s chunks in S3 as it received them, without compressing them again. Ursula compresses its own Raft group snapshots with Zstandard before it writes them to S3, and the Raft traffic between voters too: streamed requests always, other requests from 1 KiB.

Tonbo’s S3 settings

Setting	Value
Where	One private store per region, in the database’s region
Public access	Blocked
Encryption	SSE-S3 (AES-256) on every object
Versioning	On
Lifecycle	Replaced and deleted versions expire after 30 days; incomplete multipart uploads are aborted after 7
Storage class	S3 Standard; no rule moves objects to another class

What S3 does not give you yet

S3 holds a database’s flushed log and a snapshot body of 1 MiB or more, but not a database on its own: the newest writes are only on the voters until they are flushed, and Ursula cannot yet rebuild a deployment from S3 alone: its roadmap lists recovery from the cold tier alone as not done, and today a total loss needs a backup made with Ursula’s backup tool, which Tonbo does not run for databases. An export is what takes a database out of Tonbo (see Backups and salvage).