Delivery guarantees

Last updated 29 October 2026

View .md

Read this when you need to know what a stream guarantees: when an append is durable, that a retried append applies once, the one order every reader sees, and what each answer to an append means.

What is guaranteed

GuaranteeWhat you can rely on
An acknowledged appendOnce an append answers 200, it is durable: two of the stream’s three replicas, each in its own zone of the region, hold it, and older history moves to object storage in the same region. It survives the loss of a replica or of a whole zone.
Exactly once per producerAn append with a producer id, an epoch and a sequence number applies once, however often it is sent: a repeat answers 204 with the first one’s offsets.
One orderEvery reader sees the messages in one order, the order the stream took them in, from every producer, at offsets that never change.
Reads from any offsetAny number of readers read from any offset retention has not passed, as often as they like, from any machine.
No reader stateTonbo keeps no consumer groups, acknowledgements or leases. Each reader keeps its own offset, and reads a message as often as it reads its offset.
ClosingA closed stream refuses every append for good, and every reader at the end learns it.
DeletionA deleted or expired stream answers 404. A new stream under its name has new offsets, and a request that carries the old one’s Stream-Incarnation answers 412.
  ANY MACHINE: a laptop, a server, a sandbox, a function
  +--------------------------------------------------+
  |  your program, in any language                   |
  |    POST {url}   append          GET {url}?...    |
  |    with a key, over HTTPS       read or tail     |
  +-------------------------+------------------------+
                            |  HTTPS, the region's Streams origin
                            v
  TONBO, in the stream's region: three replicas, one per zone
  +--------------+       +--------------+       +--------------+
  |    zone a    |<----->|    zone b    |<----->|    zone c    |
  +------+-------+       +------+-------+       +------+-------+
         |                      |                      |      t1  two of three hold it:
         +----------------------+----------------------+          the append answers
                                |  in the background
                                v
                      +-------------------+
                      |  object storage,  |                     t2  older history and
                      |  the same region  |                         snapshots move here
                      +-------------------+

What happens to an append

What to do depends on the answer. With producer headers, sending the same request again is always safe: it applies at most once (see exactly-once writes).

AnswerDid it applyWhat to do
200 with Stream-Offset and Stream-Next-OffsetYes, onceSend the next sequence number.
204 with the same offsets as beforeYes, once, when it was first sentNothing: it is a repeat. Send the next sequence number.
204, sent without producer headersYes, and a repeat would apply againSend producer headers when repeats would do harm.
No answer, a timeout or a 5xxMaybeSend the same request again, unchanged, after any Retry-After.
409 PRODUCER_SEQ_CONFLICTNoSend the sequence number Producer-Expected-Seq names.
403 with Producer-OutcomeNoStop this writer: a newer instance of it has taken its place.
Any other refusalNoDo what Troubleshooting says for it, then send it again if it says so.

What readers see

A reader gets every message from the offset it reads from, in the stream’s order, and an answer says where to read next. A read normally comes from the replica that answers, so it may not yet show an append acknowledged a moment before; HEAD and a read with consistency=leader see every append acknowledged before them (see Read). A tail delivers each append as it arrives, and a reader whose connection drops opens a new one from the last offset it saw.

Tonbo does not track what a reader has done with a message. A reader that crashes after acting on messages, but before it saved the offset after them, reads them again. To act on each message once, save the offset in the same transaction as what the reader did, for example in a database (see the rule).

When an append is durable

An append answers once two of its stream’s three replicas hold it, each in its own availability zone of the region, so from its answer on it is kept through the loss of a replica or of a zone, and older history then moves to object storage in the same region. A replica that restarts or comes back catches up from the other two. Tonbo restarts the replicas one at a time for maintenance, and an append may wait a few seconds while another replica takes over.

Retention and deletion are yours: a stream keeps everything until you advance its retention (see Retention) or delete it (see Delete a stream), and closing it (see Close) keeps it as it is.

Each stream is a durable stream, the open protocol from ElectricSQL, served by Ursula, Tonbo’s open-source stream server. Ursula in Tonbo describes its replicas, their quorum and their restarts.