Mail flows
Mail flows
This page walks through the main request flows between the React client, Pages Functions, D1 and the mail provider.
List messages
GET /api/messages?accountId=all&folder=inbox- The list is served from the D1 mirror first. If the folder has never been synced, the request bootstraps the newest headers from IMAP; otherwise a stale folder returns the cached list immediately and schedules a background sync.
- The sync reads
UIDNEXTfromSELECTand fetches new UIDs plus bounded backfill windows instead of runningUID SEARCH ALLon every pass.BODYSTRUCTUREis parsed locally to detect attachments without downloading message bodies. - Headers are parsed with
postal-mime; results from all accounts are merged and sorted. Relay accounts contribute their D1messagesrows. - The merge sorts with
compareByDateDesc(sent_atdescending,iddescending) — the same order as the SQLORDER BY sent_at DESC, id DESCand the keyset cursor (sent_at < ? OR (sent_at = ? AND id < ?)). The two sides must agree: with an ascending tie-break, two messages sharing a timestamp would be returned twice and, after the client deduplicates withmergePage, silently disappear from the list. - After the response, the server prefetches the newest few bodies (one message per account in the unified view, up to three for a selected account) into the D1 body cache with
UID FETCH ... (BODY.PEEK[])on a single connection.
Search
GET /api/messages/search?q=...returns matching messages from the D1 mirror (Inbox, Sent, Drafts, Spam, optionally one folder) plus matching contacts.- Matching is a case-insensitive substring test (
lower(column) LIKE ?), not a tokenized full-text index: no stemming, no fuzzy matching, and every term must appear inside the searched columns (subject,snippet,from_email,from_name,to_json,cc_json, andsearch_textorbody_text). imap_messages.search_textkeeps up to 4.000 characters of body text, filled when a body is parsed (message opened, archived, or snippet backfill). Search therefore matches body words only for messages indexed so far; the rest still match sender, recipients and subject. Relay rows search their storedbody_text.- Results are paginated with the same
(date, id)cursor and merge order as the list endpoint.
Read a message
GET /api/messages/:id- The ID encodes the account, folder and UID (
i_<account>.<folder>.<uid>) or a relay row (r_<uuid>). - The raw MIME is served from the D1 body cache when available, then from the R2/Drive archive, then from IMAP. A body fetched from IMAP or the archive is stored in the D1 cache (up to 512 KB) to make later opens and attachment requests fast.
- Text, HTML and attachment metadata are returned. The message is marked
\Seenfor Inbox and Spam items. Cache/archive hits update D1 immediately and mark the provider asynchronously, so the HTTP response does not wait for a second IMAP connection. - HTML is rendered by the client in a sandboxed iframe with a restrictive CSP; scripts and forms are disabled. Remote images are blocked by default and can be enabled per message. Small
cid:images are embedded as data URIs in the detail response; larger ones fall back to the authenticated attachment endpoint.
Message actions
POST /api/messages/actionswith{ ids, action: "delete" | "spam" | "move", target?: "inbox" | "spam" | "trash" }.- IMAP messages are grouped by account and folder; one connection copies the UIDs to the target with
UID COPYand removes them from the source withUID STORE +FLAGS (\Deleted)plusUID EXPUNGE. - Delete moves the message to the provider Trash; spam moves it to the
\Junkfolder; not-spam moves it back to Inbox. - The D1 mirror is updated and the source/target folders are queued for sync. Moved rows stay in D1 until the target sync transfers labels through
stable_key, then the source sync prunes them.
Send
POST /api/messages/send- The server builds a complete MIME message.
- IMAP account: authenticate and send through the account’s own SMTP.
- If
auto_saves_sentis false, the same MIME is appended to the provider’s Sent folder (\Sent, fallbackSent). - If
draftIdis present, the original draft is deleted. - Relay account: store the
.emlon the destination chosen instorage_driver, send through the selected HTTP relay, then record metadata in D1.drive/r2: the object is written before the send, so a failed send still has its copy.imap: nothing is written to object storage; the same MIME is appended to the Sent folder of the account insent_copy_account_idafter the relay accepts the message,storage_keykeeps a pointer (imap:<account>:<folder>:<uid>) and a Sent sync is queued for that account. If the append fails the message is still sent and recorded: when the deployment has any object storage (Drive connected or theBUCKETbinding) the MIME is written there as a rescue copy andmessages.storage_driverrecords that driver, so attachments and the raw view keep working; without object storage only the D1 body remains and the response carries a warning.
Reply
The reply composer takes its sender from
pickReplySender, in this order:accounts.reply_account_idof the mailbox that received the message (explicit override);- the account whose address is the alias the message was actually sent to:
aliasAddresses()collectsDelivered-To,X-Delivered-To,X-Original-To,Envelope-To,X-Envelope-To,X-Forwarded-To,X-Forwarded-For(Cloudflare Email Routing),Resent-To,Apparently-ToandReceived … for <addr>, in header order (topmost first), and the client also looks at the message To/Cc. Only accounts that can send are considered (relay configured, or IMAP connected). When no account has exactly that address, a relay withrelay_any_addresswhose domain the alias is on is used instead (see step 2). Both kinds of match go through the same checks inpickReplyAccount()(shared/from-address.ts): a forwarding header only counts when the receiving server vouches for its domain, the first such header that names an account wins, and To/Cc only count when they name exactly one such address; - the global “reply sender” preference (
autoby default); - the account that received the message.
The alias step is what makes a forwarding domain work like Gmail: mail addressed to
[email protected]and forwarded to[email protected]keeps that address in the headers, so the reply leaves from the domain account.If the chosen account is a relay with Send from any address on this domain on (
accounts.relay_any_address), the reply leaves from the address the message was sent to, as long as it is on the account’s exact domain (pickDomainAlias()inshared/from-address.ts):- the first address on the domain in
aliasAddresses(), read top-down, but only when the receiving server’s topmostAuthentication-Resultsreports a passing DKIM signature for that exact domain (verifiedDomains()). Cloudflare Email Routing signs forwarded mail with the domain; mail delivered straight to the mailbox can’t carry such a signature, so a forgedX-Original-Toon it is ignored. On forwarded mail, every server prepends its headers, so the forwarder’s own record sits above anything the original sender wrote; - otherwise To/Cc, only when exactly one address there is on the domain;
- otherwise the account’s own address.
With several such relays, the trusted forwarding headers are searched across all of them before any To/Cc, and To/Cc only count when they name exactly one address across all relay domains and accounts (
pickReplyAccount()). The receiving mailbox’s own address, which often heads the headers on a shared domain, is skipped while looking for the alias; mail addressed only to it stays with that account. Replying from Sent keeps the address the message was sent from, since To/Cc there are the recipients, and a relay’s own Sent copy is answered by that relay. Only addresses the send endpoint accepts are offered:[email protected]is read whole, then not used, rather than cut down to[email protected]. Two relays allowed to send as the same domain make the alias ambiguous: WhelkMail then offers no alias at all, and the reply leaves from the account’s own address.The composer shows the address as its own entry in the From list, so it can be switched back before sending.
POST /api/messages/sendtakes it asfromEmailand refuses (from_address_not_allowed) anything that isn’t the account’s own address or, with the setting on, another address on its exact domain (no subdomains). The From in the MIME, the relay call and the stored Sent copy all use it. A draft keeps only the account, so a reopened draft sends from the account’s own address.- the first address on the domain in
Everything else is the normal send flow, so the reply leaves through the chosen account’s own SMTP/relay and its copy follows that account’s destination. Addresses of both accounts are excluded from To/Cc to avoid replying to yourself.
Conversations
GET /api/threads?accountId=&folder=returns one row per conversation. The group key isCOALESCE(thread_key, stable_key), so messages mirrored before threading existed stay visible until the backfill reaches them.- Pagination is a keyset cursor on
(last message date, thread key): the scoped query decides which conversations are on the page (HAVING last_at < ? OR (last_at = ? AND key < ?)), then every message of those conversations is loaded from all folders of the account (drafts excluded) to build the row: newest message, unread count, labels and the distinct senders, so the list reads like Gmail (Mario, Luca +2 5) and an answered thread counts your reply even when it lives in Sent. - Threads are per account: the same subject on two mailboxes stays two conversations. The unified view lists both, never merges them.
- Ordering and the cursor use the same pair as the SQL,
(last_at, thread key), so conversations sharing a timestamp cannot repeat or vanish between pages. GET /api/threads/messages?accountId=&threadKey=matchesCOALESCE(thread_key, stable_key)('r:' || idfor relay rows), the same keys the list used, and returns the whole conversation (Inbox, Sent and Spam of that account, drafts excluded), oldest first, so your replies appear inside the thread. The reading pane stacks one header row per message with the opened message expanded below it; clicking another row loads that message in place.- The endpoint does no IMAP work at all: it reports
syncing,staleAccountsandactiveAccounts. AuseEffectrunning once the list is on screen callspickSyncRequests()(src/lib/sync-queue.ts, keys areaccountId:folder, so switching folder does not hit the cooldown of the previous one): at most three folders, never a locked one, never one asked in the last 3 seconds. Each selected folder is then driven byrunSyncChain()(src/lib/sync-chain.ts), which keeps callingPOST /api/accounts/:id/syncwhile the answer sayshasMore, stops on error, stops when another worker holds the lock (locked) and gives up after a cap of 30 chunks or after 3 chunks in a row that do not advance the chunkprogress— mirrored rows plus archived bodies — so a healthy archive run is not mistaken for a stall.partialmeans “more to fetch”, never “already fetching”. Accounts are queried in parallel. The messages of the visible conversations are loaded with a per conversation cap (ROW_NUMBER() OVER (PARTITION BY key …) <= 40), so a busy thread cannot push another one out of the page, while an exactCOUNT(*)/SUM(unread)per key keeps the totals right even when only part of the history is loaded. When a chain ends withoutdone(cap reached, or a lock held by another worker) the client schedules another chain after a short pause (nextChainDelay: 500 ms, 5 s for a lock), keeping one chain per folder, so a very large mailbox keeps going todone. The chain lives in the browser: closing the tab leaves the folder partial, and it resumes on the next load (or with the cron worker). thread_keyis the Message-ID of the root, computed bythreadKeyFor()(References[0]→In-Reply-To→ own Message-ID). Replies written in the app sendIn-Reply-To/References, so their copies thread in any client. A bounded sync phase re-reads only the headers of rows still missing a key.- Label views and search results are grouped as well: a label view loads the whole conversation for the conversations that carry the label (
GET /api/threads?labelId=), and search collapses the matching messages of a thread into one row whose count refers to the matches on that page. Deleting a thread row acts on the newest message shown. - When a message has no
References/In-Reply-To(broken chains, forwarded copies),normalizeSubject()stripsRe:/R:/AW:/SV:/Fwd:and the conversation already stored for that subject is adopted — the same fallback Gmail uses. Only messages whose subject carries a reply prefix take part, so two unrelated mails sharing a subject keep their own threads; the parent lookup also covers themessagestable, so a reply to something sent from the app joins the right conversation. - Settings → System → Conversations shows how many mirrored messages still have no key per account and rebuilds them in bulk (
GET/POST /api/threads/rebuild, headers only, 200 messages per call).
Drafts
- IMAP:
APPENDto the\Draftsfolder with\Draft \Seen; delete withUID STORE +FLAGS (\Deleted)+EXPUNGE. - Relay,
storage_driver = 'drive' | 'r2': the MIME is stored on the object archive (drafts/<id>.eml, oraccounts/<id>/drafts/<id>.emlfor R2), metadata in D1 withfolder='draft'. - Relay,
storage_driver = 'imap': the draft is appended to the Drafts folder of the account insent_copy_account_id, exactly like a draft written from that account.- The folder is resolved against the target account: its
drafts_folderfirst, then the\Draftsspecial-use flag, then known names (Drafts,INBOX.Drafts,INBOX/Drafts,Bozze,INBOX.Bozze,Bozza,INBOX.Bozza,Draft,INBOX.Draft). A missing folder is created withCREATEand the append retried. - Flags are
(\Draft \Seen). The UID comes fromAPPENDUIDwhen the server supports UIDPLUS, otherwise from the folder’sUIDNEXT - 1or from the highest UID after a search. - The row in
messageskeepsfolder='draft',storage_driver='imap'and a pointer instorage_key. Reopening the draft fetches that message back withUID FETCH; autosave appends the new version and expunges the previous copy; deleting or sending removes the copy from that mailbox. - There is no automatic archive on Drive/R2 in this mode. If the target account is missing or not connected, saving fails with
sent_copy_target_not_connected. When the APPEND itself fails and object storage exists, the draft is rescued there instead of being lost; with no object storage the save fails withdraft_save_failed. - The mailbox copy is mirrored as an ordinary draft of the target account and filtered out of the Drafts list and draft searches, so the draft is listed once, under the relay account that owns it.
- The folder is resolved against the target account: its
- Autosave runs 5 seconds after the last change and every 30 seconds while changes remain. Closing the composer saves pending changes first; if the draft cannot be saved the user is asked whether to discard it.
- An explicit Save draft closes the composer and opens the Drafts folder.
Where a send-only account keeps its copies
storage_driver picks one destination per relay account; the two mailbox modes never run together with the object archive:
storage_driver |
Sent copies | Drafts | Requirement |
|---|---|---|---|
drive |
sent/<id>.eml on Google Drive |
drafts/<id>.eml on Google Drive |
Google Drive connected |
r2 |
accounts/<id>/sent/<id>.eml in R2 |
accounts/<id>/drafts/<id>.eml in R2 |
BUCKET binding |
imap |
APPEND to the target account’s Sent folder |
APPEND to the target account’s Drafts folder |
sent_copy_account_id pointing at a connected IMAP account |
Switching back to drive or r2 clears sent_copy_account_id. Copies appended to a mailbox are real messages on the provider: deleting the account or a row in D1 never deletes them. Each message keeps the destination it was written with, so changing destination does not orphan the older copies.