Source: https://fireloom.io/sync-engine

> What happens between Firestore and Postgres: the first load, real-time listeners, checkpoints, deletes, subcollections, consistency checks and what it costs in Firestore reads.

# Inside the sync engine.

How Fireloom keeps Postgres in step with Firestore: the first load, the change stream, restarts, deletes, subcollections, and the checks that stop drift. For the setup itself, see [how it works](https://fireloom.io/#how).

## 1. The first load

A new sync has to copy what's already there. Fireloom **starts listening first**, from “now”, so nothing that changes during the load is missed. Then it loads every document, in one of these ways:

- **Export for me** (the default): Fireloom runs one Firestore export into a bucket of its own in your project, bulk-loads it, and deletes it.
- **An export you already have:** no Firestore reads for the load if it's recent.
- **Directly from Firestore:** one query that returns every document at every depth.

Changes that arrive while the load runs are applied by version, so an older copy from the export never overwrites a newer change.

## 2. Streaming changes

Fireloom keeps Firestore listeners open, one per collection ID, which covers that collection at every nesting level. Each batch of inserts, updates and deletes is written to Postgres in a single transaction, usually within a second of the write in Firestore. Upserts only write when a document's `update_time` changed, so replays are harmless.

## 3. Restarts and outages

Each listener's resume token and read time are saved as a checkpoint. After a deploy, restart or network blip, Fireloom resumes from the checkpoint and receives only what changed meanwhile, deletes included. If a listener was away too long for Firestore to resume it (about 30 minutes), that collection is re-read once.

## 4. Deletes

Deleted documents are deleted from Postgres while streaming. During a load, recent deletes are remembered in a small internal table, `_fireloom_tombstones`, so a document deleted mid-load doesn't come back from an older export.

## 5. Subcollections

Every document lands in one table, `documents`, keyed by its full path, with `collection` and `parent_path` indexed for filtering and joins. `customers/ada/orders/1042/items/1` is synced like any top-level document, even if a parent document doesn't exist. New top-level collections are found within minutes; a subcollection with a brand-new name is found at the next consistency check or on Resync.

## 6. Consistency checks

Once a day (configurable from 1 to 168 hours) Fireloom compares a `COUNT()` of each source with Postgres. That costs about one Firestore read per 1,000 documents. Only a mismatch triggers a reload. Resync reloads on demand.

## 7. When something breaks

Errors restart the stream with exponential backoff, from 5 seconds to 5 minutes. If a sync stops, or is still retrying after 15 minutes, the owner gets one email, and another when it's fixed. Errors retrying can't fix, like a revoked key, stop the sync until you change it.

## Security

- Firestore access is read-only (Cloud Datastore Viewer), plus an export-only role that can't import or overwrite data when Fireloom takes the export.
- Service account keys and connection strings are encrypted with AES-256-GCM at rest and never returned by the API.
- Fireloom writes only to tables it created, which carry a marker comment, and refuses connections to private network addresses.
- Documents pass through Fireloom in memory on their way to Postgres; they aren't stored on Fireloom's servers.

## See it on your own data

Connect Firestore and Postgres in about five minutes. Free up to 100,000 documents.

[Start syncing free](https://fireloom.io/app?signup=1)