Fireside 0.1.0-next.6

Write-behind durability, scan and import performance. Published on , as a preview.

Published as @fireside-dev/cli, before Firenook’s rename. About the former name.

A newer release is out: Firenook 0.2.0, the first release for public use.

What changed

The changes, as the release notes give them.

  1. Write-behind durability is the default (#39). A Firestore commit is acknowledged after its journal write and a non-durable redb commit; a background flusher (every second) and shutdown sync the journal, persist the commits with one durable redb commit and truncate the journal. Storage keeps the same contract with its own metadata journal and syncs uploaded files before the metadata that references them becomes durable. Killing the process loses nothing acknowledged (reopen replays the journal); only a kernel crash or power cut can lose the last interval. --durability per-commit restores the previous flush-per-write behaviour. Measured on a Mac: single document set 13.1 ms → 0.59 ms. In the consumer's Linux soak the write-commit median went from 23.9 ms with per-commit durability to 3.2 ms (official emulator on the same host and workload: 197.5 ms) and the 64 KiB Storage upload from 18.0 ms to 2.7 ms.

  2. Every HTTP front (REST, Storage, Auth, hub, UI, Requests feed) sets TCP_NODELAY on accepted connections, as the gRPC front already did (#35). A 64 KiB download over a reused keep-alive connection on Linux fell from 40.7 ms to 0.5 ms (Nagle waiting on the client's delayed ACK); fresh connections were unaffected.

  3. Queries not served in __name__ order no longer decode every scanned document (#38). Stored values gain a field directory that lets filters, order keys, cursors and vector distances read one field path; only result pages are decoded. __name__ ==/in filters read the named documents directly and count-only aggregations use a dedicated count path. Values written by earlier versions stay readable without migration. On a 10,918-document collection of ~30 KiB documents: orderBy field limit 20 642 → 106 ms, count() 737 → 91 ms, __name__ in ten ids 646 → 10 ms.

  4. REST ListDocuments for a named collection reads the scoped collection index instead of scanning the whole database (#37): pageSize=1 on a 211,202-document store 1.3 s → 1–4 ms.

  5. Fresh-start seed import runs inside one redb transaction with pipelined export decoding and concurrent Storage object copies (#36); Storage import of 33,353 objects 11.1 s → 3.9 s on a Mac. Persistent resume is unchanged.

What got faster

Measured on the same data with 0.1.0-next.5 and with 0.1.0-next.6. Lower is better.

A write, on a Mac

22× faster

Every multiple is rounded down. Every release’s measures.

Known limits

What this release was known not to do yet, as its notes recorded it.

  • Limitations recorded with this release: ordered full-collection scans and count() over a large collection of large documents still run at roughly half the Java emulator's speed (CPU-bound copying of every candidate's bytes out of redb), and fresh import of a large seed remains slower than the Java in-memory import (16 s versus 7 s in the consumer's measurement, the Java figure carrying no durable state). State written by this version carries the field directory and a durability journal, so it is not meant to be reopened by an older package version. Prerelease on next; no stable or universal-compatibility claim.

How it was checked

The source it was built from, the runs it passed, and how it was published.

  • Pin the engine to 572d4fb5f9d986accf2473858930c1e9bc3e5d57, the merge of PR #39 on the next.5 engine. This exact revision passed the seven-job CI, the five-platform packed-install matrix and a representative private consumer's fresh, sequential official-then-Fireside full-data acceptance (two two-hour soaks, initial and post-restart browser journeys, lifecycle parity, fresh setup and regression commands, all with zero errors); see support/phase-f-qualification.md.
  • Dependencies: tower-http 0.6.11 → 0.7.1 and tokio-tungstenite 0.29 → 0.30 (Rust), actions/setup-java v6.0.1 (CI). No change to the pinned firebase-tools (15.22.0), the Rust toolchain or the package layout.

The release on GitHub.

The latest is 0.2.0

The first release for public use. Install it, and read what changed on the way.

npm install --save-dev firenook@0.2.0
Read the 0.2.0 notes