# Recovery after a crash

> What a crash or a kill loses, what the next start does about it, and every refusal Firenook gives for kept data, with its fix.

Kept data survives a killed process. This page explains what a crash can
lose, what the next start does after one, what the messages about
unfinished stops and locked data mean, and how to fix each refusal.

## What a crash loses

By default Firestore writes are **write-behind**: Firenook answers a write as
soon as it is in the store's journal, and syncs the journal to the drive
once a second and at every stop.

- **A killed Firenook loses nothing.** A write is in the journal before it
  is answered. In a test, 500 documents written right before a `kill -9` of
  the engine were all there at the next start.
- **A kernel crash or a power cut** can lose the last second of writes. The
  store stays consistent: you get the data as it was a moment before.

`--durability per-commit` syncs every write to the drive before answering
it, so not even a power cut loses an answered write. Each write then waits
for the drive's flush, which is much slower than write-behind:

```sh
npx firenook emulators:start --keep-data --durability per-commit
```

## The next start after a kill

A store that was not closed cleanly is repaired at the next start, before
the emulators answer.

Under write-behind the store saves what a repair needs with each sync, so
the repair takes moments and the start looks like any other.

Under `--durability per-commit` the repair reads the whole store once. The
terminal shows its progress on the Firestore row, `repairing NN%`, with the
store's size. The plain log prints:

```text
firenook: Firestore was not closed cleanly last time; repairing it reads the whole 10 MB once. Stopping now only postpones the repair.
firenook: repairing Firestore: 60% after 0 s
firenook: repairing Firestore: 90% after 0 s
firenook: Firestore repaired in 0.0 s (10 MB)
```

Let a repair finish. Stopping it is safe, but the next start begins it again.
On a store of tens of gigabytes it takes minutes.

Data last written by a release before 0.2.0-next.5 has no saved repair
information, and is repaired by reading the whole store once, whatever the
durability.

## When the engine is killed

If Firenook's engine ends while it runs, the terminal says so and names the
signal:

```text
✕ Firenook stopped unexpectedly                                     signal SIGKILL

Data          the next start checks .firenook/state first, and repairs it if it needs to
```

The plain log says the same in one line, and the command exits with code 1:

```text
firenook: the engine was ended (SIGKILL) while it was running; the next start checks the data first, and repairs it if it needs to
```

If it is killed after you asked it to stop, but before it finished, the
stop did not save cleanly:

```text
✕ Firenook did not finish stopping                                  signal SIGKILL

Data          the next start checks .firenook/state first, and repairs it if it needs to
```

The plain log says the same in one line:

```text
firenook: the engine was ended (SIGKILL) before it finished stopping; the next start checks the data first, and repairs it if it needs to
```

In both cases, start again with the same command. Nothing else is needed.

Killing the `firenook` command rather than its engine, even with `kill -9`,
is not a crash: its engine stops too, saves the data and frees the ports
within a moment, and the next start picks the data up.

A stop while Firenook is still starting is safe: it ends a seed check or a
repair at once, with nothing opened. An import into kept data finishes
first, because half an import could never be picked up.

## Data in use by another Firenook

Only one Firenook can use a state folder at a time. A second start on the
same folder is refused before it opens anything:

```text
Saved data  ✕   0 ms  in use by another Firenook

Problem       suite state is locked: lock acquisition failed because the operation would
              block, so Firenook will not pick up .firenook/state

Fix           another Firenook is using .firenook/state: stop it first, or start this one
              with another --state-dir
```

If no other terminal runs Firenook, look for the run that holds the folder,
for example one a script started in the background. Find it by its port and
stop it with `SIGINT`, which saves the data:

```sh
lsof -nP -iTCP:4400 -sTCP:LISTEN
kill -INT <pid>
```

Never delete the lock file or the state folder to get past this message.
Firenook never deletes kept data, and nothing on this page needs you to.

## Refusals and their fixes

Every refusal leaves the data as it was and says so: `nothing was deleted`
in the terminal, `existing data preserved` in the plain log.

| The message says | What it means | Fix |
| --- | --- | --- |
| `the seed <folder> changed after it was imported` | The seed's content changed since the first start | Import it into a new `--state-dir`, or run `state:adopt-seed` ([seeds](https://firenook.dev/docs/data/seeds#when-the-seed-changes)) |
| `it was imported from <folder>: pass --import <folder> to keep using it` | You started seeded data without its seed | Start with the same `--import` |
| `this data was started without a seed, so it cannot take the seed <folder>` | You gave `--import` to data kept without a seed | Start without `--import`, or import into a new `--state-dir` |
| `Existing nonempty state requires explicit --resume-state (or --keep-data)` | The `--state-dir` folder already holds data | Add `--resume-state`, or name an empty folder |
| `--resume-state requires the firestore, auth and storage services` | `--only` left out one of the three | Start all three ([keep data](https://firenook.dev/docs/data/keep-data#what-is-kept)) |
| `suite state is locked` | Another Firenook uses the folder | Stop it, or use another `--state-dir` |
| `the import that made it did not finish` | The first start was stopped before its import ended | Import again into a new `--state-dir` |
| `it belongs to project <id>, not <id>` | The folder holds another project's data | Start with that project's id, or another `--state-dir` |
| `it holds the Storage buckets <list>, and this start serves <list>` | The buckets in `firebase.json` changed | Restore the buckets, or use a new `--state-dir` |
| `it was written in state format <n>, and this engine reads format <m>` | Another release wrote the data | Start it with that release, export, and import the export |
| `nonempty state has no completion receipt` | The folder holds files that are not Firenook's kept data | Name an empty folder |

## Next

Make a copy you can always go back to with [export and
snapshots](https://firenook.dev/docs/data/export), or see every flag in the [CLI
reference](https://firenook.dev/docs/reference/cli).
