Operations Runbook¶

Applies to: CPersona 2.6.x. This page is the canonical operations reference: backup, degradation detection, recall tuning, CJK guidance, and corpus indexing patterns. The behaviour facts it relies on are contracts — see Behavior Contracts.

Backup and restore¶

The database is a single SQLite file (CPERSONA_DB_PATH) running in WAL mode. A plain cp of a live WAL database can straddle a checkpoint and produce a corrupt copy, so do not script one.

Recommended, in order:

Online physical backup (first choice — safe while the server runs):

sqlite3 "$CPERSONA_DB_PATH" ".backup 'cpersona-backup.db'"
# or
sqlite3 "$CPERSONA_DB_PATH" "VACUUM INTO 'cpersona-backup.db'"

Both produce a consistent snapshot under concurrent writes.

Offline copy: stop the server, then copy the .db together with its -wal and -shm siblings.
Logical backup (recommended as a low-frequency complement): export_memories writes JSONL that is independent of the schema version, and import_memories is idempotent (memories and episodes already present are skipped). A profile is the one record an import replaces rather than skips: the file's copy replaces the stored one unless the stored one is newer, in which case the stored profile is kept and the line is reported in errors. A drill with an older export therefore does not move a profile backwards. Monthly is a reasonable cadence. The file carries the declared associations (entities, aliases, mentions and relations) as well as the records, and a restore points them at the records it restores. A file written by an earlier version has no associations in it and still imports as it always did. A file written by 2.4.37 or earlier also has no project or channel axis: its rows are restored into the global pool, where every project reads them, and the import says how many in errors. An import refuses a file larger than CPERSONA_MAX_IMPORT_BYTES (100 MiB by default), and an export has no such cap: compare the size of each backup file with it, and raise it (configuration) before a restore needs it.

The .db is not the whole instance. State lives outside it that none of the backup forms above touch:

<CPERSONA_DB_PATH>.calibration.json — per-agent vector thresholds, gate state, scoring version, and the per-agent set_recall_precision betas.

Restore without it and the measurements repair themselves. With the default CPERSONA_CALIBRATE_ON_MODEL_CHANGE=true, a startup with no sidecar recalibrates rather than falling back, and deep_check reports never_calibrated if it did not.

The betas do not repair themselves. They are policy inputs, not measurements, and nothing re-derives a preference you stated. An operator who tuned precision loses that tuning without being told, and the recalibration measures every gate at the default beta. Copy the sidecar beside the database, or re-apply set_recall_precision after a restore. - ~/.cpersona/operating-context.toml (or CPERSONA_OPERATING_CONTEXT_PATH) — the operator instructions served to every client. - The ACL file, if CPERSONA_ACL_FILE is set. Restoring without it changes who may connect. - The alias ledger, if any ACL row sets per_subject: alias_ledger.json beside the database, or the file CPERSONA_ALIAS_LEDGER_FILE names. The server writes it when a subject first signs in, and it is the only record of which u- agent belongs to which person. Restore the database without it and the server refuses to start, naming those agents. Restore the ledger to that path, then start the server again. Do not rename the agents or remove per_subject instead: either one leaves every signed-in subject with a fresh alias and none of the memory it stored. - The external vector index, under CPERSONA_VECTOR_SEARCH_MODE=remote with CPERSONA_STORE_BLOB=false. In that configuration the embeddings exist only in the remote index, so restoring the database and every file above still comes back with no vector arm. check_health reports it as no_local_vector_fallback. Back the index up on the same schedule, or keep STORE_BLOB=true so the .db stays self-sufficient. - Not <CPERSONA_DB_PATH>.memories.vecindex or .episodes.vecindex, the contiguous vector indexes. They are derived from the database, and are rebuilt rather than restored. A backup that includes one wastes space, and a restore that brings back a stale one is corrected by the next build. See the contiguous vector index.

Keep the live database outside cloud-sync folders (Dropbox, Drive, and so on). Sync clients interact badly with SQLite WAL files. Sync the backups instead.

Detecting a dead embedding server¶

Vector search is the strongest retrieval layer, and it degrades without announcing itself at the network boundary. Three detection surfaces exist, and the calling agent should be instructed to watch the first.

advisory on recall responses (v2.4.33+, the primary surface). A state machine observes real embedding failures, and degraded recalls carry advisory = {degraded, severity, reason, evidence, runbook, advisory_scope} — "the vector layer is down; serving FTS + keyword only".

Instruct your agent to surface this field to the user and follow its runbook (usually: start or repoint the embedding server, then recall again). A shortened runbook means "you were already told", and advisory_scope says who "you" was: session when the process is yours alone, process on the HTTP transport, where the state is the whole server's. On a shared server a fault repeats its full runbook rather than assume your session saw it (bug-251).

CPERSONA_DEGRADED_ADVISORY=false silences the advisory. Set it to record that the operator accepts running without an embedding backend, which is a supported fallback and not a recommended configuration. Design: DEGRADED_ADVISORY_DESIGN. 2. embedded on store responses. Every write reports whether its embedding was persisted. Writes made while the server is down come back embedded: false, and those rows are repairable (next item). 3. check_health(agent_id, fix=true) detects rows stored with NULL embeddings and re-embeds them once the server is back. The dimension check still skips rather than fails when the endpoint is unreachable, but the endpoint itself is now watched: embedding_backend reports warn, with the failing call's own evidence, when a configured backend does not answer. When the run's own probe reaches the backend and gets a vector back, that answer outranks a fault latched earlier: a restarted backend reads as connected, and its NULL rows are re-embedded on the same run. 4. Read not_probed for what it is. check_health(fix=false) does not judge liveness. It can still contact the backend — the dimension check embeds a probe text, the model check asks the backend for its identity, and the node scan asks for token counts — but it reads none of those calls as liveness evidence. It says so — embedding_backend returns not_probed — rather than leaving you to infer health from silence.

A fix=true run returns it too when the probe was answered from the embedding client's five-minute embed cache: the dimension comes back without a request reaching the endpoint, so nothing was learned about it either way. The finding's reason says which of the two happened. A fault a recall already latched is still reported there.

With no backend configured this check is quiet by design. That is a supported configuration, and the advisory surface is where it is raised.

Tuning recall¶

The knobs, in the order you should reach for them:

set_recall_precision(agent_id, precision) — the main policy knob, and under the default fusion modes effectively the only one. It moves the operating point β of the fused quality gate: strict=2.0 (fewer contaminants, more misses), balanced=1.0 (default), lenient=0.5 (fewer misses, more contaminants). It takes effect immediately, with no restart.
calibrate_threshold(agent_id) — re-derives the gate positions from the corpus, with no labels needed. Called with an agent_id (and the fused gate enabled), it calibrates both the vector threshold and the fused gate; called without one, it calibrates the vector threshold only. The value it prints is the vector-side threshold, while under fusion modes the component actually cutting results is the fused gate. Re-run it after the corpus changes substantially, after a bulk rebuild, and after changing the embedding model.
CPERSONA_AUTOCUT_MIN_RESULTS — autocut fires on similarity-scale signals: under confidence scoring, or on a homogeneous raw-cosine list, which is what cascade produces with confidence on or off. It is deliberately inert under rsf and rrf (contract §6), so under the default configuration this knob does nothing. Note that it is the fusion mode that decides that, not the confidence flag.
CPERSONA_FUSED_GATE_ENABLED=false — last resort. Without the gate, filtering falls back to the pool-size heuristic (_adaptive_min_score), which still rejects weak matches: it is a coarser gate, not an open door. What you give up is the operating point measured for this corpus.

Choosing a precision. The trade is asymmetric per use case. For index-style corpora, where recall feeds an AI that can discard irrelevant rows, a miss is worse than a contaminant: run lenient for a few days and step back to balanced only if contamination becomes a real cost. For contamination-sensitive contexts, where recall is injected into a small window, prefer balanced or strict.

When not to rely on recall¶

Probabilistic retrieval can lose. Facts whose absence from context causes harm — the current top-priority decision, standing safety rules — belong in a deterministically injected surface: CLAUDE.md, a system prompt, or an index file your agent always loads. They do not belong primarily in memory.

A useful split: what should fire without being asked goes in deterministic injection; what should be findable when asked goes in memory. Two corollaries follow.

Update decisions by overwrite, not append. When a decision changes, update_memory the old row (it re-embeds automatically). The most reliable way to stop a stale decision from winning recall is for it not to exist in the search space.
lock_memory protects against loss, not against losing a ranking (contract §9).

Prompting recall from Claude Code hooks¶

An agent does not always look in memory before it answers, and an instruction in an always-loaded file is read once and then competes with everything else. A UserPromptSubmit hook fires on every prompt. examples/claude-code-hooks/prompt_hook.py is a minimal one, standard library only: it adds a note telling the agent to call reconstruct for the request, with the prompt as a suggested query, and to confirm "there is no such record" with an exact search rather than a score (contract §12). Copy it to .claude/hooks/ in the project and register it in .claude/settings.json:

{
  "hooks": {
    "UserPromptSubmit": [
      {"hooks": [{"type": "command",
                  "command": "python \"$CLAUDE_PROJECT_DIR/.claude/hooks/prompt_hook.py\"",
                  "timeout": 5}]}
    ]
  }
}

Two mistakes fail silently on Windows. A production user made both and got no recall from the hook for five days:

Read and write UTF-8 yourself. Python's standard streams use the console code page on Windows (cp932 on Japanese Windows), while Claude Code sends and reads UTF-8, so a hook that reads sys.stdin as text fails on a Japanese prompt and adds nothing. The example reads and writes bytes and decodes UTF-8 itself.
Give the script an absolute path. A relative path stops resolving when the session's working directory is a subfolder. $CLAUDE_PROJECT_DIR expands to the project root.

Japanese and CJK corpora¶

Set CPERSONA_RECALL_MODE=rsf. FTS5 tokenizes CJK poorly, and rsf keeps the keyword channel's score magnitude in the merge, which is the signal that compensates. No further CJK-specific settings exist.
Known characteristics of jina-v5-nano — the model this project runs, and the one the reference embedding server downloads by default — on Japanese, confirmed by long-term production use: strong when at least one proper-noun or identifier anchor overlaps between query and memory; weak on pure concept matches with no shared vocabulary. Phrasing queries with a concrete anchor term ("keyword anchoring") is a correct adaptation to the current model, not a workaround to feel bad about.
The model slot is replaceable (CEmbedding provider). If you swap the embedding model, re-embed the corpus in full and then run calibrate_threshold. The server auto-recalibrates on a dimension change, but a same-dimension model swap needs the manual recalibration.

Corpus indexing and sync patterns¶

CPersona's primary design center is memory that accrues from conversation. Using it as a search index over canonical Markdown files works, but two facts shape the correct pattern. CPersona is a passive server — it watches no files, and ingestion is always caller-driven — and dedup is skip, not upsert (contract §5).

Pattern A — rebuild the index as a disposable projection (recommended first).

Give the index a dedicated agent_id (for example md-index). Do not mix index chunks into a conversational agent's memory. A separate agent also keeps the episode boundary machinery out of the picture.
When the source documents change: delete_agent_data(agent_id="md-index"), then re-store every chunk.
Re-run calibrate_threshold after each rebuild, and re-apply set_recall_precision if you use it. delete_agent_data discards that agent's calibration state along with its rows. If the rebuild is a nightly batch, make the recalibration part of the batch.
Cost: re-embedding a few thousand chunks on CPU takes minutes to tens of minutes, depending on the environment. What you buy is no diff logic, and an index that matches the source once the rebuild finishes.

It does not match during one. delete_agent_data commits before the re-store begins, so a recall issued inside that window sees a partial index or none at all. Run rebuilds when nothing is querying, or build under a second agent_id and switch readers over when it is complete.

Pattern B — differential updates with an external content-hash ledger.

store each chunk with a stable msg_id (for example path#heading) and the source document in source.id. The source_id recall argument then doubles as a per-document filter: a value that ends in a letter or digit (docs/a.md) matches that one document exactly, and a value that ends in a delimiter (docs/) matches every document under it.
Keep a caller-side ledger of key → content hash, and process only chunks whose hash changed.
Changed chunks must go through update_memory, or delete_memory plus store. Re-storing changed content under the same msg_id is skipped without a word. Unchanged chunks may be re-submitted blindly, because exact-match dedup guarantees that is harmless.

Prefer A until the corpus is large enough that rebuild time actually hurts. It has no ledger to corrupt and no drift mode.

Scale¶

Growth needs no archival or thinning routine. The vector layer scans a recency window (CPERSONA_MAX_MEMORIES, default 10,000). The FTS and keyword channels reach the full history, and old rows sink through windows and decay rather than being deleted.

To let the vector arm reach rows past the window, set CPERSONA_VECTOR_REACH above it or turn on the far seats (CPERSONA_FAR_SEATS_ENABLED). Widening the window itself also removes the recency prior it provides: measured, it cost 20 NDCG@10 points on recent answers to gain 5 on old ones (contract §4).

The contiguous vector index¶

The local vector scan reads every embedding in its window out of SQLite one row at a time, and that read — not the arithmetic — is where the scan's time goes. The contiguous index is a file beside the database holding the same embeddings in the layout the arithmetic wants, so the scan reads them in one pass. Same rows, same scores, same order: only the latency changes. On the reference machine at 100,000 rows, the vector arm went from 604 ms to 77 ms.

There is one index per table. Memories and episodes are scanned separately, so each has its own file, and each is built and checked on its own. An unindexed episode table costs more per query than an indexed memory table five times its size, so a deployment that builds one should build both.

It is a derived artifact. The database is the only source of truth. The index is a projection of it and is treated the way a cache is treated: not backed up, never repaired, safe to delete at any time. If anything about it looks wrong, delete the file and build again.

It is read through a change log. A row the file holds can change after the build: update_memory replaces its text and vector, the missing-embedding repair fills its vector, a re-tag moves it, a repair rewrites a time, or it is deleted. Triggers log the id of every such row, and a query reads those rows from the database instead of the file, so the answer is the one the scan gives. The log runs only while an index file exists: build turns it on, and the server turns it off at startup when no file is left. An index file written before the log existed is refused as unusable; build it again after upgrading.

delete_agent_data removes it along with the rows it held, but only when the caller's grant covers every agent (always, with no ACL): the file holds every agent's rows, so a grant on one agent does not reach it. After a purge under a grant on one agent the file stays until its next build, and the purged rows are not served from it in the meantime, because recall reads every candidate back from the database. A build the server was running when a purge removed the file discards its output rather than put the purged rows back. The build command runs in a process of its own and is not ordered against a purge, so run it again if a purge happened while it ran.

Building it is an operator action. Nothing in the running server builds or refreshes the index. The entry point is a command, made for a cron job or a systemd timer:

python -m cpersona.vector_index --db "$CPERSONA_DB_PATH" build
python -m cpersona.vector_index --db "$CPERSONA_DB_PATH" --table episodes build
python -m cpersona.vector_index --db "$CPERSONA_DB_PATH" status
python -m cpersona.vector_index --db "$CPERSONA_DB_PATH" --table episodes status

--table selects the file (memories is the default).

build reads the rows without writing them and replaces the index atomically. Its one write is the change log's switch: it turns the log on before it reads, and afterwards drops the log rows every index file present already holds. The server picks the new file up on its next query, with no restart. It exits 0 when built, and 1 when it declined, printing why: a corpus with no embedded rows, one that carries two embedding widths at once (the transient state of a model change; build again when the re-embedding has finished), or a database no server has migrated yet (start the server once).

status reports whether an index is present and usable, how many rows it holds, and how many rows have been written and changed since it was built. It also reports how many rows every query reads from the database instead of the index (rows_read_exactly). That count includes the rows written or changed since the build, and the rows the build named because it could not hold them: rows whose created_at is not in the standard form (excluded), and rows with no embedding yet (unembedded). A named row costs a read only once it has an embedding, and filling missing embeddings is what check_health with fix=true does. So after that repair, rows_read_exactly can grow while rows_since_build stays at 0. It exits 1 when there is no index, and 2 when the file exists but cannot be used, which includes a change log that can no longer vouch for it. Add --json to either for a machine-readable line.

For memories, build also writes the coarse index, a second derived file beside the first (<database>.memories.coarseindex). It holds the same rows, each reduced to one bit per dimension, and it is read only by the reach described in Binary coarse search: the far seats, when turned on, and a time cue's search of its period past the scan window, which since 2.6.4 uses the file by default when it exists. It is handled like the contiguous index: never backed up, never repaired, safe to delete, and removed by a purge under the same rule. With both files, build exits 0 only when both were built. status reports the coarse index under coarse without changing its own exit code, because nothing reads the coarse index unless a setting asks for it.

When a setting does read it, health reports it. With CPERSONA_FAR_SEATS_ENABLED=true or CPERSONA_CUE_COARSE_ENABLED=true, and an agent holding records past the scan window, a recall that cannot use the coarse index reads the stored vector of every one of those records from the database: the answer is the same, and the cost grows with the store instead of the window. check_health raises coarse_index_absent (with the records past the window and the bytes a recall reads), coarse_index_unusable, coarse_index_dimension_drift or coarse_index_log_refused as warnings, and coarse_index_tail_grown as an observation, and says nothing while both settings are off or the store fits the window.

With CPERSONA_CUE_COARSE_ENABLED unset, its default, a time cue asks the file only: without a usable one the part of a period past the window is not searched, and the recall says so in time_cue.remainder. While the far seats are off, no recall then pays for a missing file, so the same four states are reported as observations (info), not warnings. A recall that searches vectors locally, whose scope holds 1,000 or more records with a stored vector past the window and no file, also carries a suggestion, once per session, for the agent to relay to the user. Building the file is never automatic. check_health(checks=["coarse_index"], fix=true) builds the file again, from the first record past the window; it holds every agent's records, so the repair asks for write access to every agent. The findings reach get_session_findings like every other check's, so a monitor can hear of the cost before anyone notices it as latency. Whether a finding leads to the repair is the monitor's decision, not the server's.

What happens between builds. The index knows the highest row id that existed when it was built. Rows written after that are not in it, and are not lost: every query reads them from the database exactly as the scan always did, and merges them with the indexed rows. Rows it holds that changed since are read the same way, through the change log.

So a late rebuild never changes an answer. It only grows the part of each query that is still read row by row, until latency drifts back toward the unindexed figure. Rebuild frequency is a performance setting, not a correctness one. Nightly is a reasonable default, hourly for a corpus that grows fast, and status shows how far behind the index is at any moment: rows_read_exactly.

When the index is not used. The scan the index replaces is still there, and is the fallback. Recall falls back to it — slower, and still correct — when the file is missing, when it fails its own integrity check, when the query vector's width does not match the index, and when the change log cannot vouch for the file: the log was turned off or restarted since the build, a later build dropped changes this file needs, or the schema changed since.

Each of these is reported where an operator reads health. check_health raises vector_index_absent (with the count of rows it would cover) when there is no index, vector_index_unusable when the file cannot be trusted, and vector_index_log_refused (with the reason) when the change log cannot vouch for it — one line per table, each naming its table — and the server logs a warning on every query it had to hand back to the scan.

An index that had been unusable for a week without anyone noticing would otherwise read as "somehow not faster", which is the failure the reporting exists to prevent.

There is no configuration for the index. Its path is derived from CPERSONA_DB_PATH, and the scan window it serves is the same CPERSONA_MAX_MEMORIES the scan uses.

The block index file¶

Unless CPERSONA_BLOCK_INDEX=false, the block arm reads its rows from a third derived file, <database>.blocks.blockindex, instead of reading up to 250,000 rows out of SQLite on every recall. The answer is the same either way (contract §3); only the time differs. Like the other two files it is never backed up, never repaired, safe to delete, and removed by a purge under the same rule.

The file takes 28 bytes per block row plus one bit per dimension of the model: 124 bytes with a 768-dimension model, 156 with a 1,024-dimension one. On Linux and macOS it is mapped, so the operating system pages in what a recall reads; on Windows it is read into memory whole. To go without it, set CPERSONA_BLOCK_INDEX=false and restart: the server stops the log at startup, recalls read SQLite, and the file can be deleted.

You do not build it by hand. While the setting is on, the server keeps a log of the records whose blocks change, and its queue builds the file at startup and after it writes blocks, whenever the file is absent, unusable or too far behind that log; one build is queued at a time. Until a build finishes, recalls read SQLite. With the task queue disabled, or to build it now:

python -m cpersona.block_index --db /path/to/cpersona.db build
python -m cpersona.block_index --db /path/to/cpersona.db status

status says whether a recall can read the file now (current) and how far the log has moved since the build (changes_since_build). It exits 1 when there is no file, and 2 when the file exists but cannot be used. To see which read served a particular recall, ask for its trace (trace=true): block_source says file, or sqlite with the reason the file was not read (recall trace).

check_health reports block_index_absent, block_index_unusable and block_index_behind as observations while the setting is on: every recall still gets its answer, from SQLite. With the setting off, the one finding is block_log_without_index, a log left running (for instance by a build from the command line) that adds a row to every block write with nothing to read it; check_health(checks=["block_index"], fix=true) stops it, and so does starting the server with the setting off.

Maintenance cadence¶

Monthly: check_health(agent_id, fix=true) for deterministic integrity checks and auto-repair, deep_check(agent_id, fix=true) for the semantic pass, and a logical export_memories backup.
After corpus upheaval (bulk import, rebuild, model change): calibrate_threshold(agent_id).
On a timer, if the vector index is in use: python -m cpersona.vector_index build, and the same with --table episodes. Nightly by default; see the contiguous vector index for what a late rebuild costs (latency, never correctness).
Version upgrades: the schema migrates itself forward. Calibration is fingerprinted to the scoring function and embedding dimension, and the server recalibrates at first boot when either changes (v2.5.2+).