Aller au contenu

Sense 37: The Snapshot — A Witness Made Citable

Ce contenu n’est pas encore disponible dans votre langue.

Sense 37 · Forming · Last touched 2026-10-07

  • last_verified: 2026-07-27

Synced from docs/design/sense_37_the_snapshot.md in the engine repo — that’s the source; this page is a build-time mirror.

A working KLS witnesses what is. A snapshot witnesses what was, as of when we said it was. Same substrate, different commitment.

Photograph, not livestream.

The KLS file format is one shape; the snapshot is a posture you can take with it.

This Sense draws the line between the witness (any KLS, by virtue of carrying its own provenance — see the in-progress Sense of the KLS as Witness) and the snapshot (a witness someone deliberately froze in time so it could be cited later).

The companion KLS as Witness Sense, in draft on the Bell side, names the structural property: every KLS carries breadcrumbs (git_branch, git_commit, git_dirty, engine_version), the source AFS packed as a blob, an extract audit trail, and a build log. This Sense names what deliberately stopping the clock does to those breadcrumbs.

Working KLSSnapshot KLS
JobWitness what is.Witness what was.
LifecycleOverwritten on every jin make.Immutable. Refuses overwrite.
Filename<pack>_<tenant>_kls.duckdb<pack>_<tenant>_<tag>_kls.duckdb
Citation”The model as of now.” Moves.”The model as of <tag>.” Pinned.
Common usesLive exploration, daily authoring loop.Auditor citations, customer handoffs, before/after comparisons, scenario reference points.
Dirty treeFine — author is mid-edit.Warns by default; --strict-tree refuses.

Both are KLSes. Both witness. The difference is the commitment to be cited later.

Three properties separate a snapshot from “the KLS file that happened to exist when I copied it.”

jin make --snapshot <tag> writes a file the engine refuses to overwrite. A second invocation with the same <tag> errors out with E_TARGET_ALREADY_EXISTS. There is no --force flag (deliberately): re-deriving a snapshot under the same tag would silently shift what every prior citation points at.

If the snapshot turns out to be wrong, the answer isn’t to overwrite it — it’s to take a new snapshot with a new tag (e.g. post-audit-v2) and update the citation. The old snapshot stays as record.

The snapshot filename carries pack and tenant, so the file is unambiguous when copied out of its directory:

numetrix_rmc_post-audit_kls.duckdb

Not post-audit_kls.duckdb (legacy shape, still readable for back-compat) — the post-audit of what? Of which tenant? The new convention answers that without needing the directory context.

This matters because snapshots get carried places: attached to an audit response, sent to a partner, archived for compliance, mounted by Compare’s diff loupe. The filename should travel with the file’s meaning.

Default behaviour: jin make --snapshot <tag> on a dirty AFS working tree warns and proceeds, recording git_dirty=true in the snapshot’s _jinflow_snapshot table. The author knows their tree wasn’t clean; the snapshot honestly records that fact.

jin make --snapshot <tag> --strict-tree: refuse to snapshot a dirty tree. The snapshot can only be cited as “commit X on branch Y” if there are no uncommitted bytes between commit X and what got baked.

The strict gate is the right default for the cloud-publish path (once jin make runs as a worker producing customer-facing artifacts). For local exploration, the warning is enough.

A scenario branch is the workspace for what if? explorations:

main ← the canonical analytical framework
scenario/lower-thresholds ← "what if we tightened revenue leakage?"
scenario/zeta-onboarding ← "does the pipeline work for the new tenant?"

The scenario branch is alive while it’s being explored — commits land, the analytical state moves, the user iterates. But a moving branch can’t be cited. “The model on scenario/lower-thresholds” points at HEAD, which moves; tomorrow it’ll point somewhere else.

The snapshot solves this. jin make --snapshot 2026q2-thresholds on the scenario branch produces an immutable file whose name and breadcrumbs answer:

  • Which tenant? The filename’s <pack>_<tenant>_ prefix.
  • Which branch? The snapshot table’s git_branch.
  • Which commit? The snapshot table’s git_commit.
  • Which framework? The packed _jinflow_afs_archive.

Now the scenario is referenceable. You can:

  • Compare 2026q2-thresholds against the main snapshot via the Diff Loupe — see exactly what the scenario changes.
  • Hand the snapshot to a stakeholder — they get a frozen artifact they can explore independently.
  • Re-derive the snapshot weeks later — the AFS archive carries everything needed to rebuild the same KLS deterministically.
  • Cite “the model as of 2026-Q2 thresholds” in any external document, knowing the citation won’t drift.

Scenarios need snapshots to be referenceable; snapshots don’t need scenarios to be useful. A snapshot of main is just as valid as a snapshot of scenario/X — it’s a citation of the current canonical state. The scenarios-and-snapshots pairing is strongest where scenarios are most active (experimentation, counterfactual analysis, customer-specific tuning), but the snapshot primitive lives below that.

Beyond the scenario case:

  • Audit citation. An auditor asks “what did the report say on 2026-03-08?”. Without snapshots, you have to reconstruct from git history + a fresh build (risky — engine versions move). With snapshots, make --snapshot 20260308-1430 at the time pins the answer.

  • Customer handoff. A snapshot delivered to a customer is a self-contained artifact: the KLS data + the AFS framework + the breadcrumbs. The customer can verify the derivation if they care; they can also just open it and not care.

  • Before / after comparisons. The Diff Loupe surface in the JinDesk takes two snapshots and surfaces what changed. A natural workflow: snapshot before a change, make the change, snapshot after, run the comparison.

  • Pinned demo state. A demo-stable snapshot can be the thing a public JinDesk instance serves, while the working KLS keeps moving. Cloud-make + R2 publish makes this routine once the dirty-tree gate is enforced.

A working KLS and a snapshot KLS share the same file format, the same DuckDB schema, the same _jinflow_snapshot table. They even share the same writer (_stamp_kls). At the bytes level they’re indistinguishable from each other.

What makes the snapshot a snapshot isn’t a property of the file — it’s a property of the commitment:

  • The author committed to a tag (named the moment).
  • The system committed to immutability (refuses overwrite).
  • The framework committed to citability (the tag becomes the reference, not a path or an inode).

That commitment shape is the Sense. The file is incidental.

  • Created by: jin make --snapshot <tag> (named) or jin make --snapshot (auto-tag YYYYMMDD-HHMM).
  • Refused overwrite: the engine errors out if a file at the target path already exists. Choose a new tag.
  • Strict-tree gate: --strict-tree refuses dirty AFS working trees. Default warns and records git_dirty=true.
  • Filename convention: <pack>_<tenant>_<tag>_kls.duckdb. Legacy <tag>_kls.duckdb still readable for back-compat; newly-created snapshots use the prefixed form.
  • Listed via: jin ls --snapshots [tenant], surfaced in the JinDesk’s version picker (Compare page).
  • Opened via: jinflow desk --snapshot <tag> or by selecting the tag in the version picker.
  • The KLS as Witness (in draft, Bell sister): the broader concept this Sense specializes. A snapshot is a witness made deliberate.
  • Sense 42: The Landscape: the system surface as a vantage; the Garden is the structural view within. Each stage of the Garden can in principle cite a specific snapshot to show “here’s what the model looked like at this stage’s moment”.
  • Operating modes (docs/design/operating_modes.md): the Make-as-a-Service mode (Mode 5) depends on snapshots being citation-safe — the cloud worker must produce snapshots whose provenance is verifiable. --strict-tree is the gate that makes that safe.
  • jin scenario rebase — a workflow command to propagate an edit on main into every active scenario branch. Today’s workflow is git cherry-pick per scenario. Not blocking; the pairing of snapshots and scenarios works without it.
  • Snapshot drift dashboard at /{tenant}/scenarios/{branch}/drift: counts how far a scenario has diverged from main. Parked until scenarios are in real use. See letter-to-atelier-sister-2.md Phase 2.
  • Cloud-publish gate: the cloud Make-as-a-Service worker will refuse to publish any snapshot whose git_dirty=true. Hooks into --strict-tree once the worker is wired.

The snapshot primitive works locally end-to-end today. To carry it across the cloud Fly + R2 + KLS-store boundary so Compare/Diff Loupe becomes useful for cloud-hosted tenants, four pieces still need to land. Each one is named here so the work can be picked up as a unit.

A1. jinflow r2 sync learns to push snapshots. Today the sync walks {pack}/{tenant}/store/{pack}_{tenant}_kls.duckdb only. Snapshots (<tag>_kls.duckdb / now <pack>_<tenant>_<tag>_kls.duckdb) are skipped. Add --include-snapshots per-call and include_snapshots: true per profile in ~/.jinflow/cloud.yml. Optionally snapshot_filter: [v0.3-baseline, v0.4] so dev-noise snapshots stay off the wire. File: jinflow/cli/commands/r2.py.

A2. klsStore.ts learns to discover snapshots. Extend KlsEntry with kind: 'working' | 'snapshot', parentCompositeId: 'pack.tenant', tag: string. One ATTACH loop, one cache, one ETag layer. Catalog naming: working stays {pack}_{tenant}_kls; snapshot becomes {pack}_{tenant}__{tag}_kls (double underscore so tag-internal underscores don’t collide). findKlsEntry() keeps returning only kind === 'working'; a sibling findKlsAndSnapshots(parentId) serves the Compare page. Guard readTenantMetadata() to skip snapshots (their metadata is stale by design). File: explorer/src/lib/server/klsStore.ts.

A3. Compare page abstracts local-vs-cloud discovery. Branch on isKlsStoreMode(): local keeps fs.readdirSync, cloud calls getKlsEntries() filtered by parentCompositeId. Refactor queryStats() to accept either (catalog, schema) (cloud) or rely on setRequestContext (local). Tiny helper: qualified(entry) → { catalog, schema }. File: explorer/src/routes/[tenant]/compare/+page.server.ts.

A4. Compare sub-endpoints. Same seam — resolveKlsTarget(tag) returns { mode: 'file', path } (local) or { mode: 'catalog', catalog, schema } (cloud). deltas and dbt-docs-diff are trivial catalog-qualified SQL swaps; afs-diff is the riskier one because it selects a BLOB from the AFS archive and unzips to /tmp. Verify that DuckDB’s node binding returns BLOBs as Buffer against the shared in-memory DB; if not, sidestep via COPY (SELECT archive FROM …) TO 'file.zip'. Keep the existing unzip / hash / categorize loop verbatim — only the source-of-bytes path changes. Files: explorer/src/routes/[tenant]/compare/{deltas,afs-diff,dbt-docs-diff}/+server.ts.

  • ATTACH count growth. Each tenant goes from 1 catalog to N+1. 5 tenants × 3 snapshots each = ~20 catalogs in one in-memory DB. Spike before committing. Mitigation if it hurts: lazy-attach only on Compare requests; LRU detach.
  • afs-diff BLOB extraction in cloud. Single biggest unknown in A4 — see the verify note above.
  • Deploy ordering on demo day. A2-A4 (JinDesk) must reach the cloud JinDesk before A1 sync pushes snapshots to R2; otherwise snapshots land but are silently ignored, making it ambiguous whether discovery is broken. Bump KLS_SYNC_INTERVAL to 60s on demo day so a fresh snapshot appears without a manual restart.
  • SIS is mutable and not snapshotted. A snapshot freezes the AFS + KLS but never the Session Interaction Store. Documented here so the Diff Loupe note doesn’t promise notebook rollback.
  • Demo-specific content (which two snapshots a given tenant uses to tell a story, what the welcome-tour says about Compare). That’s tenant authoring work — lives in the tenant AFS, not in the Sense.

Not yet manifest in the engine. Status: proposed. See the implementation phases / open questions above for the path from this paper to running code.


Numerical neighbors: ← Sense 36: The Lineage — A Tenant Remembers Where It Came From · Sense 38: The Scribe — Who Carries the Author’s Action Across a Credential Boundary →

jazzisnow jinflow is a jazzisnow product
v0.65.0 · built 2026-11-13 10:20 UTC