Sense 21: The Heartbeat — keeping the system in rhythm

Sense 21 · Folded in · Last touched 2026-08-20

Folded into Sense sense-32-the-chores.

• last_verified: 2026-07-27

Synced from docs/design/sense_21_the_heartbeat.md in the engine repo — that’s the source; this page is a build-time mirror.

Information captured anywhere must reach everyone who needs it. Everyone who needs to know what changed must be told. Without breaking the determinism that makes jin make trustworthy.

A heartbeat is regular, observable, one-way per beat. The system needs one.

Status: proposed Author: the owner + Claude (conversation, 2026-04-27 — triggered by the six-day-stranded SIS cell incident)

---

What pulled us in

On 2026-04-21 at 15:42 UTC, the owner was on a call with a viewer and a subject-matter expert. The owner typed four answers into a notebook cell in JinDesk. Six days later — on 2026-04-27 — the cell was still stranded in SIS, never written to AFS, invisible in the JinDesk’s notebook view, and at risk of being lost on the next SIS reset.

Nobody had been alerted. The cell was found by accident, asking “why does JinDesk say there are unpublished changes?”

The cell was recovered. But the incident named a coherent design gap:

Today’s reality:

• jin make is one-way, AFS → KLS, deterministic. Good.
• JinDesk captures user input into SIS, fast. Good.
• The two layers don’t talk to each other reliably. Bad.
• When AFS changes upstream, nobody is told. Bad.
• When SIS has unpublished content, only one page of JinDesk knows. Bad.
• jin sis flush deletes settled SIS rows — its name suggests it publishes, it doesn’t, the doc comment in cli/__init__.py:2575 says it does, that comment lies. Bad.

Today’s broken paths:

Direction	Mechanism today
AFS → KLS	jin make (works when it runs; change-detection is too coarse — see Symptom B below)
AFS → SIS	Lazy hydration on first edit (one-shot, never refreshes)
SIS → AFS	No working path. Was Phase 1d of make, disabled 2026-04-16 due to the duckdb-segfault bug.
AFS upstream → user awareness	None. User must remember to git pull.
SIS unpublished → user awareness	Per-page banner only. Easy to miss.

This is not one bug. It’s a missing organ.

Two arms, one heart

The heartbeat has two arms, joined at Beat 1.

The system-data arm carries data from the source-system inbox up through extraction, DLZ-sync, bronze, silver, gold. Its beats are deterministic and largely already implemented (see extractor_discipline.md): every extraction is contract-pinned by SHA-256, every step is logged to afs/state/extract_log.jsonl and afs/log/build_log.jsonl. What’s missing is a place to see the rhythm — nobody can tell at a glance whether the last extract succeeded, whether the source bytes still match the pinned hashes, whether bronze is in step with what landed in source/, whether the latest gold model materialised on the same commit it should have.

The human-edit arm carries information from user edits in the JinDesk up through SIS, publish-back, sync, notify, reconcile. This arm is the one the original Sense (the lost-cell incident) focused on. It’s mostly not implemented.

The two arms meet at Beat 1 (Build) — jin make consumes both the extracted source bytes (system arm) and the published AFS YAMLs (human arm) and produces the KLS that downstream sees. A failure in either arm shows up at Beat 1 as either “no data” or “stale data”; the Heartbeat is the discipline that makes those failures visible before they become bugs in someone’s analysis.

Symptoms in the wild

A growing log of incidents that share the same root cause. Each one is a limb of the same missing organ.

Symptom A — the six-day-stranded SIS cell (2026-04-21 → 2026-04-27)

The trigger for this paper. The owner captured a viewer’s and an SME’s answers to four temporal-pricing questions live in JinDesk; the cell sat in SIS for six days, never reaching AFS, invisible in the notebook view, recovered only by accident. Beat 3 (publish) is the missing organ.

Recovery: hand-transcription into AFS YAML. Clean fix requires the publish-back to be a real beat with a command, a log, and a UI surface.

Symptom B — coarse change-detection skips meaningful edits (2026-04-27)

While decomposing the GADOVIST class of pricing errors, an edit to nb_rmc_pricing_model_workshop.yaml (adding two new cells: a four-class taxonomy + a new structural action for an SME) was correctly compiled into the dbt model SQL — but jin make’s incremental change-detection declared “data unchanged” and skipped the dbt run that would have re-materialised the note_registry table. The KLS table reflected the old YAML state. --clean forced a full rebuild and the new content landed.

This is the same heartbeat principle, viewed from a different angle: Beat 1 (build) is one-way and deterministic, but its skip decision is incorrect. A change happened; the change-detector missed it; the beat didn’t fire when it should have. That’s an arrhythmia.

Root cause hypothesis: the make hash currently looks at AFS YAML mtime or content hash at a level that doesn’t propagate through the compiled- SQL → dbt-target pathway. The compiled SQL changed (we verified by grep); the materialised table didn’t. Some link between them is optimised out.

Fix space:

• Option A: make hashes the compiled dbt SQL output, not the YAML. More accurate but more expensive.
• Option B: any YAML change in notebook/, signals/, etc. invalidates the corresponding dbt run unconditionally. Simpler but loses the optimisation.
• Option C: dbt’s own state-aware run (--select state:modified) with proper manifest tracking. Potentially the cleanest; needs the prior-manifest to live somewhere reliable.

Phase 1 of the Heartbeat must include a fix for this — otherwise Beat 1 itself is unreliable, which makes everything else moot.

Symptom C — three flavors of modified_at, only one is honest (2026-04-27)

While adding a “Source note (updated DATE)” column to the actions board — the cheap-now precursor to per-cell timestamps — surfaced that modified_at means three different things in three different places, and only one is auto-tracked.

Surface	Source	Honesty
Dimensions list (/dimensions)	YAML explicit field OR file mtime fallback (jinflow/afs.py:1404)	Honest — auto-tracked
Notebook list / actions board	YAML explicit field OR author.date fallback OR compile-time (scripts/notecompile.py:79)	Manual claim — goes stale if the author doesn’t bump it
SIS-side notes	DuckDB row updated_at, auto-stamped on every write	Honest — auto-tracked

Concretely: a notebook YAML can collect 12 new cells without its modified_at changing, because the field is treated as authorial metadata rather than a system observation. Meanwhile an entity YAML edited the same way picks up a fresh file mtime automatically.

This is the same heartbeat principle viewed from a third angle: a beat happened (Beat 2 capture: edit YAML), the system ought to record it, and only some surfaces do. The dimensions surface is honest; the notebook surface is not. Awareness (Beat 5) cannot be built on inconsistent timestamps.

Phase 1 of the Heartbeat must include normalising modified_at semantics — picking one rule and applying it everywhere. Lean: file-mtime fallback (the dimensions rule) for all YAML-backed artifacts, plus per-cell auto-stamping for cell-level edits as part of the cell schema work.

Symptom D — placeholder

Future incidents will land here. Each one is evidence that the Heartbeat is the right framing.

---

The Vision

The system is alive. It has parts that produce information and parts that consume it, and these parts are distributed across machines and people. For the system to function, information must move in discrete, observable, predictable beats.

A heartbeat is:

• Regular — it happens on a cadence, not at random.
• Observable — every beat has a log. You can ask “did it happen?”
• One-way per beat — each beat does exactly one thing. No circles.
• Detectable when missed — the absence of a beat is itself a signal.

The system already has some beats:

• Build (Beat 1) — jin make. AFS → KLS. Works.
• Capture (Beat 2) — JinDesk writes → SIS. Works.
• Sync down (Beat 4) — git pull of the AFS. Works (manual).

It’s missing:

• Publish (Beat 3) — SIS → AFS. The lost-cell beat.
• Notify (Beat 5) — change → user awareness. The “did a teammate push?” beat.
• Reconcile (Beat 6) — when two beats produced conflicting state, resolve it deterministically. The beat that lets multiple humans edit safely.

The Heartbeat is the discipline that names these beats, gives each a command, a log, a cadence, and a UI surface — and keeps jin make non-circular by ensuring every other beat lives outside it.

---

What Already Exists

A meaningful amount of the plumbing is in place. We’re not building from scratch.

• AFS — git-backed, durable, source of truth. Per-pack and per-tenant. Already supports push/pull through the owner’s standard git flow (jinflow afs push).
• KLS — DuckDB build output of jin make. Read-only, snapshottable, shippable as a single file.
• SIS — DuckDB owned by JinDesk writes. Per-tenant. Holds notes, bookmarks, recents, navigation_journal. Per consistency_plan.md Phase 2 (shipped 2026-04-13): make is read-only on SIS; JinDesk owns all writes + schema lifecycle.
• Lazy hydration — when a user first edits a cell of a KLS-only note, ensureSisNoteHydrated() (in explorer/src/lib/server/queries/notes.ts:26) copies the cells JSON from KLS into SIS. The note then exists in both places. Subsequent edits update the SIS-side copy.
• Drift detection — the notebook page’s +page.server.ts checks whether SIS has cells that KLS doesn’t and shows an amber banner when so. Per-page only.
• jinflow sis flush — deletes SIS rows whose canonicalized cells match AFS YAML. Cleanup, not publish.
• jinflow sis prune — deletes old navigation_journal rows beyond an age cutoff.
• The shared notes branch — git branch separate from main that collects per-note YAML state. Referenced by notecompile.py:378 and the jin make output (Synced 1 note(s) to shared notes branch).

What’s missing is the discipline that connects them. The Heartbeat is that discipline.

---

The Heartbeat Protocol

Six beats. Each has a command, a log, a cadence, and a UI surface.

Beat 1 — Build

Command: jin make Direction: AFS → KLS Cadence: on-demand or post-publish trigger Log: afs/log/build_log.jsonl (already shipped) UI surface: the build banner in JinDesk Determinism: strict. Same AFS commit → same KLS bytes (modulo deterministic-build work in flight).

Beat 2 — Capture

Source: JinDesk user actions (edit cell, drop bookmark, navigate) Direction: JinDesk → SIS Cadence: synchronous on user action Log: SIS table notes.updated_at, bookmarks.created_at, navigation_journal.visited_at UI surface: none today; should be a Recent Edits view Determinism: doesn’t apply. SIS is the messenger.

Beat 3 — Publish (the missing organ)

Command: jin publish (NEW) Direction: SIS → AFS Cadence: on-demand initially; later periodic (cron) or pre-make hook Log: afs/log/publish_log.jsonl (NEW) UI surface: “Unpublished changes — publish?” banner site-wide, not per-page Mechanic: for each SIS row that differs from its AFS YAML counterpart, rewrite the YAML to absorb the SIS-side changes. The YAML rewrite is content-addressable and structurally identical to hand-edits.

The duckdb-segfault constraint: the existing Phase 1d was disabled because Python’s bundled DuckDB and Node’s duckdb-async use different native versions, and Python opening a SIS that Node last wrote crashed the frozen PyInstaller binary. Two viable fixes:

• (a) Spawn a Node helper from the Python CLI. Bundle a small Node binary in the deploy and shell out to it for SIS reads in the publish step. Adds dependency weight.
• (b) Use @duckdb/node-api from a parallel Python process. The same library JinDesk migrated to in 2026-04-15. We’d need a thin RPC layer.
• (c) Migrate Python’s bundled DuckDB to match JinDesk’s. Pin versions across both runtimes. Fragile; new releases break it.

Recommended: (a). Smallest blast radius; Node is already a deploy dependency.

Beat 4 — Sync down

Command: git pull (existing) or future jin sync Direction: AFS-remote → AFS-local Cadence: on-demand initially; later automatic Log: git reflog (free) UI surface: the next page load shows a “N new commits since you last pulled” banner

Beat 5 — Notify (the missing nervous system)

Command: jin pulse (NEW) — or, more realistically, an JinDesk service that polls Direction: AFS-state, SIS-state → user-aware UI Cadence: every page load + on user-trigger refresh Log: _<tenant>.notification_journal table UI surface: a system-wide bell icon with unread counts, plus inline banners on relevant pages

Three notification classes:

Class	Trigger	Example
Inhale	new AFS commits available	”A teammate pushed 3 commits at 09:14. Pull?”
Exhale	SIS has unpublished content	”5 cells in 2 notes haven’t been published. Publish?”
Echo	a cell I co-author was edited by someone else	”An SME edited your cell q5_gadovist 12 minutes ago”

Beat 6 — Reconcile (the conflict resolver)

Trigger: Beat 3 (publish) detects that the SIS row’s pre-edit state diverged from the current AFS state — i.e. someone else edited the same cell while you were editing yours. Direction: structural merge or human prompt Log: afs/log/reconcile_log.jsonl UI surface: a conflict-resolution dialog (initially: prompt user to choose; later: structural merge for non-overlapping fields)

For Phase 1, “reconcile” is a coarse strategy: last-publish-wins, with a warning. Cells edited concurrently produce a publish-log entry naming the loser. Real conflict resolution is Phase 4.

The system-data arm — Beats 0 and 0b, sharing Beat 1

The six beats above describe the human-edit arm. The system-data arm adds its own upstream beats, all already implemented (the discipline they realise is extractor_discipline.md); the Heartbeat just gives them their place in the protocol.

Beat 0 — Extract

Command: jin make Phase 0 (also jin make --extract-only, jin inspect --check) Source: extractor scripts declared in afs/scripts/pipeline.yml, each pinned by SHA-256 against a source-system file or feed Direction: source system → DLZ → source/ Cadence: on-demand; incremental — skipped when the pinned hash still matches and structure contracts still pass Log: afs/state/extract_log.jsonl (append-only) UI surface: the Pulse page (one row per pipeline.yml entry showing last extracted, hash status, expected vs observed) Determinism: the configuration IS the contract. See extractor_discipline.md.

Beat 0b — DLZ-sync

Command: jin make Phase 0b Source: the Data Landing Zone — a sibling git repo at dlz_root Direction: dlz_root/<pack>/<tenant>/... → source/ Cadence: every jin make, fast-skipped when nothing changed Log: part of the build journal entry UI surface: the Pulse page (a “DLZ in sync” indicator + per-file delta when stale)

Beat 1 (shared) — Build

Same Beat 1 as the human arm. The system arm’s contribution: feed the extracted source bytes through validate → enrich → compile → dbt → stamp → snapshot. A failure in any sub-phase shows up on the Pulse page as a stalled heartbeat — extract and DLZ-sync keep beating while the last successful build timestamp stops advancing.

The system arm has no Beat 3 (publish) or Beat 6 (reconcile) — by construction. Source data is one-way: the extractor’s SHA-pinned contract is the trust boundary, and there is no “user edits the source data back up.” The system arm does share Beat 5 (notify) — a failed extract, a hash drift, or a structure-contract violation each constitute a notification class on the Pulse page.

---

The Vocabulary

The five proper-noun verbs:

Verb	Beat	What it means
make	1	Build the KLS from the AFS. Determinism guaranteed.
publish	3	Send SIS edits up to AFS. From-now-on the durable truth.
sync	4	Pull AFS state from upstream. Brings in others’ publishes.
pulse	5	Tell the user what changed. Both directions.
reconcile	6	Settle conflicts when two publishes touched the same artifact.

These names mean only this. Don’t use “publish” for “make a Slack message.” Don’t use “pulse” for “show a loading spinner.”

Related nouns we already use:

• AFS = the durable layer (truth)
• KLS = the deterministic stamp (read-only build output)
• SIS = the messenger (mutable user state)
• Beat = one observable, atomic flow between layers

New nouns introduced by the Heartbeat:

• inhale = the receiving side of sync (informal noun)
• exhale = the sending side of publish (informal noun)
• echo = a notification about your own work being touched by someone else
• rhythm = the cadence at which beats fire (per-mode)

Heartbeat / Pulse / pulse — three uses of one word

The word “pulse” appears in three places. To prevent confusion:

Term	Part of speech	What it means
Heartbeat	proper noun, the discipline	This whole Sense. The protocol, the organ, the principle that beats.
Pulse	proper noun, the page	JinDesk surface at /system/pulse. Where you see the heartbeat — animated beats, per-artifact freshness, drift indicators, last-extracted-at, stalled-beat alerts.
jin pulse	CLI verb (Beat 5)	The command that fires the notify-beat and writes entries to the notification journal that the Pulse page renders.

The page is named after what you do with it: you take a pulse to check that the system is alive and in rhythm. The CLI verb is one of the things that contributes to what the page shows. The Sense itself stays “Heartbeat” — the organ, the discipline, the principle that makes the beats observable.

---

The Cheap Beat — keeping “no change” actually free

The cadence rule for every beat reads “every jin make, fast-skipped when nothing changed.” For the rule to hold in practice, each phase needs (a) a freshness fingerprint that costs less than the work it gates, (b) durable state to remember the previous fingerprint, and (c) an explicit override for when the user knows the cache is stale.

When any of those three is missing, the heartbeat gets expensive at rest. The cycle runs through phases that produce bit-identical output to the last cycle — just slower. A heartbeat that’s expensive at rest can’t beat regularly without burning the body it lives in.

Three failure modes, three fixes (2026-06-02):

Failure mode	Phase	Why it happened	Fix
Missing freshness state	Phase 0b for Sense 31 channel tenants	Phase 0b skips for channel-sourced tenants, so the per-file source manifest is never written, so _source_csvs_changed defaults to True every cycle. Forced full dbt rebuild on every run.	Save a _dir_stat_hash fingerprint of source/ when no _manifest.json exists. New helper sorts (relpath, size, mtime_ns) into one digest — millisecond stat()s instead of byte reads (rmc’s source is 52 GB; byte-hashing would be slower than the rebuild).
Unconditional re-emit	Phase 4d PDF extraction	Every cycle re-read every PDF blob from _{tid}.report_pdfs and rewrote it to disk. When data_changed=False the on-disk PDFs were already bit-identical.	Gate on the existing any_dbt_ran signal — same gate every other Phase 4 step already uses.
Uncached network round-trip	Phase 1b+ git fetch origin	Every cycle ran git fetch origin + rev-list for ~1–3 s, even on a back-to-back jin make.	Cache last_fetched_at in build_meta.json, skip when the cache window is still warm (default 120 s). --force-fetch overrides; --pull and --offline already bypass.

The pattern. Each phase declares its freshness contract:

needs_to_run = not incremental
            or input_fingerprint != saved_fingerprint
            or explicit_override_flag

input_fingerprint is the cheapest faithful signal — a stat-hash, a content-hash, a time-window. saved_fingerprint lives in build_meta.json (one journal, multiple keys). The override matters: without --force-fetch, --reset, etc., the only way to bypass a sticky cache is to mutate state by hand, which silently violates the heartbeat’s “every beat is observable” contract.

The rule of thumb. If you write a phase that emits to disk, ask two questions before merging:

1. On a no-op cycle, does this phase produce the same bytes as last cycle? If yes, it should skip on that cycle.
2. On a no-op cycle, does this phase still cost more than reading one stat() result? If yes, the freshness check itself is too expensive — find a cheaper signal.

Without those questions, the “fast-skipped when nothing changed” cadence line becomes aspirational. With them, the heartbeat stays cheap and the body stays cool.

---

Operation Modes — How the Heartbeat Works in Each

The five deployment topologies (cf. operating_modes.md) give the Heartbeat five different shapes. The principle is the same; the implementation per beat varies.

Mode 1 — Local

• KLS, SIS, AFS-local: all on developer’s laptop
• AFS-remote: shared git
• Beats:
  • Make: trivial, on the same machine
  • Publish: SIS → AFS (file paths within ~/.jinflow/live/...)
  • Sync: git pull of AFS
  • Pulse: per-page-load checks against local file mtimes
  • Reconcile: rare; only if user edits in two places

This is the simple case. The Heartbeat exists but barely beats.

Mode 2 — P2P2P (Owner-on-Laptop, Viewers-via-Tunnel)

The hard case.

• KLS, SIS-owner: on owner’s laptop
• Viewers: see JinDesk in browser, queries route via Cloudflare Tunnel to owner’s running proxy
• AFS-remote: shared git

The challenge: when a viewer edits a cell, where do their SIS writes land? Three options:

1. Block writes when owner is offline — viewers get read-only capability. Loses data.
2. Buffer writes per-viewer locally — each viewer maintains a tiny “shadow SIS” in browser localStorage; flushes to owner when tunnel reconnects. Best of both worlds, complex.
3. Always-on tunnel — viewers’ writes go to owner’s SIS in real time. Requires owner to keep machine on. Easy to implement but fragile.

Recommendation: Phase 1 is option 3 (always-on tunnel; assume owner is around). Phase 4 is option 2 (shadow SIS for resilience).

The publish in P2P2P: publish runs on owner’s machine only. Owner is the only person who can write to AFS for this tenant. This is a feature: it preserves the P2P2P sovereignty principle (the data’s gatekeeper is the human at the keyboard).

The pulse in P2P2P: notifications are bidirectional via the tunnel. Tunnel disconnects manifest as “viewer last seen 4 min ago” banners. The pulse is also a heartbeat in the medical sense: its absence detects the system going dark.

Mode 3 — Cloud (R2)

• KLS: in Cloudflare R2 (read-only object)
• SIS: needs a hosted database (DuckDB-on-Cloudflare, D1, or Postgres) — new piece
• AFS-remote: shared git

The challenge: R2 is read-only. SIS can’t live there. Where does it go?

Options for hosted SIS:

Option	Pros	Cons
Postgres on Fly	Mature, transactions	Adds ops surface
Cloudflare D1	Native Workers integration	SQL dialect mismatch with DuckDB
DuckDB on R2 + write-coordinator	Same engine as KLS	We’d build write coordination ourselves
Hosted DuckDB (e.g. MotherDuck)	Drop-in	Vendor lock-in

This is an open question for Phase 5. Until decided, Cloud-mode deployments tolerate SIS-only state on each user’s browser (localStorage) and resync when the user returns to a known device.

Mode 4 — Semi-Cloud

• KLS: on owner’s machine (data sovereignty)
• SIS: in cloud (browser-accessible from anywhere)
• AFS-remote: shared git

The challenge: publish-back goes from cloud SIS to owner’s local AFS. The AFS is local; the SIS is remote. The bridge needs to either:

1. Webhook from cloud SIS to owner’s machine — owner runs a tiny listener service. Owner’s machine must be reachable.
2. Pull-publish on owner’s machine — owner periodically (or on-demand) pulls cloud SIS state and runs jin publish locally. Simpler, but introduces lag.

Recommendation: option 2 for Phase 1. Adds jin pull-sis (or just jin sync extended) which fetches cloud SIS state into a local mirror, then jin publish runs from there.

Mode 5 — Make-as-a-Service

• KLS: built by cloud worker, lives in R2
• SIS: cloud-hosted (whichever Phase 5 picks)
• AFS-remote: shared git
• Build trigger: git push to AFS triggers cloud worker

The fully-cloud case. The Heartbeat is centralised. Publish runs as a cloud worker too, on a cron, against the central SIS. Identity & Passes carry the per-user authority for who’s allowed to publish.

This is the simplest implementation of the Heartbeat — everything in one cloud — but the slowest organizationally (we’re not there yet).

---

The Four Original Concerns, Mapped

The conversation that triggered this Sense named four concerns. Here they are, mapped to beats:

Concern	Maps to	Phase
Timely exchange of information	Beat 3 (publish) + Beat 4 (sync)	Phase 1
Notify users of relevant changes	Beat 5 (pulse)	Phase 2
Full roundtrip of bookmarks/notes/recents without headache	Beats 2+3+4 working consistently	Phase 1 + Phase 3
Make stays non-circular	Beat 1 stays read-only; everything else lives outside	Architecture, not phase

Plus the silent ones the conversation didn’t name:

Concern	Maps to	Phase
Identity attribution (who edited what)	Beat 2 + Beat 5	Phase 3 (depends on Identity & Passes Phase 1)
Conflict resolution when two users edit the same cell	Beat 6	Phase 4
Tombstones — intentional deletes that must propagate	Beat 3 design detail	Phase 1
Schema migrations as the data model evolves	Out-of-scope; lives in consistency_plan.md Phase 4	Separate
Audit trail for who ran which beat when	Each beat has its own log	Phase 1 (logs) + Phase 3 (auditor role)

---

Implementation Roadmap

Phase 0 — This paper

Establish vocabulary, the protocol, and the operation-mode shapes. The lost cell incident is the prompt.

Phase 1 — Local mode publish-back + Beat-1 correctness

Scope: Mode 1 only. Owner’s laptop, single user.

Deliverables — Beat 3 (publish):

• ✅ New jin publish command (2026-06-12). Single-name umbrella over the per-kind publishers — runs jin bell flush’s logic + jin sis publish’s logic in one round. Each sub-publisher commits under its own Scribe (Sense 38); the umbrella adds no commit of its own. Per-kind targeting via --kind {bell,notes,bookmarks,recents,all}.
• ✅ afs/log/publish_log.jsonl append-only. One Sense-47 row per round summarising kinds + counts + outcome. The higher-level “Beat 3 fired” record, distinct from the per-commit afs_commit_log entries the sub-publishers already write.
• ✅ The Node-helper trick to read SIS without the segfault. (Lives in publish_via_node; preserved through the umbrella.)
• Drop the misleading docstring on jin sis flush and rename it to jin sis prune-redundant.
• Site-wide “unpublished changes” banner replacing the per-page one.
• Tombstone propagation: deleted SIS cells produce YAML diffs that remove the cell, with an audit entry.

Deliverables — Beat 1 (build) correctness, per Symptom B:

• Audit jin make’s change-detection. Confirm it currently misses YAML edits that propagate through the compiled-SQL → dbt-target pathway.
• Pick a fix from {hash compiled SQL output | unconditional invalidation per artifact directory | dbt state:modified}. Lean: hash compiled SQL.
• Add a regression test: edit a YAML cell, run jin make (no flags), assert the corresponding dbt model re-materialises.

Deliverables — modified_at normalisation, per Symptom C:

• Pick the file-mtime rule (the entity/dimensions one in jinflow/afs.py:1404) as the canonical fallback for ALL YAML-backed artifacts. Apply it in scripts/notecompile.py and any sibling compilers using author.date as fallback.
• Cell-level edits (Beat 2 captures) auto-stamp modified_at on each cell. Per-cell created_at is set on creation; never modified after.
• Update the actions board’s “updated DATE” to use cell-level timestamps once they exist.

Exit criteria: the owner can edit a cell in JinDesk, run jin publish, see the YAML change. The owner can edit a YAML directly, run jin make (no --clean), see the change in the KLS. No more lost cells, no more silently-skipped edits.

Phase 2 — Awareness (pulse)

Scope: Mode 1, then Mode 3.

Deliverables:

• _<tenant>.notification_journal table.
• Per-page-load AFS-upstream check: count commits-ahead, surface in bell-icon UI element.
• “Recent Edits” page surfacing all SIS writes across notes/ bookmarks/recents.
• Inhale/exhale/echo notification classes wired up.

Exit criteria: when a teammate pushes AFS, the owner sees a banner on next page load. When the owner has unpublished SIS, every page tells them.

Phase 3 — Identity attribution

Scope: depends on Identity & Passes Phase 1 landing.

Deliverables:

• Every Beat 2 write captures the editor identity.
• Every notification names the actor.
• The editor_id column propagates through publish into AFS YAML (per sense_25_identity_and_passes.md §12 RACI extension).

Exit criteria: “Updated by <user> 2026-04-21” replaces “Updated 2026-04-21”.

Phase 4 — Conflict resolution

Scope: Mode 1 + Mode 2 (P2P2P).

Deliverables:

• Pre-publish conflict detection: did the AFS-side change since the user’s edit started?
• Structural merge for non-overlapping fields (status + result on the same cell can both publish without conflict).
• Human-prompted resolution UI for overlaps.
• P2P2P shadow SIS for offline resilience.

Exit criteria: two collaborators can edit the same note simultaneously and both publishes succeed.

Phase 5 — Cloud-mode SIS

Scope: Mode 3, 4, 5.

Deliverables:

• Decision on hosted SIS technology.
• jin pull-sis for Mode 4 (owner pulls cloud SIS state).
• Cloud-worker publish for Mode 5.

Exit criteria: the Heartbeat works the same way regardless of where SIS physically lives.

Phase 6 — Make-as-a-Service rhythm

Scope: Mode 5.

Deliverables:

• Push-triggered build.
• Scheduled publish-back from central SIS.
• Identity & Passes-gated per-user publish authority.

Exit criteria: a tenant can be operated entirely from a browser, with no local jinflow install.

---

Open Questions

Numbered for cross-reference; assigned tentatively.

1. jin publish cadence. On-demand only? Pre-make hook (publish then make)? Cron? Combination? Lean: on-demand initially, then pre-make hook by default.

2. Per-user vs per-tenant SIS. Today there’s one SIS file per tenant. In Mode 3, do we shard by user? Per-tenant means simultaneous editors share a sandbox; per-user means publish-back has to merge. Open.

3. The Node-helper trick. Is bundling Node a price worth paying to keep the Python CLI segfault-free? Alternatives are uglier. Lean: yes.

4. Tombstone semantics. When the owner deletes a cell in JinDesk, should jin publish remove it from YAML, or write a tombstone marker? Lean: remove from YAML; the git history IS the tombstone.

5. Notification storage. _<tenant>.notification_journal lives in KLS or in a sibling SIS table? It’s mutable but it’s also tenant-bound. Lean: SIS, with a last_seen per user.

6. Bookmarks/recents lifecycle. Are these “notes-shaped” first-class artifacts that publish to AFS YAML, or are they ephemeral SIS-only data with a different bar? Open. Today they silently never publish. Probably they should — at least bookmarks.

7. The shared notes branch. Where does it fit? Today it’s a separate beat (notecompile.py:378). Should it be retired in favor of the unified publish flow? Open; revisit after Phase 1.

8. Echo opt-in. Does the owner want a notification every time someone touches a cell he co-authored? Some users will. Some will be buried in alerts. Lean: per-user preference.

---

Connections

• Sense 16 (P2P2P) — defines Mode 2. The Heartbeat in Mode 2 is the most interesting case (offline owners, viewers’ writes, bidirectional pulse).
• consistency_plan.md — Phase 1 (idempotent make) and Phase 2 (SIS write-purity) are prerequisites. The Heartbeat is the Phase 3+ story those phases set up.
• sense_25_identity_and_passes.md — Phase 3 of the Heartbeat depends on Identity & Passes Phase 1 landing (per-user identity at write time). §12 of that paper covers RACI; the Heartbeat is where RACI plays out at runtime.
• extractor_discipline.md — same shape as the Heartbeat for source-system data: extraction is its own beat with logs and a contract. The Heartbeat is for the human-edit boundary; extractor discipline is for the system-source boundary. Same principle, two domains.
• build_log_jsonl — already an example of a per-beat log. The publish-log and reconcile-log will be siblings.

---

How this manifests in jinflow

The Heartbeat is the loop between JinDesk’s mutable workspace (SIS) and the durable git-backed AFS. Three beats run today: Beat 1 — cells stored in SIS, hydrated lazily into the KLS. Beat 2 — the Atelier writes directly to AFS via the Scribe (Sense 38). Beat 3 — jin publish drains pending SIS notes + bookmarks into the AFS as commits, with an append-only publish_log.jsonl. The remaining beats (drift detection, modified_at semantics, deeper audit) are in the implementation roadmap above.

• In the app: Inbox / Bell (Sense 33 — the pub/sub channel that fires on every save) · the Atelier surfaces (Sense 29) for the Beat 2 write path
• CLI: jin publish (Beat 3 umbrella · flushes SIS notes + bookmarks into AFS commits) · jin sis flush (notes only) · jin bell (pub/sub channel inspector)
• Logs: state/publish_log.jsonl (append-only · one row per publish round) · state/extract_log.jsonl (extraction boundary, Sense 31) — both feed Sense 47 The Log

Status: Beats 1–3 shipped (Phase 2 of the consistency plan landed April 2026, Beat 3 / jin publish shipped June 2026). Beats 4–6 are forming; see the Implementation Roadmap section above for current state.

---

The Elevator Pitch

jinflow has a brain (KLS, deterministic), a memory (AFS, durable), and a mouth (SIS, mutable). The brain and the memory talk reliably. The mouth talks reliably into the brain via lazy hydration. But the mouth doesn’t talk back to the memory, and nobody knows when anything has changed.

The Heartbeat fixes that. Six discrete beats, each with its own verb, log, and UI. No circles. Every beat is observable. The system is alive, in rhythm, and the absence of a beat is itself a signal.

No one will ever lose another cell.

---

Numerical neighbors: ← Sense 20: The Seam — Where Pack and Tenant Meet · Sense 22 — The Legend: Implementation →