Sense 46: The Roster — Clerk authors, AFS materialises

Sense 46 · In bloom · Last touched 2026-08-23

• last_verified: 2026-06-05

Synced from docs/design/sense_46_the_roster.md in the engine repo — that’s the source; this page is a build-time mirror.

The list of who belongs lives where identity already lives. The book of names on the wall is a copy, not the truth.

What pulled us in

A pattern that kept tripping us up across Sense 25.

Phase A through Phase C of the invitation flow shipped clean: schema, jin invite, /api/invitations firing real Clerk invitations with publicMetadata.jinflow.{tenant, identity, principal_id, …}. The mechanics worked. Then Phase D — stamping clerk_user_id back onto the AFS principal YAML on first-bind — refused to settle.

Three proposals fell over in sequence.

Stamp at first-bind in JinDesk hook. Per-request AFS writes during a session-resolve handler dirties git silently, scatters binding events across whatever branch is checked out, and only fires when the invitee opens a local JinDesk. Most invitees touch cloud first.

Clerk webhook on user.created. Architecturally clean but needs writable AFS in the cloud JinDesk — which the sovereignty model forbids by construction. Cloud JinDesk is read-only against AFS; opening that door is its own (much bigger) conversation.

Fold into jin make. Better — owner-machine, sovereignty-clean, single audit point. But identity has a different lifecycle from analytical builds. Users join and leave continuously; makes happen when the owner happens to rebuild. Tying authorisation to the make cycle is a category error: someone signs up today, and they don’t exist to the system until the owner runs make next week.

Each of these tried to solve the same misframing: AFS as the authoritative store for identity. The reframe falls out when you stop trying.

The Concept

Identity is cross-tenant by nature. A single human is in zero, one, or many tenants; her membership in inspire has no special relationship to her membership in hospital_zeta. The AFS git, by construction, is tenant-scoped. Putting cross-tenant data in a tenant-scoped store is fundamentally the wrong substrate.

Clerk already holds cross-tenant identity. It’s the only system in the picture that natively sees a human across tenants. We’ve been pretending the AFS YAML is the source of truth and Clerk is “just a binding” — but the binding direction was inverted. Clerk is the authority; the YAML was always a denormalisation.

The Roster is what you write on the wall. It’s the materialised view of the authoritative record. Useful for offline display, signal authorship attribution, snapshot reproducibility — anywhere you want the names without round-tripping to the authoritative system. But the wall is a copy. The truth lives elsewhere.

This Sense codifies:

• Clerk is the source of truth for tenant membership + identity. publicMetadata.jinflow.tenants[T] = { identity, principal_id, …} is the authoritative claim.
• AFS principals are a materialised view. Generated from Clerk by an owner-initiated command (jin sync-principals, also run as a phase of jin make). Lives on the principals orphan branch alongside notes, bookmarks, recents, bell, suggestions. Tenant-scoped (each tenant’s roster lives in that tenant’s AFS git).
• KLS holds the baked view. _<tenant>.principal_registry, same as today. What JinDesk reads for display.
• JinDesk never reads AFS. Authorization comes from Clerk (live); display data comes from KLS (cached at last bake).

What this means in practice

The Roster pattern has four working parts:

1. One write surface, owner-controlled. jin invite is the only way into a tenant. Writes to Clerk’s publicMetadata. No dashboard-created Clerk user gets auto-enrolled — if it didn’t come through jin invite, the system doesn’t know about it. This is a feature: the owner controls enrolment.

2. Materialisation is an owner-initiated, idempotent port. jin sync-principals walks Clerk invitations, filters by publicMetadata.jinflow.tenants[<this tenant>], writes principal YAMLs onto the principals orphan branch, commits. Re-runnable; safe to invoke at any rhythm. Folds into jin make as Phase 0d for the routine case.

3. Two reads, two purposes. Authorisation reads publicMetadata.jinflow.tenants[T] from Clerk (live; falls through to anonymous when absent). Display reads principal_registry from KLS (cached; what the snapshot saw). The two answers can briefly disagree — Clerk added a member five minutes ago, KLS hasn’t rebaked — and that’s fine. The new member is authorised (Clerk says so), they just don’t appear in the principals tile until next make.

4. The orphan branch is sovereignty-aligned. Each tenant’s principals branch carries only that tenant’s people. No cross-tenant index lives anywhere on jinflow infrastructure except in Clerk itself. A tenant’s AFS clone reveals only that tenant’s roster.

The Roster surface in detail

Clerk shape — publicMetadata.jinflow

The tenant-keyed map shape that supersedes Sense 25 §6.5’s flat single-tenant shape:

{
  "jinflow": {
    "tenants": {
      "numetrix.inspire": {
        "identity": "owner",
        "principal_id": "alice_smith",
        "invited_by": "mig",
        "invited_at": "2026-05-27T10:00:00Z"
      },
      "hospital_zeta": {
        "identity": "analyst",
        "principal_id": "alice_clinical",
        "invited_by": "mig_zeta",
        "invited_at": "2026-06-01T14:23:00Z"
      }
    },
    "display_name": "Alice Smith"
  }
}

• tenants is the load-bearing surface. Tenant slugs are the keys; per-tenant entries carry the assignment.
• display_name is per-human (sits at the jinflow level, not inside tenants).
• principal_id is per-tenant because each tenant gets to name its principals in its own analytical vocabulary.
• invited_by and invited_at are denormalised onto the assignment for fast audit reads.

AFS shape — principals/<principal_id>.yaml on the principals orphan branch

Generated from Clerk at sync time. Lives only on the orphan branch (not on main). Each tenant’s AFS git has its own principals branch carrying only that tenant’s roster.

principal_id: alice_smith
kind: user
display_name:
  en: Alice Smith
email: alice@example.com
clerk_user_id: user_abc123
clerk_invitation_id: inv_xyz789
invited_by: mig
invited_at: 2026-05-27T10:00:00Z
accepted_at: 2026-05-28T09:14:22Z
status: active
identity: owner
extra_roles: []

Everything except extra_roles is derived from Clerk on each sync. extra_roles is the one tenant-local override: roles unioned on top of the identity-derived role set, without round-tripping through Clerk. Useful for tenant-specific assignments that don’t fit the identity scheme. Sync-principals preserves it across syncs.

KLS shape — _<tenant>.principal_registry

Unchanged from Sense 25 Phase 1.5. principalcompile.py reads YAMLs (now from the orphan branch via worktree), bakes into _<tenant>.principal_registry. JinDesk reads from there for display, signal authorship attribution, etc.

Read-path architecture

                  Owner action            Storage             Reader
                  ─────────────           ───────             ──────
authorisation     jin invite ──▶ Clerk publicMetadata ──▶ hooks.server.ts
(live)                            jinflow.tenants[T]        (every request)

display data      jin sync   ──▶ AFS principals branch ──▶ jin make ──▶ KLS
(cached)          jin make                                              registry
                                                                          │
                                                                          ▼
                                                                       JinDesk
                                                                       (display)

The authorisation path is live; the display path is cached. They can briefly disagree — that’s design, not bug. A new member is authorised before they appear in the registry; a removed member disappears from authorisation before their YAML is sync-archived. The eventual consistency is bounded by the owner’s jin make rhythm.

What jin sync-principals does

1. Determine target tenant (from --tenant flag or default).
2. Walk getInvitationList({status:'accepted', limit:500}) paginated.
3. For each invitation:
     - Skip if publicMetadata.jinflow.tenants[<tenant>] is not set.
     - Resolve to the Clerk user that accepted it.
     - Build a principal record from the user + invitation metadata.
4. Walk getInvitationList({status:'pending'}) for state=invited records.
5. Compare against current principals/*.yaml on the principals branch:
     - New users → write new YAML.
     - Existing users → update derived fields; preserve extra_roles.
     - Missing users (revoked / expired) → mark status=archived.
       Never delete (analytical authorship may reference them).
6. Commit changes on the principals branch.
7. Print summary: N new, M updated, K archived.

Idempotent. Runs over the network (Clerk API). Owner-initiated.

The extra_roles preservation rule is load-bearing: it’s the one field the owner can edit directly in the YAML without round-tripping through Clerk. The sync command reads the existing YAML before overwriting, lifts extra_roles, and reapplies. Other fields are overwritten unconditionally.

What changes from Sense 25

This Sense absorbs §6 (invitations) and §7 (CLI auth) from docs/design/sense_25_identity_and_passes.md. Sense 25 retains:

• The four-level cascade: identity → role → pass → capability.
• The system/identities/, system/roles/, system/passes/, system/capabilities/ YAML model — analytical framework, not per-human data.
• Phase 1.5 principals registry shape (the YAML schema, the bake, the KLS read).

Sense 25’s authorship model also stays: AFS-baked passes and capabilities define what each identity means analytically. The Roster only changes the substrate for who holds which identity. That moves to Clerk.

What retires from Sense 25:

• §6.5’s email-bind fallback. No orphaned YAMLs to bind to — every YAML is Clerk-derived and has clerk_user_id filled at materialise time.
• §6.5’s complex state machine (proposed → invited → accepted → active). States derive from Clerk invitation status; YAML status is active / archived only.
• Phase D as originally conceived. Stamping is dissolved by the inversion.

Consent and opt-out

A human’s identity record can never be modified by another human without their consent. Not even by the owner of a tenant. Not even with the best intentions. This applies in both directions: to enrollment, and to removal.

The Roster surface (Clerk publicMetadata) is one human’s record. The owner of a tenant authors invitations; the invited human authorises the actual record change. The two never collapse into one step.

Enrollment paths

Situation	Flow	Where consent lives
Invitee has no Clerk account	createInvitation → email → Clerk-hosted sign-up → publicMetadata applied on sign-up	Sign-up is the act of consent
Invitee already has a Clerk account	createInvitation (with ignoreExisting: true) → email → invitee clicks accept while signed in OR signs in → publicMetadata MERGED	The accept click is the act of consent
Owner edits the YAML directly	(not a path)	—
updateUserMetadata silently called by /api/invitations	forbidden	no consent surface

The current /api/invitations (Sense 46 Step 2) handles only the first case. Existing-user handling is queued; the implementation must route through the invitation-accept flow, never through a direct updateUserMetadata call.

Merge semantics

When an existing user accepts an invitation, Clerk merges the invitation’s publicMetadata.jinflow.tenants map into the user’s existing map. The user keeps every tenant they already belonged to; the new tenant is added. The owner of the inviting tenant cannot see, modify, or clobber assignments from other tenants — they only contribute the entry for their own tenant.

This is Clerk’s standard behaviour and the only sensible semantic for the model. Replace-on-accept would let one owner’s invitation silently destroy assignments held in other tenants.

Opt-out

A user can leave a tenant at any time, on their own initiative, without owner action:

• An JinDesk surface (likely /<tenant>/account or a global “my tenants” page) lists the tenants the signed-in user is enrolled in, with a “Leave this tenant” action per entry.
• On leave:
  • The user’s publicMetadata.jinflow.tenants[<tenant>] entry is removed (their own consent removes their own record).
  • The corresponding AFS principal YAML is marked status: archived (does NOT delete — analytical authorship references must survive, e.g. signals authored by this person stay attributable).
  • The owner is notified out-of-band (email, in-app banner) so they can update workflow expectations.
• The user’s Clerk account itself is untouched. They keep their account; they just no longer belong to this tenant.

Owner-initiated removal (revocation) is a separate concern from opt-out — symmetric in mechanism, different in initiator. Both end up in the same place: principal status=archived, publicMetadata entry removed. The two never share an endpoint because the authorisation logic is different.

Why this matters

• Sovereignty extends to identity records. Sense 16’s “your data on your machine” principle applies here for identity: the human owns their record. Owners describe assignments; humans authorise them.
• GDPR-aligned by construction. Consent at enrollment, freedom to withdraw at any time, no silent modifications. The architecture matches the legal posture without bolt-on compliance work.
• Trust between owners. Bob can never accidentally — or deliberately — overwrite Alice’s enrollment in Carol’s tenant by inviting Alice to his own. The metadata surface is per-human, modifiable only by that human’s explicit action.

Implementation status

• New-user invitation (consent via sign-up): shipped (Sense 46 Step 2 + the live-confirmed test on rmc.jinflow.io, 2026-05-31).
• Existing-user invitation (consent via accept-click, merge semantics): shipped (ab80b0ea — ignoreExisting: true on createInvitation; live-confirmed on rmc 2026-06-03 with the same Clerk user across revoke→re-invite, publicMetadata merge verified).
• Owner revocation (separate from opt-out): shipped (cdfb5851 — jin revoke + /[tenant]/api/revocations; resurrection on re-invite at 51a91904).
• Opt-out surface in JinDesk: shipped 2026-06-05. POST /[tenant]/api/leave (cookie-session auth, surgical updateUserMetadata, no-orphan-owner invariant); UI as a “Leave this tenant” item in the SignedInBadge dropdown.
• AFS reconciliation on opt-out: shipped 2026-06-05. jin sync-principals now reads each accepted invitation’s user’s CURRENT publicMetadata.jinflow.tenants[T]. When it’s absent, the matching local YAML is archived and an opt_out event lands in afs/log/membership_events.jsonl. Bell-style runtime notifications weren’t the right substrate (Bell rings are analytical observations baked from the KLS, not runtime events); a per-tenant append-only event log is the v1 audit trail. The /principals display surface that surfaces these events is the next follow-up.
• Owner revocation also emits to the event log: shipped 2026-06-05. jin revoke appends a kind: revoke entry where actor != subject (vs kind: opt_out where they match).
• Display surface for membership events: pending. A “Recent membership changes” section on /[tenant]/principals that reads the baked _<tenant>.membership_events table.
• KLS bake step for membership_events: pending. Reads afs/log/membership_events.jsonl → _<tenant>.membership_events on jin make. Until shipped, events are visible only via tail afs/log/membership_events.jsonl.

Browser-side writes and the SIS overlay (2026-07-09)

Sense 46 §6-8 described jin invite and jin revoke as CLI-initiated — the owner runs a command on their laptop, the AFS YAML is written locally, Clerk gets updated, jin make bakes the new state, jin ship uploads. That path works when the owner is at their laptop. It fails the “I’m on rmc.jinflow.io in Safari and want to add someone” case, which is the case owners actually reach for.

The browser admin UI at /[tenant]/admin/users fills that gap. It gets to write to Clerk (Sense 46 already made that the source of truth) but it cannot write to the AFS — the cloud JinDesk has no local AFS. That leaves a felt gap between “invitation sent” and “row appears in the roster,” bounded by however long the owner takes to run jin sync-principals → jin make → jin ship. Often that’s minutes; sometimes it’s a day.

The overlay

We close that gap with an SIS-hosted UX overlay. Same pattern as Sense 33 (Bell) uses for resolutions over rings: the KLS holds the reconciled truth; the SIS holds the not-yet-reconciled hint. The roster page reads both and unions them.

Table: sis.principal_overlay

column	type	note
principal_id	TEXT	slug of the invited or revoked principal
tenant	TEXT	pack-qualified, e.g. numetrix.rmc
action	TEXT	'invite' or 'revoke'
email	TEXT	invite rows only
identity	TEXT	invite rows only
display_name	TEXT	invite rows only
created_at	TEXT	ISO 8601
created_by	TEXT	inviting owner’s principal_id

Primary key: (tenant, principal_id, action). A repeat click after a transient error upserts rather than duping.

Read: filter, then prune

On every roster load the page server:

1. Loads the KLS-baked principal_registry.
2. Loads all overlay rows for the tenant.
3. Drops overlay rows that are already reconciled:
   • invite rows whose principal_id is now in the KLS registry (any status) — the reconciliation has landed.
   • revoke rows whose principal_id is now absent from the KLS registry, OR present with status='archived'.
4. Renders the unioned view.
5. Best-effort prunes reconciled rows from SIS so the file stays lean.

This is a self-cleaning contract. No cron, no jin sis flush, no version epoch. The moment the KLS round-trip completes, the overlay becomes silent on its own.

Why no ship-back for principals

Sense 33’s Bell overlay needs a ship-back path because SIS is the source of truth for resolutions — a dismiss done on rmc.jinflow.io must eventually reach the owner’s laptop or it evaporates on the next KLS refresh.

The principal overlay is different. Clerk is the source of truth for tenant membership (Sense 46 §Concept). A cloud-side invite writes to Clerk and to SIS; the CLI’s jin sync-principals reads from Clerk, not from the cloud SIS. The overlay never has to travel back — Clerk already carries the round-trip.

Rephrased as a rule: when SIS overlays a Clerk-backed truth, no ship-back is needed. When SIS overlays a SIS-backed truth (notes, bell resolutions), a ship-back step is required for the round-trip to close. The overlay pattern is safe here specifically because Sense 46 chose Clerk.

KLS_CACHE_DIR fallback in resolveSisPath

Historically sisdb.resolveSisPath() looked only at JINFLOW_LIVE / JINFLOW_LIVE_ROOT. Fly deployments never set either — Sense 33’s SIS-write path silently no-op’d on rmc.jinflow.io, dev.jinflow.io, and this-is-jinflow. The overlay would have inherited the same silence.

Fix: the resolver now falls through to KLS_CACHE_DIR (Fly: /data/cache) when live-root env is absent. That directory is where klsStore.ts already lands SIS files from R2 at boot. The addition is symmetric to the KLS_LOCAL_DIR fallback chain in klsStore.ts.

Side effect: notes / bell / bookmarks writes in the cloud now persist to the container’s SIS cache — previously they no-op’d. The ship-back gap noted above applies to those; they need a download-from-R2 counterpart step to close the loop for the non-Clerk substrates. Out of scope for this addendum, queued as a follow-up.

What CLI operations see

jin sync-principals doesn’t read the SIS overlay. It reads Clerk directly and writes to AFS. That means:

• A cloud-side invite reaches AFS on the next jin sync-principals run (because Clerk carries it).
• A cloud-side invite that never got a matching sync-principals run eventually shows up anyway — the next sync picks it up regardless of when it happened.
• The overlay never needs a “flush to CLI” mechanism. It’s a read-side concern only.

What this means in practice

• Owner clicks Send on /[tenant]/admin/users → row appears instantly with an amber “invited / awaiting accept” chip.
• Invitee accepts (magic link → /accept-invite) → they’re bound in Clerk. The overlay row still shows amber on the owner’s roster; Clerk state is one step ahead.
• Owner runs jin sync-principals on their laptop → AFS YAML gets written. jin make bakes the KLS. jin ship uploads.
• Next roster load: KLS carries the principal; overlay row is reconciled and disappears (and gets pruned from SIS).

The owner does the same commands they would have run anyway. The overlay just hides the timing gap between “browser click” and “KLS caught up.”

Related Senses

• Sense 16: The P2P2P. The Roster reuses the sovereignty principle. Tenant data stays on the owner’s machine; cross-tenant identity stays in Clerk. Neither system ever holds the other’s scope. The pattern Sense 16 established for data applies here for identity.

• Sense 25: Identity & Passes. Sense 25 defined the cascade and vocabulary. The Roster reframes the substrate question while preserving everything Sense 25 says about what an identity means. The two read together; neither replaces the other.

• Sense 36: The Lineage. Same materialised-view pattern at a different layer. Sense 36 has the tenant remember its birth pack via .pack-init.yml; The Roster has the tenant remember its members via principal YAMLs. Both are derived from an upstream authority, persisted locally for offline + snapshot semantics.

• Sense 27: The Funnel. When Clerk is unreachable, the cached KLS principal_registry is what JinDesk can still display. Authorisation degrades to “anonymous fallback”; display stays intact. The narrowing-at-the-source principle applies — KLS holds enough to keep the picture coherent.

• Sense 33: The Bell. Direct precedent for the SIS-as-overlay pattern documented in §Browser-side writes above. Bell overlays rings (KLS) with resolutions (SIS); The Roster overlays the KLS-baked principal_registry with sis.principal_overlay. Same read shape (KLS LEFT JOIN SIS), different substrate for the source of truth (Clerk vs SIS), so the round-trip stories differ.

Open questions

1. accepted_by_user_id on Invitation — resolved 2026-05-28. Clerk’s standard Invitation type does NOT expose the accepting user_id. Confirmed against @clerk/backend/dist/api/resources/JSON.d.ts InvitationJSON: no user_id, no accepted_by, no hidden field. (Clerk’s Organization-flavoured sibling OrganizationInvitationAcceptedJSON does carry user_id: string natively — a signal that Clerk Organisations are the architecturally pure substrate if scale ever demands it.) Workaround: clerkClient.users.getUserList({ emailAddress: [<emails>] }) batchable per sync call. Two API round-trips per jin sync-principals invocation at the owner-scale. Negligible.

2. Revoked / expired invitations. Archive (set status='archived', keep YAML) vs delete (remove YAML). Lean archive — analytical authorship references shouldn’t break. Decision pending in Phase 3 (jin sync-principals).

3. Owner edits between syncs. sync-principals overwrites Clerk-derived fields on next run. extra_roles is preserved. Other owner edits to a sync-managed YAML are clobbered by design. Worth surfacing in the CLI output (“preserving 2 extra_roles overrides; overwriting 4 status fields”).

4. Anonymous tenant access. JINFLOW_ALLOW_ANONYMOUS=1 flag bypasses Clerk entirely. Confirm it still works under the new model (no Clerk userId → no publicMetadata read → fall through to anonymous). Worth a spec test before Phase 5 ships.

5. Cross-product identity (cuebook). The cuebook letter thread agreed both products use separate Clerk apps, with publicMetadata.jinflow.* and publicMetadata.cuebook.* namespaced for any future shared-app scenario. The Roster’s map shape is jinflow-only; cuebook owns its own equivalent. Cross- product mapping doc on cuebook side per their commitment.

6. Clerk Organisations as future substrate. Each tenant could alternatively be a Clerk Organisation, giving native enumeration via getOrganizationMembershipList. Architecturally pure; major refactor. Out of scope for The Roster v1; flagged for a future Sense if scale demands.

Build order

Each step independently shippable.

1. This doc. Sense 46 codifies the model and absorbs Sense 25 §6+§7. Doc-only.
2. /api/invitations updated to write tenant-keyed map shape. Transition: writes both old flat shape AND new map shape for a release or two so existing read paths keep working.
3. jin sync-principals command. Implements the materialisation step. First version writes to current afs/principals/ on main; defer the orphan-branch move.
4. Register principals as an A-tier artifact branch. Add to ARTIFACT_BRANCHES in jinflow/cli/commands/afs.py:95 and add principals to BranchIndicator.svelte’s ORPHAN_BRANCHES set. The existing dual-storage pattern (used by notes, bookmarks, recents, bell, suggestions) applies unchanged: principals/*.yaml lives on main for working read AND on the principals orphan branch as a durable mirror. Sync directions: main → orphan on git push; orphan → main on jin make Phase 1c. The orphan branch is auto-created on first push when a tenant has principal YAMLs — no manual migration command needed.
5. JinDesk read-path migration. hooks.server.ts reads from publicMetadata directly. KLS lookup simplifies to display-only (no more email-fallback).
6. Retire email-bind fallback. Once steps 1-5 are stable in production tenants.

What this changes for the reader

If you’ve been reading Sense 25 looking for the invitation flow, the canonical answer lives here now. Sense 25 §6 and §7 will carry forward-pointers to this doc; the cascade material in Sense 25 stays where it is.

If you’ve been reading the spike doc at docs/design/principals_substrate_spike.md — that becomes the research output; this doc is the design.

---

Numerical neighbors: ← Sense 45: The Loom — two axes, woven · Sense 47: The Log — the system’s write-once memory →