Sense 44: The Forge

Sense 44 · Forming · Last touched 2026-08-20

last_verified: 2026-07-27

Synced from docs/design/sense_44_the_forge.md in the engine repo — that’s the source; this page is a build-time mirror.

The Workshop carries the why. The Forge carries the what.

Every pack is composed of the same eighteen universal subsystems. The first six packs discovered this list one missing piece at a time. The seventh shouldn’t have to.

Status: proposed Date: 2026-05-24 Author: jazzisnow Architect + Claude (a conversation about whether we know what it takes to make a new pack) Adjacent:

Sense 26: The Workshop — the pack ships its own thinking. The Workshop is where the analytical conversation happens; The Forge is the tooling that turns that conversation into a working pack without forgotten pieces. Different problems, paired solutions.
Sense 40: The Metamodel — pulls shared concerns engine-side so packs stop reinventing them. The Forge is the measurement surface that exposes when a pack has reinvented something it shouldn’t have.
Sense 36: The Lineage — packs remember their birth pack via .pack-init.yml. The Forge will write this at scaffold time.
Sense 14: The Signal — defines the analytical chain the matrix references.
Sense 23: The Engine’s Seam — the engine/pack boundary. Forge tooling lives engine-side; pack content lives pack-side.

What pulled us in

A jinflow pack today has eighteen universal subsystems and another six that mature packs add — twenty-four directories of conventions, plus a manifest, plus dbt project layout, plus engine-side LAYER_CONFIG registrations, plus a glossary, plus a naming convention with at least two known drift cases.

The first pack (numetrix) discovered this list across three months of iteration. The second (millesime) and third (alptrack) discovered it again, faster, by copying numetrix’s shape. The fourth (interlogic) discovered it by copying alptrack. The fifth (lexflow) discovered it by copying interlogic. The sixth (hrcentral) discovered it by copying lexflow.

Each copy forgot something different. The first time anyone ran jin pack doctor against the packhub on 2026-05-24, the matrix lit up:

millesime still has diagnoses/, hypotheses/, probes/ — pre-Sense-14 vocabulary that never got renamed. Three folders of stale terminology, hiding because the pack worked anyway.
Five of six packs lack a jinflow_engine_version pin in their manifest.
Four of six packs lack the source_system_columns.sql dispatch macro that makes multi-source ingestion possible.
All six packs had at least one empty placeholder directory the author meant to fill later and forgot.

None of these are bugs that break a pack. All of them are debts the next pack author will inherit if nothing measures them.

That’s the cost of forgetting. The Workshop addresses it as a posture problem — “ship the thinking” — and that’s necessary but not sufficient. Even with the thinking preserved, the building keeps slipping. What the project needed was a way to walk a pack and count what’s missing.

The insight

A pack is a structured artifact, not a script. Structured artifacts can be measured against a spec. The spec doesn’t have to be invented — it can be derived from the packs that already work.

That’s the empirical move The Forge makes: instead of writing a “minimum viable pack” spec in the abstract and arguing about what should be required vs optional, it looks at the six existing packs and says — these eighteen subsystems exist in every one. Whatever else you add or omit, this floor is what “pack” means. The matrix isn’t a design decision; it’s an observation.

Three result levels capture the gradient between bare floor and rich ceiling:

Level	Meaning
✓ ok	Present + populated, OR genuinely optional and absent
⚠ warn	Present but empty, OR mature-pack-ceiling subsystem missing, OR engine-side gap (e.g. a `LAYER_CONFIG` registration for a layer the pack uses), OR naming drift
✗ fail	Required subsystem missing or malformed manifest

The two verbs that turn the matrix into tooling:

jin pack doctor — measures distance from the ceiling. Walks a pack, runs the checks, prints the report. No writes. Catches the millesime-style stale-vocabulary drift before the next refactor inherits it.
jin pack create — builds the floor. Scaffolds a new pack directory with the universal-floor subsystems present, populated with annotated examples, a manifest pinned to the current engine, and a .pack-init.yml written for Sense 36 lineage. (Phase 2; not in the first shipping iteration.)

The pair forms a closed loop. Create gives you the working floor. Doctor tells you what’s still placeholder. As you fill placeholders, doctor’s warnings drop. When doctor is all green, you have a pack at parity with the canonical reference.

What’s shipped (2026-05-24)

jin pack doctor lands as the first half of The Forge:

jin pack doctor                  audit the pack at $PWD
jin pack doctor <name>           resolve against JINFLOW_PACKHUB
                                 (default ~/Programming/jazzisnow/jinflow-packhub)

What it checks:

Manifest — jinflow.yml + about.yaml present; engine pin declared; source_systems and contracts declared.
18 universal subsystems — signals, perspectives, theses, verdicts, smebits, bitbundles, entities, lineage, contracts, glossary, csv_schemas, calibration, reports, scripts, tenant, models, docs, dbt (every existing pack has all of these).
6 mature-pack-ceiling subsystems — lenses, notebook, uoms, field_manual, evidence, tenants (mature packs add these; absence warns, not fails).
dbt project — dbt_project.yml present, macros/ present, source_system_columns.sql present for multi-source dispatch.
Engine-side LAYER_CONFIG — every layer used by the pack’s dbt models must be registered in scripts/generate_pipeline_graph.py. Catches the “renders with default styling” gap (e.g. the aggregates and notes layers exist in vai but were never registered engine-side; the pipeline diagram showed them with the fallback grey-box treatment).
Naming drift — known cases: notebooks/ vs notebook/ (millesime is the odd one out), tenants/ vs tenant/ (singular = the schema, plural = overrides; easy to confuse).

The output is plain text with ✓ / ⚠ / ✗ symbols and a totals line. Exit code is non-zero only when a ✗ fail is present, so doctor is safe to wire into CI for packs that want to gate their own state without flapping on the inevitable ⚠s.

The 15 unit tests pin the matrix shape, the path resolution (JINFLOW_PACKHUB env-var override, walk-up-from-$PWD), the manifest validation, the subsystem-counting logic (top-level YAMLs vs nested content like models/<layer>/*.sql), and the folder→layer mapping that mirrors scan_dbt_models. The matrix itself is in code, not config — adding a new subsystem is one row in SUBSYSTEMS and one new check function if it needs custom validation.

What’s next (Phase 2 — `jin pack create`)

The scaffold half. Surface sketch:

jin pack create <name> [--from <existing-pack>] [--domain <short-description>]

Generates a new pack directory at <packhub>/jinflow-pack-<name>/ with:

jinflow.yml manifest pinned to the current engine version, with source_systems and contracts stubbed.
about.yaml skeleton.
Every universal-floor subsystem present as a directory, each containing one annotated example (a signal that explains the YAML schema in comments, a thesis that walks the evidence-chain shape, a smebit that shows both Statement and Check levels, etc.).
dbt/<name>/ scaffold: dbt_project.yml, packages.yml, models/{bronze,silver,gold}/ with one example per layer using the standard dispatch macros, macros/source_system_columns.sql with the dispatch skeleton for the first source system.
glossary/registry_glossary.yaml skeleton (trilingual, empty).
tenants/<demo>/tenant.yml for one demo tenant.
docs/ARCHITECTURE.md stub (the entry point for the Sense 26 “ship the thinking” discipline).
.pack-init.yml (Sense 36) recording engine version, init date, and --from provenance if used.
README.md quickstart.

--from numetrix copies the structure (not the content) of an existing pack as a stronger starting point — useful when the new domain is roughly the shape of an existing one.

A fresh create followed by doctor should land all-green-or-warn. Failures from doctor immediately after a create are scaffold bugs, not pack bugs.

Future iterations

What	Why
`jin pack doctor --json`	Machine-readable output for CI gates and dashboards.
`jin pack doctor --fix`	Auto-apply the safe fixes: add missing `.gitkeep` files, write missing `jinflow_engine_version` pins, register missing `LAYER_CONFIG` entries with sensible defaults. The non-trivial cases (naming drift, missing entities, stale vocabulary) remain manual.
`jin pack create --from numetrix --strip-data`	Use numetrix’s structure but strip its tenant data, signals, theses, etc. — a richer scaffold than the bare-floor default.
Cross-pack diff	`jin pack diff numetrix hrcentral` — which subsystems differ in shape (file count, naming conventions, presence). Useful for “what should I add to my pack to look more like the reference?”
Subsystem maturity scoring	Per-subsystem score (rich / partial / empty / absent) so doctor can give a single 0-100 “pack maturity” number across all subsystems. Possibly noise; possibly motivating.
Glossary integration	`doctor` checks that every signal/thesis/verdict referenced in YAML appears in `glossary/registry_glossary.yaml` with trilingual prose. The Bell-style “your pack has 14 unregistered signals” report.

What this Sense does NOT cover

The Workshop (Sense 26) — why a pack exists. The Forge is silent on motivation.
The Metamodel (Sense 40) — pulling shared concerns out of pack-land into the engine. The Forge measures pack-land; the Metamodel reduces what pack-land has to do. Complementary, not overlapping.
The Lineage (Sense 36) — packs remembering their birth. The Forge writes .pack-init.yml at create time but doesn’t define its semantics.
Pack-level governance — review processes, sign-off, publishing to the packhub remote. The Forge is local-author tooling.
The Atelier (Sense 29) — JinDesk-side authoring of Subject Matter, theses, etc. The Forge is CLI-side; the Atelier is web-side. They operate on the same underlying YAMLs but from opposite ends.

A short history

Date	What
2026-03	Two new packs (interlogic, alptrack) land in one week. Pack-creation friction recognised.
2026-04	Sense 26 (The Workshop) drafted: a pack ships its own thinking. Posture response.
2026-05-20	Sense 40 (The Metamodel) — six packs of duplication motivates engine-side consolidation.
2026-05-24	The Forge proposed. `jin pack doctor` lands as Phase 1. First run uncovers millesime’s
	stale pre-Sense-14 vocabulary, missing engine pins, missing source-system dispatch macros.

Future iterations will keep adding to this list. The Sense is the worldview those iterations should be consistent with.

The Workshop is where you think. The Forge is where you build. A pack is heavy enough that both deserve their own room.

How this manifests in jinflow

Not yet manifest in the engine. Status: proposed. See the implementation phases / open questions above for the path from this paper to running code.

Numerical neighbors: ← Sense 43: The Cascade — Configuration as Layered Authority · Sense 45: The Loom — two axes, woven →

jinflow is a jazzisnow product

v0.64.7 · built 2026-09-20 19:48 UTC 