Week 16 / 2026 — The DuckDB migration and the monolith drain begins

13 April – 19 April 2026 · 178 commits · 0.43.17 → 0.45.1

After two weeks of sprinting (the Sense 14 rename in W14, the Phase 1.5 sprint plus everything else in W15), W16 was a different kind of week. Two long-running engineering threads started in earnest: the migration from duckdb-async to @duckdb/node-api (driven by a SIGBUS bug nobody had a good explanation for), and Phase 5 of the cli/init.py decomposition, where the engine’s 12,062-LOC monolith began draining into focused command modules. The Pipeline Cathedral got its guided tour. Sense 17 (The Cathedral), Sense 18 (The Ledger), and Sense 19 (The Simulation) all got concept papers. And Playworks — a data-aware Playwright harness for the Explorer — landed as a complete test framework.

Three quiet weeks in a row would have meant trouble. This week was quiet and substantive.

The DuckDB migration

The headline incident: Playworks (the new test harness) was crashing with SIGBUS on macOS arm64 after the fourth or fifth distinct table query. Root cause hunt ran for a day. The conclusion: duckdb-async@1.4.2 had a native handle leak that accumulated state per table access. Two distinct DuckDB instances trying to mmap the same SIS file collided.

Thursday: 834d6014 — “migrate from duckdb-async to @duckdb/node-api”. The new binding (maintained by DuckDB Labs) uses a different native architecture that doesn’t accumulate state. Five hours of follow-up commits replaced the dependency across the Explorer build, Docker, build-deploy, and build-desktop scripts (26cca389, 1c913e7f, d927e541). 9f0edeb8 — “close duckdb instance in addition to connection” — closed the second-order leak the migration surfaced.

The SIGBUS still surfaced once more in the frozen binary path. 0005c8df lazy-imported duckdb in commands/sis.py. 12e262e5 disabled Phase 1d SIS publish — “same DuckDB segfault root cause.” e0c8d48f removed the SIS staleness check that crashed. f504ee7b — “don’t ATTACH SIS to KLS Store — eliminates SIGBUS on macOS arm64” — the load-bearing fix. The SIS dual-handle mmap conflict from the W15 baking architecture finally got its name and its workaround.

The monolith drain — Phase 5

jinflow/cli.py had grown to 12,062 LOC. Time to drain.

Phase 5 ran across this week with surgical care. Each commit extracts one command group into a commands/<name>.py module, leaves a re-export shim, and verifies the test suite still passes. The pattern:

• 5a.0 (54c5eac5) — consolidate engine_root resolution
• 5a.1 (da54e6c6) — “rename jinflow/cli.py → jinflow/cli/init.py”. The monolith is now a package.
• 5b (35241341) — build-meta helpers → jinflow/cli/_build_meta.py
• 5c (f398ab60) — cmd_version + cmd_ls → commands/info.py
• 5d (a0415a8d) — cmd_us → commands/config.py
• 5e (f78d556e) — cmd_status + cmd_stat → commands/status.py
• 5f (4fc5205f) — cmd_sources → commands/sources.py
• 5g (f5ae1fde) — cmd_explore + cmd_evolve_* → commands/launch.py
• 5h (13f534f3) — cmd_sis_* → commands/sis.py
• 5i (cf1149fb) — cmd_afs_* → commands/afs.py
• 5j (84c765e6) — cmd_init family → commands/bootstrap.py
• 5k (b1a08812) — cmd_update + cmd_reset + cmd_purge
• 5l (b5086569) — cmd_r2_* → commands/r2.py
• 5m (2f719c67) — cmd_extract → commands/extract.py
• 5n (ac1d6e39) — auth + packs + snapshot (5 commands)
• 5o (a30b0cdb) — cmd_generate + cmd_scripts + cmd_notebook_pdf
• 5p (3def9198) — misc standalone commands (5 of them)

Sixteen extraction commits in three days. By Sunday, the monolith was ~5,100 LOC — a 58% reduction. The pattern that would carry the drain to ~340 LOC over the next two weeks was established here.

Phase 6 followed: 396c98d6 — “CLI smoke suite — 54 —help invocations across all commands”. Every command in the monolith got an automated smoke test, gated in CI.

Playworks — the harness

Monday: 6a3f0d7f — “playworks — data-aware black-box test harness for Explorer.” A Playwright-based test framework where specs declare what they expect (signal counts, money_at_risk numbers, route renderings) and the harness validates against a real KLS. By midweek it had:

• 44d9fc25 — playworks 2a fix + 2b canary + 2c visual scaffold
• d2d813a4 — subset recipes (ui, routes, cardinal, anchors)
• c632d431 — UI cardinal + testid placements + split anchors
• bd534e74 — “playworks stability — drop duckdb override + single-tenant dev mode”
• d334ed1e — commit visual baselines — un-gitignore snapshot PNGs

By Sunday, 188 tests were green. The harness would generate two follow-up structural improvements (W18) but the foundation was built here.

Sense 17 — The Cathedral, in three phases

The pipeline cathedral that landed Sunday W15 was a sandbox. This week made it a guided experience.

• Tuesday: edefbc23 — “docs: Sense 17 — The Cathedral (guided 3D pipeline walkthrough)”
• 1f5b695d — “generate Pipeline Cathedral 3D as third pipeline_graph artifact”
• bdac5ddd — “Sense 17 Phase 1 — guided tour replaces FPS sandbox in Pipeline Cathedral”
• Companion: 9bca6e76 — test that locks in the marker-replacement behaviour

What changes: the FPS sandbox (move around with WASD) becomes a scripted camera rail. Eight stops, progressive reveal, tenant-specific narration, warm atmosphere. From “demo mode” to “executive walkthrough” in 48 hours.

Two more senses on paper

• Sense 18 — The Ledger (c9754e96) — “computational provenance as narrative.” Tuesday.
• Sense 19 — The Simulation (e20dd599) — “from suggestion to modeled impact.” Sunday.

Sense 18 names what the lineage panels from W12 were reaching for: not just “this finding comes from these rows” but “this verdict was derived through this chain of computational acts.” Sense 19 introduces Intervention and Scenario as first-class concepts — the simulator that takes a verdict and projects its forward impact.

aebced9b shipped “The jinflow Experience — extend analytical pyramid to full decision loop” the same day Sense 19 landed — connecting the Simulation back to the rest of the architecture.

The vocabulary sweep finishes

The Sense 14 rename and the Subject Matter / Statement / Dossier display rename had been mostly closed in W15. This week’s sweep finished the long tail.

• 2a17e70e — “vocabulary sweep phase 1 — CLAUDE.md + terminology.yaml”
• 1c7ce973 — phase 2: 62 doc files (archive + notes + rewrites)
• 692e05fa — phase 3: Explorer in-app content
• 19cc835c — phase 4: Python scripts, artifact decoder, etc/
• c40cbd32 — “test: scan only git-tracked files for legacy vocabulary” — the CI guard against regression
• 225600ec — “Sense 14 vocabulary sweep — engine + scripts”
• d7150c28 — “archive migrate_sense14.py — one-time migration is done”

The migration script gets retired with a thank-you commit. The legacy vocabulary CI guard (7a78a3e3) ensures no PR can reintroduce the old terms.

The Four Convictions

Wednesday: 04fd9bad — “Four Convictions — manifesto tab on /what-is-this (EN/DE/FR).” Declarative. Transparent. Independent. Human. The first time the project’s values get a stable rendering as the first tab of Explorer’s What-is-this page.

Smaller but consequential

• Cathedral hosts a tab system (665bb5a0 from W19) — but the foundation work (bac868e9) lands this week: mermaid diagrams for Mental Model + Domain tabs on /what-is-this.
• just ship accepts patch/minor/major (54a4a5e1) — version bump as a CLI argument
• Release notes pipeline (2446d582) — generator, release.sh, Explorer, docs site, all wired
• Backfill releases.md (adbb3a9d) — “complete history (220 releases, v0.2.0–v0.45.0)”
• Dynamic Gold ER diagram from entity_registry + entity_relationships (d2e16f4e) — the diagram is data, not Mermaid markup
• Subject Matter + Dossier vocabulary completes (6e7ff3fd) in terminology.yaml
• Two Worlds design note (8e89b658) — Subject Matter vs Analytical Activities
• Sidebar restructure (a39ea5df) — Backstage, Tools reorg, “What is this?” to top
• Three-logo hero on What is this? (274cedf3 from end of W15, refined this week)
• Build → ship / dbt-build / ship recipe sweep (5d2273d1, eebe6701, 0c7dab9c) — just build → just dbt; just release → just ship + just build for chain
• Playworks gate (01aaab81) — self-contained test gate for the release chain
• Backslash escapes in unquoted search terms (ace7bc63)
• jin make --clean actually wipes build/dbt (9829d6fc)
• JINFLOW_READONLY env var (966a9f44)
• klsStore cold-start visibility + 10s self-heal retry (c533c51b)
• dev:solo sets KLS_SOURCE + JINFLOW_LIVE_ROOT (096bc90f) — the commit that, four weeks later, would surface the dev-server reload regression

Numbers

• 178 commits, 7 days
• 16 Phase 5 extractions; monolith down ~58%
• 1 dependency migration (duckdb-async → @duckdb/node-api)
• 1 SIGBUS root-cause solved (SIS dual-handle mmap)
• 4 vocabulary sweep phases completed
• 3 new design papers (Senses 17, 18, 19)
• 1 Playworks harness from concept to 188 green tests
• 220 releases backfilled into releases.md
• Release line: 0.43.17 → 0.45.1 — two minor versions

What this week made possible

The drain pattern from this week (Phase 5a–5p) would carry the monolith from 12,000+ LOC to under 400 by end of W17. The duckdb-async migration ended the SIGBUS class of bugs that had been intermittent for a month. The Playworks harness made every following feature regression-checked. Sense 17 / 18 / 19 set up the design space for the Cathedral, the Ledger, and the Simulation — three architectural threads that would dominate May.

After the loudness of W14 and W15, this week sounded like a workshop. Tools getting sharpened. Old patterns getting retired. Foundations laid that the next four hundred commits would lean on without noticing.