jinflow make

The make command is the build pipeline. It compiles YAML instruments, runs dbt, and produces the KLS.

Make is the one build command.

make is the single command that takes a tenant from source files to a KLS. It runs extraction as Phase 0 (pinned by pipeline.yml), then enriches, compiles, and builds. Every build is reproducible from the AFS commit it was built on.

See Operating Modes for how this runs across deployment topologies.

Basic Usage

jin make                                  # build current tenant
jin make --tenant hrcentral.vai            # build a specific tenant
jin make --pack hrcentral                  # build all tenants in a pack
jin make --all                            # build all tenants across all packs

Inputs Gathered Inside Make

make gathers everything it needs on the way in, and commits the result to the AFS so every build is auditable:

1. Source data — Phase 0 runs extraction, governed by the extraction contract. Every file is pinned by SHA-256 before it’s read.
2. (Phase 1d — publish to the AFS — was retired on 2026-07-26. Notes, bookmarks and other operational citizens live in the OPS and are no longer copied into the AFS at build time. The OPS outlives every KLS and is never rebuilt.)
3. DLZ sync — Phase 0b copies changed CSVs from the Data Landing Zone into source/.

After these phases make has everything it needs and runs the build deterministically from the AFS alone.

jinflow inspect                           # read-only view of the extraction surface (does NOT extract)
jinflow inspect --contract                # render the full pipeline.yml contract
jinflow inspect --check                   # verify all declared SHA-256 hashes without running extraction

Where the OPS lives depends on the operating mode. In local mode make reads a file; in cloud modes it calls JinDesk API. The developer doesn’t care.

Data Sync (—sync)

--sync copies CSVs from the Data Landing Zone (DLZ) into the tenant’s raw/ directory before building. Only changed files are copied (SHA-256 comparison against the previous manifest).

jin make --sync                           # sync DLZ → raw → build
jin make --all --sync                     # all tenants, only copy changed CSVs
jin make --tenant hrcentral.vai --sync     # single tenant with sync

Extraction (—extract)

--extract runs source-system-specific extraction scripts (e.g. Excel → CSV) in the DLZ before syncing. Implies --sync.

jin make --extract                        # extract Excel → CSV → sync → build
jin make --all --extract                  # extract + sync + build all tenants

Extraction is governed by the extraction contract (pipeline.yml at the tenant AFS root), which pins every source file by SHA-256, declares its expected schema, and fails hard on any mismatch. Read the Extraction Guide for the full trust-boundary model, the six source types, and the jin inspect --contract inspection command.

Not all source systems have extraction pipelines. Tenants without one skip extraction and proceed to sync.

Operating Modes

make adapts to the operating mode without changing invocation:

• Local — make runs on your machine, reads and writes local files.
• Proxy (P2P2P) — make runs on the data owner’s machine; the proxy serves the resulting KLS via tunnel.
• Cloud (R2) — make runs locally, then jinflow cloud sync uploads the KLS to R2.
• Semi-Cloud — make runs locally; the OPS lives in the cloud and is read directly, not fetched into the build.
• Make-as-a-Service — make runs in the cloud on push or cron; no local invocation.

Clean (—clean)

--clean clears intermediary build artifacts (compiled SQL, enriched CSVs) and forces a full rebuild. Does not touch the AFS or raw data.

jin make --clean                          # clean rebuild
jin make --tenant hrcentral.vai --clean    # reset single tenant

Snapshots (—snapshot)

--snapshot freezes the KLS after a successful build. The snapshot is immutable — make will refuse to overwrite it.

jin make --snapshot                       # auto-tagged: YYYYMMDD-HHMM
jin make --snapshot post-audit            # named snapshot

Flags

Flag	Effect
--sync	Copy changed CSVs from DLZ → raw/
--prune	Remove files from raw/ that no longer exist in DLZ (with —sync)
--extract	Run extraction scripts, conditional on Excel file changes (implies —sync)
--clean	Clean rebuild, clear intermediaries
--snapshot [tag]	Freeze KLS after build
--all	Build all tenants across all packs
--pack <name>	Build all tenants in one pack
--tenant <id>	Build a specific tenant (pack.tenant)
--continue-on-error	Don’t stop on compiler/dbt errors
--quiet	Suppress dbt output
--skip-enrich	Skip CSV enrichment phase
--skip-validate	Skip CSV schema validation
--metadata-only	Only rebake metadata (logo, branding, about) — skip all build phases

Build Phases

0a  Extract          (--extract, source-system-specific)
0b  Sync from DLZ    (--sync, copies changed CSVs to raw/)
1a  Validate CSVs    (schema check against source system definition)
1b  Enrich CSVs      (taxonomy generation, backfill, normalization)
2   Compile          (YAML → dbt SQL: signals, theses, verdicts, ...)
3   dbt build        (Bronze → Silver → Gold → Signals → Theses → ...)
4   Post-build       (pipeline graph, audit reports, PDF generation)
5   Stamp + package  (metadata, AFS archive, snapshot)

Conditional Behavior

• —extract hashes Excel files against _extract_manifest.json — skips extraction if unchanged
• —sync only copies CSVs whose SHA-256 hash differs from the previous sync manifest
• —prune removes files from raw/ that no longer exist in DLZ (default: warn only)
• No-extraction systems are skipped with an info message (no xslx/ directory → no extract pipeline)
• Metadata-only changes auto-detected: if only tenant.yml, about.yaml, logo, or jinflow.yml changed since the last build, rebakes metadata without running dbt. Also available as --metadata-only
• Incremental builds reuse the existing KLS — dbt skips models that haven’t changed
• Failed builds are detected on retry (missing KLS stamp → automatic fresh start)

Examples

# Daily routine: sync new data, rebuild
jin make --all --sync

# After editing signals/theses: rebuild without re-syncing data
jin make

# Fresh start after schema changes
jin make --clean

# Just update logo/branding (no rebuild)
jin make --metadata-only

# Freeze a good state before making changes
jin make --snapshot before-refactor

# Build only the millesime pack
jin make --pack hrcentral --sync

# Extract new Excel files, sync, build
jin make --tenant hrcentral.vai --extract