Instruments Guide

Instruments are declarative analytical artifacts that compile to SQL. They form the analytics pyramid — each layer builds on the one below.

The Pyramid

Alongside the pyramid, two independent systems capture human knowledge:

• SMEbits — atomic expert observations and checks (free-floating, linked by anchors)
• BitBundles — curated narratives grouping SMEbits

Workflow

All instruments follow the same workflow:

# 1. Create or edit the YAML in the AFS (signals/, theses/, smebits/, etc.)
#    Note: signals are still declared in signals/ directory using probe_id

# 2. Build — make handles validation, compilation, and dbt automatically
jin make

That’s it. make runs all validators and compilers in the correct order as part of its build DAG. You never need to run compilers manually.

To check what’s out of sync without building, run individual compilers with --check:

python3 scripts/signalcompile.py --check
python3 scripts/thesiscompile.py --check
python3 scripts/verdictcompile.py --check
python3 scripts/smebitcompile.py --check
python3 scripts/bitbundlecompile.py --check
python3 scripts/reportcompile.py --check

Signals

A signal (formerly called a probe) is a declarative diagnostic query that scans Gold data for a specific pattern. Each signal produces findings with a continuous score (0-100), impact assessment, and optional polarity (positive/negative/neutral) and direction (improving/stable/worsening).

Location: probes/probe_*.yaml (or assessment_*.yaml) — the YAML format still uses probe_id as the field name

Types: balance, ratio, duplicate, distribution_outlier, temporal_sequence, mandatory_item, trend, silver_audit, entity_filter, enrichment, reconciliation, hand_written, perspective

Output: Standardized findings with finding_id, score (0-100), entity_id, impact (value + unit + dimension), evidence

Theses

A thesis (formerly called a hypothesis) is a testable business question that aggregates evidence from multiple signals.

Location: theses/thesis_*.yaml

Thesis status: confirmed, plausible, not_observed, insufficient

Evidence roles: primary (weight 3), supporting (weight 2), context (weight 1), counter (reduces score)

Verdicts

A verdict (formerly called a diagnosis) explains why a confirmed thesis is true, with root causes and recommendations.

Location: verdicts/verdict_*.yaml

Root cause categories: process_failure, system_failure, data_quality, behavioral, structural, external

SMEbits

An SMEbit is an atomic piece of expert knowledge — attributed, scoped, with an optional testable check.

Location: smebits/smebit_*.yaml

Levels:

• Level 0 (Observation): documentation only, no SQL
• Level 1 (Check): testable assertion, produces confirmed/violated/no_data verdict

Categories: data_quality, mapping, business_rule, process, system, seasonal, historical, structural

BitBundles

A BitBundle groups related SMEbits into a narrative — the “story” layer.

Location: bitbundles/bb_*.yaml

Perspectives

A perspective (formerly called an assessment) is a special signal that aggregates findings from other signals into entity-level health scores.

Location: probes/assessment_*.yaml (type: perspective, contract: signal_contract.v1) — the YAML format still uses the assessment prefix

Reports

A report is a declaratively defined document that compiles to dbt SQL and renders to PDF via WeasyPrint.

Location: reports/report_*.yaml

Section types: kpi_grid, callout, severity_bar, table, text, footer, chart_bar, chart_trend

Output: Registry + data tables in KLS, PDF stored as BLOB in report_pdfs table. Viewable and downloadable in Explorer at /{tenant}/reports.

Queries use template helpers: {{ gold('entity') }}, {{ silver('entity') }}, {{ ref('model') }}, {{ var("tenant_id") }}

Contracts

Each instrument type has a JSON contract defining the schema:

• contracts/gold_contract.v1.json — Gold entities (signals reference this)
• contracts/silver_contract.v1.json — Silver entities
• contracts/findings_contract.v1.json — Signal findings
• contracts/diagnosis_contract.v1.json — Verdict findings
• contracts/smebit_contract.v1.json — SMEbit registry + verdicts
• contracts/bitbundle_contract.v1.json — BitBundle registry
• contracts/report_contract.v1.json — Report registry + data

The Make–Explore–Evolve Cycle

make → build KLS from data + instruments
explore → browse findings, test theses, review evidence
evolve → adjust signals, refine theses, capture SMEbits
→ make again (each cycle deepens understanding)