Sense 54 — The Typology
Sense 54 · In bloom · Last touched 2026-08-20
- last_verified: 2026-07-27
Synced from
docs/design/sense_54_the_typology.mdin the engine repo — that’s the source; this page is a build-time mirror.
Status: proposal · 2026-07-11 Author: claude (Opus 4.7) with Mig Sense number: 54 (verified free on origin/main; Sense 53 The Codex claimed the previous slot)
Refined 2026-07-12. A working session with Mig substantially sharpened this Sense. The current model — semantic type as a data-stream invariant declared at both the ingestion and Gold gates, conversions as typed edges, type-is-not- configuration, and KLS inspector tables replacing the JSON side-file — is in § Evolution — the stream-invariant model (2026-07-12) at the end. Where the sections below conflict with it, that section wins; the most-affected passages are flagged inline.
The one-line pitch
Section titled “The one-line pitch”Every Gold column declares what it is. Not at the signal boundary. Not in the UI. At the site of birth: the entity YAML. Types cascade downward, standards attach where they apply, and every downstream layer inherits — signal aggregates, JinDesk formatting, contract validation, cross-pack composition.
The Codex hosts the standards. The Typology types the columns those standards apply to. Together they answer the metrologist’s question — is this a measurement or is it just a number? — for every value that leaves Gold.
Who this is for
Section titled “Who this is for”Pack builders and canon creators — anyone who declares an entity in an AFS. The Typology is platform vocabulary of the same rank as the Codex: not engine-internal machinery, but training material a pack author learns once and applies across every entity they ship. A data-creator script for a new sensor type consults the Typology to know what kinds are available and which Codex standards it may bind to. A signal author reads it (indirectly, via the compiler) to inherit column types instead of restating them. An JinDesk render path reads it to know how to format a value in a locale. Everyone in the pack-build path sees the Typology; no one has to reinvent it.
Why now
Section titled “Why now”Because signals are the wrong site.
Sense 14.2 Phase 5 typed the aggregates: a signal’s aggregate: {column, type, unit} block validates that a physical quantity carries a UCUM-compatible unit, that a currency carries an ISO 4217 code, that a duration is ISO 8601. Every pack that adopted Phase 5 gained this discipline at the signal boundary.
But the column being read has been read a hundred times before the signal ever touches it — by Bronze macros, by Silver validators, by Gold views, by JinDesk, by every dbt test, by every notebook, by every downstream pack. Along that journey, the type is inferred by each reader from the column name or from context. Mostly they infer correctly. When they don’t, it’s silent.
The signal aggregate is where we notice the type mismatch. The entity YAML is where we should declare it.
Metrology has known this for 150 years. A measurement is not a number — it is a number + type + unit + uncertainty + reference standard. The Bureau International des Poids et Mesures does not accept a value without all five. The Guide to the Expression of Uncertainty in Measurement (JCGM 100:2008 — already in the Codex) says the same. Software conventionally strips this at the CSV boundary and asks users to remember. jinflow’s Four Convictions argue for restoring it: a value that has lost its type has lost its meaning, and a platform that declares itself “Transparent” should not participate in that stripping.
The Typology is the fix: no Gold column may present a value without declaring what kind of value it is.
The three questions the reader wants
Section titled “The three questions the reader wants”Mirroring [[sense_53_the_codex]] and [[sense_15_the_observation]]:
- “What is this column?” — the analyst asks “the value here is 72.4 — is that milligrams per litre? Something else?” Answer: read the entity YAML; every column has a declared type.
- “Does the platform know what this column means?” — the reviewer asks “or is the meaning locked in the author’s head?” Answer: the type is a declaration, not a comment. Compilers read it. Signals inherit it. JinDesk renders it.
- “Can I compose across packs?” — the pack builder asks “if riverflow declares
mass_gand numetrix declaresmass_kg, will jinflow catch it?” Answer: yes, because both declare the same UCUM-typed kind, and the compiler knows they’re comparable up to a unit conversion.
Any framework that answers only one of the three is decoration. A framework that answers all three is a Typology.
The philosophy
Section titled “The philosophy”The Four Convictions read through metrology
Section titled “The Four Convictions read through metrology”Read the [[project_four_convictions]] through the lens of measurement:
- Declarative. A number without a declared type is undeclared. The Typology says: if you claim your Gold contains measurements, you must declare each measurement’s kind. The absence of a declaration is not a permission to guess; it is a defect the compiler flags.
- Transparent. A reader must be able to trace
mass_kg→ SI base unit → BIPM. Today the trace stops at the column name. The Typology extends it all the way to the standard. - Independent. A KLS travels with its AFS archive (see [[project_the_jinflow]]). If the AFS declares types, the KLS is metrologically self-describing — the receiver needs no ambient knowledge to know what the values mean. This is the difference between shipping data and shipping science.
- Human. A person reads “72.4 mg/L per USGS PC00040, uncertainty ±0.3” and understands. A person reads “72.4” and asks. The Typology makes the first form possible without hand-authoring it in every panel.
What a Gold column is supposed to be
Section titled “What a Gold column is supposed to be”An assayed measurement, not a bare number. Assayed = declared type, declared unit where applicable, declared standard where one applies, declared uncertainty policy where meaningful. The declaration is what makes the number legible as science.
Some columns are not measurements. A status enum is a category. A site_id is an identifier. A notes field is uncontrolled text. The Typology has a place for each of these — not everything is a physical quantity, but everything is something, and the Typology insists you say what.
Why the type belongs to the column, not the signal
Section titled “Why the type belongs to the column, not the signal”The signal is the arithmetic. It sums, averages, correlates, thresholds. The type of the inputs is not the signal’s business — it’s the substrate’s business. Asking a signal author to know the type of every column they touch is like asking a chef to declare the wattage of the oven every time they bake a loaf. The oven’s wattage is the oven’s declaration, not the recipe’s.
Move the declaration to the site of birth, and every recipe becomes shorter, safer, and more composable.
The design
Section titled “The design”The load-bearing principle: types are declarations, not runtime data
Section titled “The load-bearing principle: types are declarations, not runtime data”The Typology follows the discipline that TypeScript and Rust follow: types exist to be verified at compile time; they do not travel with runtime values. Every operation the Typology governs — signal aggregation, dimensional arithmetic, standard binding, FK inheritance — is proven correct or refused before SQL reaches DuckDB. The KLS then carries only data whose semantics have already been asserted.
This is not a stylistic choice. It falls directly out of jinflow’s declarative posture: if the AFS is the source of truth, the compiler must be the only authority that decides whether the AFS is internally consistent, and the KLS must never be asked to re-verify what the AFS already declared. Baking types into the KLS would create two truth sites (the declaration and the baked reflection) that could disagree — the same failure mode that makes runtime type systems in dynamic languages feel unsafe.
Three consequences, all load-bearing:
- Declarations live in AFS YAML. Entity
column_types:blocks. Codexstandards/*.yaml. Pack manifests. Pure text, pure git, human-readable, review-able, diff-able. There is no other declaration site. - Verification lives in the compilers.
typologycheck.pyreads the declarations, walks the processing graph (signal aggregates, dbt refs, cross-entity joins), and refuses illegal operations at build time. Every SQL statement that reaches DuckDB has already been proven correct against the declared types. - Runtime artifacts (KLS) carry data, not declarations. The KLS never gets a
_typology_*table. Coverage rollups, standard bindings, and per-column type manifests live instate/as build artifacts (JSON, git-tracked) — read by JinDesk,jin doctor, CI, and any downstream tool without touching the database. (Superseded 2026-07-12 — the artifact moved into two inspector-only KLS meta tables,_<tenant>.column_types/_typology_coverage, because the cloud JinDesk has no AFS on hand. The principle holds — no processing reads them; they are debug symbols, not runtime type tags. See § Evolution.)
The bridge between AFS and KLS is the compiler. Both sides speak to it; neither speaks to the other. The KLS stays about data. The AFS stays about declarations. Types never travel across the boundary because they never need to.
The declaration site
Section titled “The declaration site”Entity YAMLs already carry a columns: block, but today it is a role registry — six slots (primary_key, business_id, label, detail, timestamp, amount) mapping roles to column names. The Typology adds a sibling block, column_types:, that describes each column’s type:
# entities/sensors.yaml — Riverflowentity_id: sensorsdbt_model: gold_sensors
columns: # role registry — unchanged primary_key: sensor_key business_id: sensor_id label: serial_number timestamp: last_seen_at
column_types: # ← new: per-column typology sensor_id: kind: identifier subkind: business_id scope: entity_local # ids that are unique within one entity type serial_number: kind: identifier subkind: external standard: null # no external standard — vendor-assigned model_family: kind: category values: [RK1, RK2, RK3] standard: null current_site_id: kind: identifier subkind: foreign_key references: deployment_sites.site_id nullable: true nullable_reason: sensor_off_fleet status: kind: category values: [active, in_stock, decommissioned, lost] standard: null # pack-defined vocabulary last_seen_at: kind: temporal subkind: instant standard: iso_8601 # ← Codex reference timezone: utc precision: second first_seen_at: kind: temporal subkind: instant standard: iso_8601 timezone: utcFor measurement columns:
column_types: measured_value: kind: quantity quantity_kind: mass_concentration unit: mg/L # UCUM-validated via Codex standard: ucum uncertainty_column: measured_value_uncertainty # column carrying u(x) uncertainty_policy: jcgm_100_2008 measured_value_uncertainty: kind: quantity quantity_kind: mass_concentration unit: mg/L standard: ucum role: uncertainty_of # points back role_target: measured_value observed_property_id: kind: identifier subkind: foreign_key references: observed_properties.observed_property_id standard: usgs_parameter_codes # ← Codex reference phenomenon_time: kind: temporal subkind: instant standard: iso_8601 timezone: utc precision: subsecondThe eight kinds
Section titled “The eight kinds”The Typology recognises eight top-level kind values. Every column is exactly one:
| Kind | Description | Example |
|---|---|---|
quantity | A measured value that has units and (usually) uncertainty | mass_g, temperature_c, pressure_kpa |
identifier | A key that identifies an entity — primary, business, external, foreign | sensor_id, serial_number, iso_country_code |
category | A value drawn from a bounded, unordered vocabulary | status, event_type, region_name |
ordinal | A value drawn from an ordered scale — rank/grade/level. Orderable but not summable | level (hierarchy depth), severity_rank, who_air_quality_band |
temporal | A time-shaped value — instant, interval, duration, period | phenomenon_time, deployment_window, retention_period |
boolean | Two-valued truth (with optional third state for tri-state cases) | is_active, has_calibration |
narrative | Uncontrolled text intended for human reading, not for grouping or filtering | notes, interpretation, technician_comment |
geometry | Structured spatial data with a coordinate reference system | boundary_geojson, geometry_geojson |
Eight is enough to express any Gold column we’ve seen across the seven adopting packs plus the geometric columns Cartography V2 introduces. Sub-kinds refine — identifier.foreign_key, temporal.duration, geometry.polygon — without multiplying the top level.
Note on the seventh kind. The original 2026-07-11 draft of this Sense described six kinds, deferring the geometry question to Cartography Phase 2. Sense 52 V2 (2026-07-11 spec) locked the decision: geometry earns its own top-level kind rather than being crammed into narrative as a serialized-string subkind. Reasoning: a polygon is not narrative. It’s structured data with a coordinate reference system, a serialization format, and spatial semantics that participate in future spatial queries. Modelling it as narrative would lose exactly the semantic content the Typology exists to preserve — and forcing every downstream reader to re-parse it out of an opaque string would violate the “types are declarations” principle by pushing type recovery to runtime. The seventh-kind path costs one more line in the kinds table; the compromise path costs every future geometric column its type-safety.
Note on the eighth kind. ordinal was added 2026-08-18. Earlier drafts modelled ordered scales as a subkind, category.ordinal — a convenient shorthand that hid a real distinction. An ordinal is not a flavour of category; it is a different measurement scale (Stevens’ nominal → ordinal → interval → ratio). Two behaviours turn on the difference. First, order: min/max/median are the ordinal’s native aggregates (they are meaningless on a nominal category), so the ingestion-agg gate accepts ordinal alongside quantity/temporal as orderable. Second, non-additivity: rank 1 + rank 2 is not rank 3, so an ordinal carries no rollup — it neither sums nor averages up a hierarchy. That makes ordinal the honest home for material_classifications.level (a hierarchy depth: ordered, comparable, un-summable), which under the six-kind vocabulary had no true kind — category lost the order, quantity invented an arithmetic the values don’t have. A subkind could not have fixed the agg gate or the rollup question, because both are decided on kind, not subkind. Hence a top-level kind, not category.ordinal.
Three subkinds worth naming explicitly because they name recurring patterns:
category.meta_annotation— a categorical column that annotates another column rather than standing alone. Quality flags (quality_flag,qartod_flag), uncertainty-source classifications (uncertainty_source), packet-status codes (packet_status). Declaresannotates: <other_column_name>so the compiler and JinDesk know the linkage.quantity.geodetic_latitude/quantity.geodetic_longitude— coordinate pairs are physical quantities (degrees) but they carry a projection standard the Typology must know about. Both subkinds bind tostandard: wgs84from the Codex. Elevation stays asquantitywith the appropriate SI unit (m) and an optional geoid-reference note.geometry.polygon/geometry.multipolygon/geometry.linestring— the three primitive subkinds Cartography V2 introduces. Each declares a serializationstandard:(initiallyrfc_7946_geojson, with room for future WKT / TopoJSON / vector-tile bindings) and acrs:(initiallywgs84, with room for tenant-specific projections like Swiss LV95). The Codexbinding_kindsfield enforces that ageometrycolumn may only bind a standard whosebinding_kindslist includesgeometry— no accidentalstandard: ucumon a polygon.
The cascade
Section titled “The cascade”The Typology honours the Sense 43 cascade:
Superseded 2026-07-12. Two claims below are retracted: (a) name-based L0 defaults are not a build-time behaviour — no name inference in
make, only in schema authoring; and (b) tenant/perspective override does not apply to type (type is not configuration and does not cascade — only display config does). See § Evolution §4–§5.
- Engine (L0) — defines the eight kinds, their subkinds, and their Codex-standard hooks. Ships default
unit/standardmappings for well-known column names (ends_with('_at')→temporal.instant, standard: iso_8601,ends_with('_geojson')→geometry.polygon, standard: rfc_7946_geojson, crs: wgs84) so early packs get sensible defaults. - Pack (L1) — declares
column_types:per entity in the pack repo. This is the primary site of authoring. - Tenant (L2) — may override a pack’s declaration when a tenant genuinely differs (a custom
statusvalue, a non-standard unit). Overrides are narrowing or substituting, never widening. - Perspective (L3) — reserved for future use. Not expected in Phase 1.
Cascade merge is deep — a pack declares column_types.measured_value.unit: mg/L, a tenant may add column_types.measured_value.reference_range: {low: 0, high: 50} without restating the unit.
The propagation
Section titled “The propagation”Once declared, types flow through every downstream layer:
- Silver — already preserves the column; now the compiler asserts the Silver column shape matches the declared type (e.g. a
quantitycolumn must be a numeric SQL type, atemporal.instantmust be a timestamp). - Gold — the Gold contract JSON (which already lives as a git-tracked build artifact, not in the KLS) is regenerated to embed the types, so cross-pack readers can consume the contract without reading the entity YAMLs.
- Signals — the
aggregate: {column, type, unit}block from Sense 14.2 becomes derived rather than declared. Signal authors writeaggregate: {column: measured_value}and the compiler fills intype: quantity, unit: mg/L, standard: ucumfrom the Typology. Signal YAMLs get shorter. - JinDesk — column formatting, precision, unit-aware display, and Codex badges all read from the Typology instead of per-column
display:config. The Sense 43display:cascade stops carrying redundant type declarations. - Bell / Landscape — surface assay coverage as a first-class quality metric: what fraction of a pack’s Gold columns carry declared types? What fraction of those bind to a Codex standard?
The relationship to Sense 53 and Sense 14.2
Section titled “The relationship to Sense 53 and Sense 14.2”The three form a stack, and Sense 54 is the missing floor:
┌────────────────────────────────────────────────────────────┐│ Sense 14.2 — Typed signals ││ Signal aggregates carry type + unit + standard ││ Inherit from the Typology when column is declared │├────────────────────────────────────────────────────────────┤│ Sense 54 — The Typology ← this doc ││ Every Gold column carries a declared type ││ References standards from the Codex where they apply │├────────────────────────────────────────────────────────────┤│ Sense 53 — The Codex ││ Platform-hosted registry of external standards ││ UCUM, ISO 8601, ISO 4217, USGS PC codes, SensorThings, ││ JCGM 100:2008, ... │└────────────────────────────────────────────────────────────┘Read top-down: a signal aggregates typed columns that reference platform standards. Read bottom-up: the platform hosts the standards, packs bind their columns to those standards, signals inherit the bindings and do arithmetic on top.
The three senses were designed in the wrong order — signals first (Sense 14.2, 2026-05→07), standards next (Sense 53, 2026-07-09), columns last (Sense 54, this doc). That’s fine. The order of design rarely matches the order of dependency. What matters is that the three now fit.
Two shared surfaces with the Codex
Section titled “Two shared surfaces with the Codex”The Typology and the Codex share ownership of two mechanisms. Neither belongs entirely to one Sense; both are cleanest as a division of labour:
-
The
binding_kindsfield on Codex standards. Eachengine/standards/*.yamlgains abinding_kindslist declaring which of the six Typology kinds a standard is legitimate to bind to. The list is authored honestly — if a standard has multiple legitimate roles (ISO 4217 values are quantities and ISO 4217 codes are identifiers; USGS PC codes act as FK identifiers and as bounded categories), the list enumerates all of them rather than pretending one is primary. Most standards carry one kind; some carry multiple; some are structural rather than column-typing and carry an empty list, telling the checker to skip them entirely.Standard binding_kindsucum[quantity]iso_4217[quantity, identifier](currency values and currency codes)iso_8601[temporal]iso_80000[quantity]jcgm_100_2008_gum[quantity](uncertainty policy on measured values)jinflow_naming[identifier]wgs84[quantity](geodetic coordinates)usgs_parameter_codes[identifier, category](pcode as FK toobserved_properties; pcode as bounded vocab used directly)iso_7027[quantity](turbidity FNU/NTU)astm_d3977[quantity](SSC gravimetric)wmo_guide_no_8[quantity](air temperature, relative humidity)ogc_sensorthings_1_1[](structural entity contract)ogc_waterml_2_0[](structural)rfc_7946_geojson[geometry](moved from[]when Cartography V2 landed the seventh kind)odbl[](licence, not column-typing)This turns
standard: ucumin a column declaration from a stringly-typed reference into a compile-time-checked pairing —typologycheck.pyrefusesstandard: ucumon anidentifiercolumn, because the Codex tells it that’s not a legal binding. Sub-kind refinement (quantity.temperature,quantity.currency) is deferred to Phase 2; Phase 1 checks top-level kind only. Sense 53 owns the field’s schema; Sense 54 owns the checker that consumes it. -
The coverage rollup —
state/typology_coverage.json. The Typology compile writes a JSON artifact (one entry per Gold column:entity_id,column,has_type,has_standard,standard_id,kind) tostate/typology_coverage.jsonand logs the write to the Sense 47 Log with an SHA-256 for auditability. This is a build artifact, not a database view — the KLS never sees it. The Codex reader consumes the JSON for the pack-adoption panel that answers “what fraction of this pack’s columns are assayed, and against which standards?” — “riverflow is 71% assayed, 62% UCUM-bound, 100% ISO 8601-bound.” Real numbers, not marketing. Multiple downstream tools read the same artifact: JinDesk,jin doctor, CI, packaging. No round-trip to the database because there was never a reason for a round-trip. The Codex owns the JSON’s schema (viastandard_contract.v1.json’s coverage-block); the Typology compile writes the rows; anything with filesystem access consumes them. This absorbs the “grounding grade” idea that was briefly a candidate Sense 55 (retired 2026-07-11 into this rollup during sister-9’s review): a per-signal grounding grade ismean(column_grades)where eachcolumn_gradeis the fill-rate of that column’s Typology declaration. Same insight; the rollup lives here, one level above the per-column truth.
The Codex loader hook
Section titled “The Codex loader hook”For declarations that reference a Codex standard, typologycheck.py must be able to resolve the reference. The engine loads all engine/standards/*.yaml at startup and exposes them as a lookup dictionary keyed by standard_id. The Typology checker consults this dictionary and refuses declarations that reference unknown standards. Cheap, and it upgrades every standard binding from a string to a typed reference. The loader lives in the Codex module — Sense 54 is a consumer, not the owner.
The migration ramp
Section titled “The migration ramp”Every pack has hundreds of untyped Gold columns. Making them all typed on day one is a fantasy. Making them typed over time, with visible progress, is achievable.
Four moves:
- Assay coverage as a first-class metric. Every
jin makewritesstate/typology_coverage.jsonwith one entry per Gold column:entity_id,column,has_type,has_standard,standard_id,kind. A pack-level rollup lives in the same file:packs/numetrixis 34% assayed,packs/riverflowis 71%. Coverage appears on the Landscape, on the Bell, and in thejin doctoroutput. The JSON’s schema is owned by the Codex (see the shared-surfaces section above); the Typology compile writes the artifact; every downstream reader (JinDesk, CI, packaging) consumes it directly from filesystem. The KLS is not involved — types are declarations, not runtime data. - The zero-day defaults. The engine ships name-based inference rules:
ends_with('_at')→temporal.instant,starts_with('is_')→boolean,ends_with('_id')→identifier. On day one, ~40% of most packs’ columns get typed for free. The residual is the interesting part — measurement quantities, categorical enums, foreign keys — which is exactly where the human declaration adds value. - The strict-mode toggle. Packs opt in per entity to
typology: strict— meaning the compiler fails if any column in that entity lacks a declared type. Strict mode spreads entity-by-entity, at the pack’s own pace, until eventually the pack manifest reachestypology: strictat the top level. That’s the moment the pack is fully assayed. - The sidecar hint channel (Phase 3+, optional). Data-creator scripts (synthetic generators, extractor pipelines) may ship a sidecar manifest declaring the shape of the data they produce — kind, unit, expected range, per output column. The Typology compile warns when the entity YAML declaration diverges from the sidecar hint. Not authoritative — the entity YAML always wins — but a diagnostic that catches “generator was updated, entity YAML wasn’t.” Analogous to how OpenAPI generators produce type hints for downstream consumers. Deferrable; not part of Phase 1.
The Phase 1 testbed
Section titled “The Phase 1 testbed”Riverflow’s substrate is the natural pilot: measurement columns already carry unit_of_measure metadata, foreign keys already reference OGC-modelled ObservedProperties, calibration binds uncertainty back to a Calibration event. But full riverflow is ~200 columns across 19 entities — enough to prove the shape and enough to exhaust a session budget before Phase 1 lands.
The pilot narrows to three entities that together hit the interesting patterns:
sensors— 7 known-shape columns. Coversidentifier,identifier.foreign_key,category,temporal.instant. The simplest testbed for the declaration mechanics.measurements— 14 columns including the quantity-with-uncertainty pattern (result_value+uncertainty+uncertainty_source+quality_flag). Exercisesquantity,category.meta_annotation, and the standard binding to UCUM and ISO 7027 / ASTM D3977 (two of the three new Codex standards landed onclaude/9thon 2026-07-11).observed_properties— 12 columns with an existing USGS parameter-code binding viapcode. Turns a de-facto binding into a declared one and validates the Codex-reference mechanism against real content.
If these three entities compile cleanly with the column_types: block, land the Typology in JinDesk, and produce a non-empty state/typology_coverage.json, Phase 1 is proven. The remaining 16 riverflow entities become clerical work; the migration to numetrix, hrcentral, alptrack, and millesime becomes the next natural cohort.
What Phase 1 catches at build time
Section titled “What Phase 1 catches at build time”The compile-time verification discipline lets typologycheck.py refuse six categories of illegal operation before dbt runs a single query. Each of these is a class of bug that today shows up as silent Bronze-to-Gold data corruption and is caught (if at all) months later when a signal produces a suspicious number. Moving them to compile time turns them into build failures — JinDesk never renders a wrong answer because dbt never ran.
- Unit mismatch on aggregate — a signal aggregating a
mg/Lcolumn and declaringunit: g/Loutput without a UCUM conversion in the aggregate spec → refused. - Dimensional incompatibility — arithmetic that would subtract a temperature from a mass, or add currency to a duration → refused (the ISO 80000 quantity-kind lattice tells the checker the operation is undefined).
- Method mismatch — a threshold rule written against ISO 7027 turbidity applied to a column that doesn’t declare
standard: iso_7027→ warned (the Codexbinding_kindsfield says the standard is legitimately bindable; the column just doesn’t cite it). - Broken FK inheritance — a signal joining on
sensor_idwhere the entity YAML declares the FK actually references a different table → refused (theidentifier.foreign_keysubkind carries its target; the checker walks it). - Cross-pack unit drift — a
mass_gcolumn in one pack and amass_kgcolumn in another compared or subtracted without conversion → catchable at cross-pack composition (both packs declare units; the checker computes the compatibility). - Uncertainty-drop violation — a signal aggregating measurement columns without carrying the paired uncertainty column through per JCGM 100:2008 → warned (the
uncertainty_column: <name>link on the primary quantity tells the checker what to expect).
All six are pure compile-time work. Zero database growth. Zero runtime cost. Every one of them is what a scientific reviewer would ask about a KLS delivered to them — “how do you know the units are consistent? how do you know the joins didn’t drop uncertainty?” — and the answer becomes “the build refused the wrong versions.”
What this isn’t
Section titled “What this isn’t”- Not a rewrite of Sense 14.2. The typed-signals machinery keeps working. The change is that the signal YAML gets shorter because the type is inherited from the column instead of restated.
- Not a rewrite of Sense 53. The Codex hosts the standards regardless of whether any column binds to them. The Typology is one reader of the Codex; SensorThings API endpoints, Bell disclosures, and JinDesk standards pages are other readers.
- Not a display concern. The Sense 43
display:cascade keeps its role: precision, format, locale. The Typology says what a column is;display:says how to render it in this locale. Two vocabularies, one cascade shape. - Not a validation framework. The Typology declares. It doesn’t check that a value in the Gold table conforms to its declared type — that’s a downstream job for Silver validation, dbt tests, or
jin doctor. Declaration and validation are separable, and mixing them is what has bitten every schema system that ever tried. - Not a separate coverage-scoring Sense. A briefly-considered Sense 55 “Bedrock” would have introduced a per-signal grounding grade (dimension + unit + method standard + range + uncertainty). Sister-9 retired that proposal during this Sense’s review (2026-07-11) because it was the same insight one level below its natural home: the coverage rollup lives in the Typology, and a signal grounding grade is
mean(column_grades). Same result, cleaner home, one fewer Sense.
What it costs
Section titled “What it costs”- Every adopting pack walks its Gold columns and assigns types. For a 200-column pack, this is 200 declarations. Realistic estimate: 8–16 hours per pack for a first pass, mostly clerical, guided by name-based defaults.
- Two new compiler steps —
typologycheck.py(reads declarations, walks the processing graph, refuses illegal operations) andtypologycompile.py(writesstate/typology_coverage.jsonand logs the write to the Sense 47 Log). Both mirror the shape ofsignalcheck/signalcompile. Neither writes anything to the KLS. - One new dbt macro —
column_type(entity, column)— so Silver validators can{{ column_type('sensors', 'last_seen_at') }}to read the declared type at build time. - JinDesk refactor to consume the typed contract instead of per-column display config. Modest, because the display cascade is already the reader; only the source of the type field moves.
What it unlocks
Section titled “What it unlocks”- Signal YAMLs shrink 15–25% because types are inherited.
- Cross-pack composition becomes safe —
mass_kgin riverflow andmass_kgin numetrix are known to be the same quantity kind and the compiler can prevent an accidental subtraction of temperature from mass. - JinDesk’s unit-aware rendering, Codex badge display, and precision formatting all become automatic instead of hand-authored per column.
- “Does jinflow speak SI?” gets a per-column answer, not a per-pack marketing claim. Every quantity column that binds to UCUM contributes to the platform’s assayed surface.
- The KLS becomes metrologically self-describing. A KLS shipped to a scientific reviewer carries not just the values but the vocabulary that makes them values.
Phase plan
Section titled “Phase plan”Phase 1 — Declaration site + coverage artifact. Add the column_types: block to the entity YAML schema. Ship typologycheck.py (validation) and typologycompile.py (writes state/typology_coverage.json, logs to Sense 47). No behavioural changes downstream — this is observation, not enforcement. Ship name-based zero-day defaults so packs open Phase 2 with 40% coverage for free. Nothing lands in the KLS. (Superseded 2026-07-12 — see § Phase 1 as shipped: typologycompile bakes two inspector-only KLS tables (not a JSON side-file); there are no name-based defaults (explicit only); pre-flight is fail-hard, not observe-only.)
Phase 2 — Signal inheritance. Modify signalcompile.py so aggregate: {column: X} (without explicit type or unit) reads the declared type from the Typology. Existing signals with explicit types continue to work; the compiler warns when the explicit type disagrees with the declared type.
The Phase 2 stress test is riverflow’s RegulatoryThreshold entity (landed on claude/9th 2026-07-11): a threshold_value column whose unit varies by row context — g/L for SSC, mg/L for DO, Cel for temperature, 1 for pH. Sense 14.2 Phase 5 could not express this and had to drop the entity-level aggregate: block. A Typology declaration per row context (probably a unit_column: unit_of_measure field on the type declaration, indirecting through the sibling unit_of_measure column) is the natural expression. If Phase 2 does not gracefully handle “type varies by row context”, we haven’t solved the composition problem.
Phase 3 — JinDesk inheritance. JinDesk’s rendering path reads Typology first, display: cascade second. Pack-level display: blocks for column-level precision/format become deprecated in favour of the Typology declaration.
Phase 4 — Strict mode. Per-entity typology: strict toggle. Compiler fails if any column in that entity lacks a declaration. Adoption is pack-by-pack, entity-by-entity, with coverage metrics visible throughout.
Phase 5 — Codex enforcement. Types that name a Codex standard get checked — a quantity column with unit: mg/L, standard: ucum gets validated against the UCUM parser at compile time. Column-level Codex conformance appears in the Bell and on the Landscape.
What we owe the reader
Section titled “What we owe the reader”A person reading the Typology should be able to answer, of any Gold column, three things in under a minute:
- What kind of value is this?
- What are its units, standards, and uncertainty policy, if any?
- Where in the AFS is that declaration written?
If the answers exist and are pointer-verifiable to a YAML file, we have a Typology. If they exist only in the author’s head, we have a schema pretending to be one.
The Codex made jinflow’s external vocabulary legible. The Typology makes jinflow’s internal vocabulary legible. Together they close the loop.
Evolution — the stream-invariant model (2026-07-12)
Section titled “Evolution — the stream-invariant model (2026-07-12)”This section is the outcome of a working session with Mig on 2026-07-12. It refines the model above and, where it conflicts with earlier sections, it is authoritative. The most-affected earlier passages carry inline “Superseded 2026-07-12” markers pointing here; they are retained for history.
1. What a semantic type truly is
Section titled “1. What a semantic type truly is”A semantic type is not a label on a Gold column at one site. It is the identity of a data stream — its meaning, dimension, and unit — that is invariant under storage and changes only under an explicit, declared conversion. It is not a fact about one place in the pipeline; it is a property that travels with the stream through every stage of the river.
This reframes the whole Sense. Typing a Gold column is not the goal — it is one of two gates on a stream whose type we track end to end.
2. Two type layers — and the storage type is the floor
Section titled “2. Two type layers — and the storage type is the floor”There are two distinct type layers, complementary, not competing:
| Layer | Where | What it says | Scope |
|---|---|---|---|
| Storage type | csv_schemas/{source_system}.yml | string / number / date / boolean / integer — how it’s encoded | per source system |
| Semantic type | the Typology (column_types:) | quantity+UCUM+dimension, identifier+FK, category+vocab, temporal — what it means | source-agnostic |
The storage layer already exists: csvschemacheck.py validates source CSVs
against the declared schema (L0 existence → L1 syntax → L2 headers), and that
schema already carries a type: per column — a storage type. The semantic
type sits on top of the storage type as its floor: a quantity must rest
on a number/integer, a temporal on a date, a boolean on a boolean.
A semantic kind on an incompatible storage type is a compile error — make
stops. (Storage type is per-source because encoding varies; semantic type is
source-agnostic because meaning does not — a concentration is a concentration
whether OPALE or SAP delivered it.)
3. The type lives at both gates — and the redundancy is the safety
Section titled “3. The type lives at both gates — and the redundancy is the safety”The semantic type is declared at two gates, and carried through the stages between:
- At ingestion (earliest possible) — a prerequisite to entering the pipeline. “This stream delivers °C / counts / meters.”
- At Gold — a prerequisite to every instrument we apply. “This column is SSC in g/L, averageable.”
Between the gates is the river, where conversions happen (raw counts → NTU →
g/L; m → ft; m × m → m²). Declaring the type at both gates looks redundant —
and in the best case it is. But the redundancy is a checksum: the source
type carried through the declared conversions must equal the independently
declared Gold type. When they agree, all is well. When they disagree, the
mismatch localizes the conversion bug to the exact stage where the carried
type stopped matching — and make stops there and names it. Data that never
reaches Gold still needs a type: an untyped intermediate feeds a derivation
that does reach Gold, and that is precisely where silent conversion errors
hide today. Typing as early as possible is what makes the failure loud and
local instead of a suspicious number three layers downstream.
4. Conversions are typed edges — explicit only, no build-time magic
Section titled “4. Conversions are typed edges — explicit only, no build-time magic”Each conversion is a typed edge: a declared operation taking a stream of
type A to type B by a known rule — a commensurable unit conversion (m→ft), a
standard-cited calibration that legally changes the dimension (counts→NTU via
ISO 7027), or a dimensional derivation (m×m→m²). Because the type is known as
early as possible, a conversion is type-driven and type-checked rather than
a hand-coded transform that is silently wrong. °C + m is refused.
No name-based inference anywhere in the pipeline. “The column ends in
_km2, so it must be km²” is forbidden — it is lying with a heuristic, right
until someone names a column depth_km. This retracts the “zero-day
name-based defaults” of the migration ramp and Phase 1 below (ends_with('_at')
→ temporal, “40% for free”), and it retires the pack config.yml unit-guessing
regex. Magic is allowed only when authoring a schema — a scaffolding tool
may propose “area_km2 looks like a quantity in km²” for a human to review
and commit. Once a schema is injected into make, it is read literally;
there is zero inference at build time. Suggestion-time magic ≠ build-time magic.
This is how trust and safety are maintained: the truthful space is fixed,
explicitly, in a reviewed schema, before a single row is built. make is the
analytical security boundary, and it stops on any violation (fail-hard,
like the extraction contract).
5. Type is not configuration (the Sense 43 boundary)
Section titled “5. Type is not configuration (the Sense 43 boundary)”The Cascade (Sense 43) is a config engine: values produced and consumed independently, contextual, overridable, cascaded L0→L3. Type is the opposite. Type and data are one — you cannot override a declared type any more than you can relabel °C as °F; changing it means it is different data. Type is upstream of the config engine and never enters it. This supersedes the cascade section’s “tenant may override a declaration” — overrides apply to display config, never to type.
What config may touch are the truthful leftovers — you can only
“configure the truth”: a render-unit via a commensurable conversion (show
m³/s as L/s), precision, and at most a choice among commensurable legal
aggregations (mean vs weighted-mean) — never whether an illegal one is allowed.
The type carries the dimension; the dimension bounds exactly which
configurations are truthful. Illegal moves are refused, not warned:
relabel-without-convert, incommensurable render (“length in seconds”), illegal
rollup (a concentration as additive).
The measure roll-up policy is a type property, not config. Whether a value
may be summed is a fact of the quantity — extensive (count, load →
summable) vs intensive (concentration, temperature → not). It is declared
once, truthfully, on the type, immutable, and does not cascade. It is named
rollup: (vocabulary additive | averageable | non_additive | distinct_count) — not aggregation:, to avoid collision with Sense 43’s
documented config-merge “aggregation policy” (replace/union/append/concat,
how config keys merge across cascade levels). Different beast.
Complete type coverage is a data-engineering requirement, not a metric. “There is only one possible unit for a stream that delivers °C.” An untyped Gold column is unfinished data engineering, surfaced loudly; strict mode (Phase 4) refuses to build an entity below full truthful coverage.
6. Where the artifacts live — KLS inspector tables, not a JSON side-file
Section titled “6. Where the artifacts live — KLS inspector tables, not a JSON side-file”This supersedes the state/typology_coverage.json model throughout the
sections above. The load-bearing principle still holds — nothing in
processing reads a type — but the artifact moved, for a concrete reason: the
cloud JinDesk has no AFS on hand, so a git-tracked side-file cannot reach a
browser in Frankfurt. typologycompile bakes two tables into the tenant meta
schema:
_<tenant>.column_types— one row per declared column (entity_id, dbt_model, column_name, kind, subkind, standard, unit, unit_column, rollup, ref_target)._<tenant>.typology_coverage— one row per Gold entity, the discipline gauge.
These are debug symbols (DWARF), not runtime type tags (RTTI). Nothing in
compile / dbt / signal processing may read them — types are inherited at
compile time; these tables exist only so JinDesk (an inspector) can
show a declared type and derive presentation. The moment a signal reads
column_types to decide a type at query time, it becomes the runtime-type-info
model we avoid. The guard is loud in the bake code and in a DuckDB
COMMENT ON TABLE. They are write-once, rebuilt every make. This is the
same pattern as tenant_standards (a497231d): descriptive metadata the cloud
viewer needs travels in the KLS meta schema. (Reconciles with §“types are
declarations, not runtime data”: processing never reads these; they are
inspector-only.)
Coverage is reconciled against the actually built Gold columns (read from
information_schema in the KLS the compile is writing into), so it reports the
honest gauge — typed / untyped / phantom (declared-but-absent, usually a
typo) — not the declaration alone.
7. Phase 1 as shipped (2026-07-12)
Section titled “7. Phase 1 as shipped (2026-07-12)”column_types:on the three testbed entities (34 columns).typologycheck.py— top-levelkindvalidity + Codexbinding_kindsconformance +rollupvocabulary. Explicit-only, no name inference, fail-hard.typologycompile.py— bakes the two inspector-only tables; coverage reconciled against built Gold; pre-flight aborts on any invalid type (never bake a lie).ucumwidened to[quantity, category](56586a2b, sister-9) so unit-of-record columns (result_unit,unit_ucum) bind legally.
The typed river — type carried through every stage, conversions as typed
edges, make refusing any edge that does not preserve or legally transform the
type — is the North Star. It is the same dimensional-propagation machinery
already flagged for derived columns (§ derived-unit computation), applied
stream-wide. Phase 1 anchors the Gold gate and the reconciliation floor;
Phases 2–5 build outward toward the full river.
8. The polymorphic edge (V2)
Section titled “8. The polymorphic edge (V2)”The V1 type_conversions edge is per-(entity, column) with a single
standard — exact for a mono-property column, but result_value is
polymorphic (§2 / unit_column), and its conversion method varies by
observed property just as its unit does: turbidity’s L1→L2 hop is ISO 7027
(nephelometry), a temperature reading’s is a thermistor calibration — not
nephelometry. So a single-standard edge tags every result_value chain with
iso_7027: correct for turbidity, wrong for temp/level. (sister-9 flagged this
after wiring the Rail to the edge table, 117aa460; the demo host seeds a
turbidity reading, so it renders correctly today.)
The gap is the same polymorphism that unit_column bears. The discriminator
is the general fix; reshape is a special-case simplification — and the
platform supports both (polymorphism is first-class; normalize only where it’s
reasonable — see feedback_polymorphism_and_normalization_both). The team
converged here (sister-9 + Mig, 2026-07-13):
- The discriminator (general). A per-
observed_propertyvariant→standard map onderived_from, keyed by the same columnunit_columnleans on (observed_property_id). Kept pure declaration (a declared map, not agold_observed_propertiesdata-join) so the edge table stays checksum-grade. Correctness first, citations later: the first win needs no new Codex — mapop_water_temp → standard_ref: nullso temp/level stop wrongly citing ISO 7027. A thermistor calibration may have no governing external standard, andnullis the honest answer; positive citations (stage-discharge → USGS rating methods) are seeded as they exist. - Reshape (the special case, not a replacement). Splitting
result_valueinto per-property columns dissolves the discriminator — but only when the discriminator has a small, fixed domain, and it walks away from the OGC SensorThings Observation model riverflow deliberately chose. Currency is the counter-proof: an open domain with time-indexed FX (amount + currency) can never be reshaped intoamount_usd / amount_eur / …— you keep(amount, currency)and discriminate. So reshape is the exception; the discriminator is the rule.
A layer separation to keep straight. A refinement edge is a cross-dimension
conversion (counts→NTU→g/L), method-cited, a type_conversions row. A currency
conversion (CHF→EUR) is same-dimension, rate-based, time-indexed — it is a
render-time data join (a rate table, the same shape as gold_calibrations),
not a type-level standard and not an edge. Money’s FX pressures the
data/join layer, never the edge-standard design.
The one genuine hinge is the riverflow reshape timeline — the pipeline
owner’s call, not the Typology’s. If a reshape is imminent for other reasons,
the discriminator’s positive-citation work is throwaway (the null-correctness
fix is not — nulling a lie is honest regardless of shape). Until then, the V1
edge is documented (in the riverflow declarations) as turbidity-scoped.
Related work
Section titled “Related work”- [[sense_14_2_typed_signals]] — the aggregate-boundary types this Sense pushes down to the substrate
- [[sense_53_the_codex]] — the platform’s standards registry, which the Typology references (and which gains a
binding_kindsfield to make bindings compile-time-checked) - [[sense_43_the_cascade]] — the cascade machinery types will inherit through
- [[sense_15_the_observation]] — the “signed strategic claim” idea; this Sense signs every column with its type
- [[sense_47_the_log]] — the write-once execution log that records
typologycompileruns and gives the coverage artifact its audit chain - [[sense_52_the_cartography]] — the geometry-serialization edge case (RFC 7946 GeoJSON) that this Sense defers to Cartography Phase 2
- [[project_unit_standards_architecture]] — three-level unit standards (ISO 80000, UCUM, quantity kinds) that the
quantitykind builds on - [[project_four_convictions]] — the philosophical frame this proposal lives under
Recently landed Codex standards that become Phase 1 binding targets (all on claude/9th, 2026-07-11):
- ISO 7027 — turbidity nephelometry (
quantity.mass_concentrationin NTU or FNU) - ASTM D3977 — suspended sediment concentration gravimetric (
quantity.mass_concentrationin mg/L) - WMO Guide No. 8 — meteorological instrument conventions (air temperature, relative humidity — deferred to a future riverflow phenomenon)
Numerical neighbors: ← Sense 53 — The Codex · Sense 55: The Broadcast →