Sense 14.2 Phase 5 — Typed Entity Aggregates

Sense 14.2 · Folded in · Last touched 2026-08-20

Folded into Sense sense-14-2.

• last_verified: 2026-07-27

Synced from docs/design/sense_14_2_phase5.md in the engine repo — that’s the source; this page is a build-time mirror.

Status: proposed Date: 2026-07-09 Origin: the Readings entity showing 59.3M CHF on delta.jinflow.io’s overview Companion docs: Sense 14.2 The Signal (typed outputs) · Sense 43 The Cascade (config engine)

Claim

Every numeric aggregate JinDesk shows carries an implicit type and unit — but currently the type/unit assignment happens by convention, not declaration. When a pack sets amount: <column> on an entity YAML, JinDesk aggregates the column and formats the result as CHF regardless of what the column actually measures. That is a bug at scale: physical quantities (sediment concentration, watershed area, reach length) show as money.

Phase 5 makes entity aggregates declaratively typed and eliminates the defaulting layer entirely. A pack states the aggregate’s type and unit; the engine validates them together at compile time; JinDesk honours them at render time. If any part of the declaration is absent — no type, no unit, or the legacy shorthand — the compiler errors. Silent fallbacks are what let the CHF-59.3M bug ship in the first place; Phase 5 removes the surface it lived on.

The typed vocabulary is exactly what Sense 14.2 Phase 0 already grounded for Signal outputs — extending it to entity aggregates is a phase, not a new Sense. The removal of defaults is the deeper move.

The “no default” principle

“Any default is technical debt.” — Mig, during Phase 5 review, 2026-07-09.

Every default in an engine is a place where a wrong assumption can silently ship. The CHF-59.3M incident is the archetype: nobody declared Reading.suspended_sediment_mg_l as currency, but JinDesk rendered it that way because the surrounding contract had a currency fallback for the amount: field. Six years of default paths from the numetrix origin story quietly propagated into every pack that came after.

Phase 5 takes the stronger position: explicit-or-error, at every layer. Either the pack declares the aggregate’s type and unit completely, or compilation fails. JinDesk never renders an aggregate it hasn’t been told the type of. There is no “fall back to X” behaviour anywhere in the chain.

Trade-off: every existing pack breaks at compile time until migrated. That is the intended cost. The alternative — additive migration with grace periods — leaves the defaulting layer alive for months and defeats the purpose. Phase 5 ships as a hard cutover: the engine change and the pack migrations land together, so main is never green with old-shape entity YAMLs.

What Sense 14.2 already delivered

Recap so this doc stands alone.

Phase 0 established five base output types for Signals:

Type	Standard	Example units
quantity	UCUM	m, km², hL, mg/L, °C
currency	ISO 4217	CHF, EUR, USD
time	ISO 8601 duration	PT10M, PT1H, P1D
category	(enum)	vendor code, batch id
flag	boolean	true/false

Signal YAMLs opt in with an outputs: block declaring type and unit per output. signalcheck.py validates UCUM codes, ISO 4217 codes, and ISO 8601 durations — a typo doesn’t reach the build only to break aggregation downstream.

Phase 2 shipped window: → SQL and typed severity binding. Phase 3 shipped the currency machinery (cascade foundation, cross-currency safety net, coverage guarantee). Phase 4 lens declarativity retired 14 economic-lens guards.

Phase 5 extends the same vocabulary to entity aggregates.

The current gap

Entity YAMLs currently declare four column roles:

columns:
  primary_key: reading_key
  business_id: reading_id
  label: null
  detail: packet_status
  timestamp: reading_timestamp
  amount: suspended_sediment_mg_l   # <-- this line is Phase 5's concern

The amount: slot has two jobs: “which column to aggregate on the overview card” AND “format the result as currency.” The second job is an unstated convention.

Consequence: any physical measurement placed in amount: renders as CHF. The Readings entity’s overview card shows the sum of suspended_sediment_mg_l across 809K rows displayed as 59.3M CHF — meaningless as a sum, wrong as a currency. Watershed.area_km2 and Reach.length_m fall into the same trap.

The immediate fix (deploying separately, ahead of Phase 5) is to remove the CHF default. But that alone leaves entity aggregates untyped — a pack that does want to display a summed currency amount can’t say so, and one that wants area in km² still has no way to render that either.

The proposal

Entity YAMLs gain a typed aggregate: block that supersedes amount:. The block declares:

• column — which Gold column to aggregate (SUM by default; other aggregations follow later)
• type — one of Sense 14.2’s five base types
• unit — the specific unit, validated against the type’s grounded standard

# Before (Phase 4 shape)
columns:
  primary_key: reading_key
  business_id: reading_id
  amount: suspended_sediment_mg_l    # implicit CHF (wrong)

# After (Phase 5 shape)
columns:
  primary_key: reading_key
  business_id: reading_id
  # amount removed; aggregate: block preferred
aggregate:
  column: suspended_sediment_mg_l
  type: quantity
  unit: mg/L

For entities where the aggregate genuinely doesn’t have a natural unit (a count of rows, or “no aggregate at all”), leave aggregate: unset. That’s the Readings case — summing 810K sediment concentrations is nonsense; no aggregate is the right answer.

For entities where an aggregate makes sense:

# Watersheds — total area monitored
aggregate:
  column: area_km2
  type: quantity
  unit: km2                   # UCUM

# Reaches — total instrumented length
aggregate:
  column: length_m
  type: quantity
  unit: m                     # UCUM

# Numetrix billing — money at risk aggregate
aggregate:
  column: money_at_risk
  type: currency
  unit: CHF                   # ISO 4217

# Alptrack revenue events — pass revenue
aggregate:
  column: amount_chf
  type: currency
  unit: CHF                   # ISO 4217

# Deployments — average duration
aggregate:
  column: duration_days
  type: quantity
  unit: d                     # UCUM

Compile-time validation — type implies unit

The rule Mig named directly: “aggregate_type implies a subset of aggregate_unit. this can be checked during compile time.”

Concretely, entitycheck.py validates that aggregate.unit belongs to the set grounded by aggregate.type:

type	unit is validated against	example valid	example invalid
quantity	UCUM code registry	m, km2, mg/L, Cel	CHF, foo
currency	ISO 4217 code list	CHF, EUR, USD	mg/L, dollars
time	ISO 8601 duration OR ISO 80000 time unit	PT10M, s, min, h	week (spelled out), 10minutes
category	(unit not applicable)	null	any value
flag	(unit not applicable)	null	any value

Same validators Sense 14.2 Phase 0 already uses for Signal outputs (ucum.is_valid_unit, ucum.is_valid_kind, ucum.is_valid_iso8601_duration). Extension is one function call at the entity check level.

Failure mode when a pack picks type: quantity, unit: CHF — entitycheck.py errors:

FAIL  entities/readings.yaml
  - aggregate.unit 'CHF' is not a valid UCUM unit (aggregate.type is 'quantity').
    Did you mean type: currency?

JinDesk render behaviour

The overview card’s aggregate is a function of type + unit + magnitude:

type	render
currency	123.45 CHF (ISO 4217 with locale-aware thousands separator)
quantity	1.24 km² / 892 mg/L / 72.3 h (UCUM symbol rendered per locale)
time	PT10M renders as 10 min (or 10 Minuten, 10 minutes, 10 minuti)
category	(no aggregate — count of rows)
flag	(no aggregate — count of true / false)
(unset)	plain integer row count, no unit

Locale-aware rendering is the config-engine cascade’s job (Sense 43); Phase 5 just declares the type+unit and hands off to that layer.

Not “kill the CHF default” — kill the defaulting layer

An earlier draft of this doc proposed a small first step: change the CHF fallback to a plain-number fallback so packs would render sensible strings by accident. That would only replace one bad default with a less-bad one; the underlying pattern — the engine assuming what a pack meant — would live on.

The corrected shape:

Symptom: Every pack that sets columns.amount: <column> sees the aggregate rendered as CHF, regardless of the column’s actual meaning.

Root cause: the aggregate formatter treats amount: as an implicit-currency contract. Historical — the pattern grew out of numetrix (which really did mean money) and never got a fallback for other packs. The bug is not the choice of fallback; it is the presence of a fallback.

Fix: the formatter reads only what the pack has declared. If the pack declared aggregate: { column, type, unit }, JinDesk renders accordingly. If the pack declared nothing (aggregate: absent), JinDesk renders no aggregate — the overview card shows the row count only, no formatted number. If the pack declared the legacy amount: <column> shorthand, compilation fails with a clear message pointing at Phase 5’s shape.

Migration for the interim between Phase 4 and Phase 5: packs that trip on the wrong CHF render remove amount: entirely (set to null). That is the fix landed on riverflow at 2026-07-09 (readings, watersheds, reaches). It is a transitional workaround; Phase 5’s compiler errors then reject leaving amount: set at all.

Files to touch:

• scripts/entitycheck.py — reject entity YAMLs that use the legacy amount: field (except explicitly-null); reject partial aggregate: declarations (must have all three of column, type, unit or omit entirely).
• scripts/entitycompile.py — pass through the declared type and unit; no defaulting.
• explorer/src/lib/utils/format.ts (or wherever the entity-overview aggregate is formatted) — read declared type + unit from the entity registry; render accordingly. If nothing is declared, render nothing (not “plain number as fallback”).

Migration path — hard cutover

Not additive. Every pack breaks at compile time until migrated. The engine change and every pack’s migration land in the same PR (or a stack of PRs merged in a single window), so main is never green with old-shape entity YAMLs.

1. Extend entity YAML schema. aggregate: { column, type, unit } block accepted. Legacy amount: field allowed only when explicitly null; any non-null value fails compilation.
2. Update entitycompile.py to write aggregate.column, aggregate.type, aggregate.unit into the entity registry. No defaulting — if the fields are absent, no aggregate is written to the registry.
3. Update entitycheck.py: fail if amount: is set to anything other than null; fail on partial aggregate: (must have all three fields or omit); validate aggregate.unit against the type’s grounded standard (UCUM / ISO 4217 / ISO 8601).
4. Update JinDesk render to read declared type + unit from the entity registry. If the registry says nothing about the aggregate, JinDesk shows no aggregate — not “plain number as fallback.”
5. Migrate every pack in the same PR wave — numetrix, hrcentral, alptrack, millesime, lexflow, interlogic, riverflow. Each pack’s entity YAMLs either declare the full aggregate or omit it. No pack keeps the legacy shape.
6. Ratchet the check — after the wave lands, the compile error for legacy amount: becomes permanent contract. No grace period, no toggle.

The trade-off is real: every packbuilder in flight has to update their entity YAMLs before their next make. That is the point. Every default we remove is a bug that cannot ship again.

Sense-cluster relations

• Sense 14.2 Phase 0 — this Phase reuses the exact type vocabulary (quantity, currency, time, category, flag) and validators (UCUM, ISO 4217, ISO 8601) already grounded for Signal outputs. No new abstractions.
• Sense 43 The Cascade — locale-aware rendering (thousands separator, currency symbol placement, unit symbols per language) is the config engine’s job. Phase 5 declares the type+unit; the cascade formats.
• Sense 50 The Beacon — same theme, different surface: units and semantics should travel with the pack, not be inferred from field name conventions. Beacon proposes it for showcase tenants; Phase 5 proposes it for entity aggregates.
• Sense 46 The Roster — the entity registry is where Phase 5’s new fields land, alongside the tenant/pack metadata Roster already stamps there.

Open questions

1. Multiple aggregates per entity. An JinDesk overview card currently shows one aggregate. Some entities have several interesting aggregates (e.g. Reading could show total temperature × count or battery voltage average). Do we allow aggregates: (list) as well as aggregate: (single)? Recommendation: start with single. Add plural in Phase 5.1 if there’s demand.

2. Aggregate function beyond SUM. Sensible aggregates are also AVG, MIN, MAX, COUNT DISTINCT. Do we make the function part of the block (aggregate.function: avg)? Recommendation: yes, but ship SUM only in Phase 5 initial; add function support in Phase 5.1.

3. Cross-currency handling. If two tenants in the same pack are in different currencies (CHF vs EUR), the aggregate can’t mix them naively. Phase 3 already established the conversion machinery; Phase 5 just needs to reuse it.

4. UCUM subset. The full UCUM registry is large. Do we curate a “practical UCUM subset” for entity aggregates, or accept anything UCUM-valid? Recommendation: accept anything valid; add a linter-warn (not error) for units outside a curated set.

5. Category and flag aggregates. For category type, the “aggregate” would be a distinct-count or top-N distribution. For flag, a true-count. Both need clear UI treatment. Recommendation: defer both to Phase 5.2; ship quantity/currency/time first.

Ready to implement

One coordinated PR, three stages, ~5 hours total. Ships as a hard cutover.

Stage 1 — Engine: explicit-or-error contract (~2 hours)

Everything the compiler and JinDesk see. Lands first, but is held in a branch until the pack migrations catch up.

• scripts/entitycheck.py

  • Reject amount: set to anything other than null (with a message pointing at this doc’s shape).
  • Reject partial aggregate: — must have all three of column, type, unit, or omit entirely.
  • Validate aggregate.type ∈ {quantity, currency, time, category, flag}.
  • Validate aggregate.unit against the type’s grounded standard: ucum.is_valid_unit for quantity, ISO 4217 code list for currency, ISO 8601 duration + ISO 80000 time-unit set for time, must be null for category / flag.
  • Error text names the incident: “an earlier CHF-59.3M incident is the reason there is no default; declare or remove.”

• scripts/entitycompile.py

  • Read aggregate.column, aggregate.type, aggregate.unit from entity YAML.
  • Write three new columns into entity_registry.sql: aggregate_column, aggregate_type, aggregate_unit — all null when the block is absent.
  • Remove the legacy amount: compilation path entirely (no fallback write).

• explorer/src/lib/utils/format.ts (or wherever the entity-overview aggregate is formatted)

  • Read aggregate_type and aggregate_unit from the registry.
  • Render according to declared type + unit.
  • When both are null, render no aggregate — the overview card shows only the row count. No fallback formatting.

• docs/design/sense_14_2.md — add a Phase 5 section linking to this document.

Stage 2 — Pack migrations (~3 hours, all packs in the same wave)

Every pack’s entity YAMLs get updated. No pack keeps the legacy shape.

• packs/riverflow — Readings (omit aggregate), Watersheds (quantity/km2), Reaches (quantity/m), Deployments (quantity/d).
• packs/numetrix — Materials, BillingEvents, and every other entity currently using amount:. Mix of currency/CHF and quantity/unit-per-entity.
• packs/hrcentral — audit every entity YAML; declare or omit.
• packs/alptrack — Slopes (quantity/m), RevenueEvents (currency/CHF), and others.
• packs/millesime, packs/lexflow, packs/interlogic — same.

Stage 3 — Land as one wave

Engine PR + one pack PR each. Merge order is Engine last (once every pack PR is passing against the engine PR’s branch). This keeps main green throughout.

After merge, every fresh jinflow make on any pack either succeeds (fully declared) or fails at Phase 1a with a clear message. The defaulting layer is gone from every code path it lived on.

What this unlocks

After the hard cutover:

• No defaulting anywhere in the aggregate path. Every number JinDesk displays with a unit or symbol was declared to have that unit or symbol by the pack that owns it. The bug class that produced 59.3M CHF on a sediment concentration cannot ship again.
• Every pack speaks the same typed vocabulary from Signals to entity aggregates. The same UCUM / ISO 4217 / ISO 8601 validators that already run on Signal outputs run on entity aggregates. No parallel vocabularies to reconcile.
• The convention-layer in the config engine can retire. Patterns like precision.patterns.match: "price|prix|_amount|_cost|_chf|money_at_risk|billing_avg" — which exists in numetrix and hrcentral today — become unnecessary. The pack YAML declares type: currency, unit: CHF on the aggregate; the cascade honours it directly.
• Future packs onboard cleanly. A packbuilder writing their first entity YAML sees the schema explicitly rejects incomplete declarations. Nobody accidentally ships CHF on grape harvest weight because the surrounding contract had a hidden fallback.

The cost is one wave: engine + every pack in a coordinated PR set. The dividend is the entire class of silent-default bugs — of which the CHF-59.3M is one — cannot exist on this surface any more.

Acknowledgements

This Sense phase exists because Mig, walking through the delta tenant’s overview cards this morning, noticed “Readings with 59.3M CHF. how can this make sense?” — and framed the fix precisely: “aggregate_type implies a subset of aggregate_unit. this can be checked during compile time.”

Then, reviewing the first draft of this doc, he strengthened the position with: “we need to get rid of these defaults… in cases where there is no explicit declaration, there needs to be an error from the compilers.” The earlier draft proposed to swap one default for a less-bad one; that would have kept the defaulting layer alive. The corrected shape removes the defaulting layer entirely.

Naming the shape twice — first the type-vocabulary extension, then the removal of defaults — is what turns a bug report into a design phase.

The cost is one afternoon of coordinated engine + pack work. The dividend is every future pack shipping with its aggregate units declared or none, no accidental CHF, and one less place where the engine gets to guess what a pack meant.

---

Numerical neighbors: ← Sense 14.2 — Typed Signals · Sense 14.2: Typed Signals — Dimensions, Units, and Standards →