evidence#

Curation steps for incorporating enrichment evidence.

Functions#

link_curated_entity(→ openplaces.io.curator.CurateState)

Join another curated entity's attributes onto the current one by a shared id.

apportion_curated_values(...)

Apportion a curated reference entity's values over the n:m link sidecar.

collect_link_ids(→ openplaces.io.curator.CurateState)

Materialize all linked reference ids per entity from a link sidecar.

merge_enrichments(→ openplaces.io.curator.CurateState)

Merge enrichment evidence into canonical columns.

Module Contents#

Join another curated entity’s attributes onto the current one by a shared id.

Reads the output of a curate-stage recipe and maps each named column onto the current entity (1:many ref -> entity) on the id key. This is how, e.g., the footprint lane consumes the parcel curation lane’s land-use decisions (the refined use_group_combined and the manufactured_home_park flag), so the referenced entity’s classification is finalized before the current one’s is inferred. Existing columns are overwritten — the referenced curated value supersedes the raw harmonized one.

Parameters:
  • recipe_id (str) – A stage curate recipe whose output supplies the attributes.

  • columns (dict) – Mapping of {ref_column: entity_column}. The entity column is the name written onto state.curated (suffix it to mirror a harmonized evidence column, e.g. _parcel).

  • entity_key (str, optional) – Id column on the current entity (default parcel_id). Unlike other cross-attributed columns, an id column keeps its bare name rather than the usual _parcel-style suffix (see _attributed_name in the harmonizer) — the entity’s own parcel_id already names the referenced row, so a suffix would be redundant. This is the referenced entity’s globally-unique parcel_id (geo_id), not parcel_id_local: the latter is only locally cross-comparable and can collide within a single admin unit, which would silently misattribute the joined columns to an unrelated parcel.

  • ref_key (str, optional) – Id column on the referenced entity’s own curated output (default parcel_id — that entity’s own key, not a cross-attributed one).

openplaces.io.curator.evidence.apportion_curated_values(state: openplaces.io.curator.CurateState, recipe_id: str, columns: dict, entity_type: str = 'parcel', link_recipe_id: str | None = None, ref_key: str = 'parcel_id', entity_key: str = 'parcel_id', priority_column: str = 'priority_on_parcel', dwelling_column: str = 'n_dwellings_overture') openplaces.io.curator.CurateState#

Apportion a curated reference entity’s values over the n:m link sidecar.

The value-column counterpart of link_curated_entity(): instead of mapping each entity’s dominant reference’s undivided value, this re-uses the n:m overlay persisted by the harmonize stage (link_to_reference with save_link: true) and the shared apportionment implementation (apportion_reference_values()) to distribute the curated reference’s values exactly the way the harmonize stage distributes raw reference values: overlap-area shares for improvement_value/n_dwellings, dwelling-linked suppression, land_value/address whole on the dominant reference of principal entities only, year_built as the linked references’ mean.

Synthetic reference-derived fallback geometries (geometry_source starting with '{entity_type}.', added by infer_spine_additions after the overlay was persisted, so never present in the sidecar) are appended as full-weight single links via their entity_key column.

Parameters:
  • recipe_id (str) – A stage curate recipe whose output supplies the values.

  • columns (dict) – Mapping of {ref_column: entity_column}. Every ref column must be one the shared apportionment knows (APPORTIONED_VALUE_COLUMNS); use link_curated_entity() for plain dominant-reference attributes.

  • entity_type (str, optional) – Reference entity type; resolves the link sidecar the same way the harmonize overlay did (default parcel). Ignored when link_recipe_id is given.

  • link_recipe_id (str, optional) – Explicit reference recipe id of the link’s other side.

  • ref_key (str, optional) – Id column on the referenced curated output (default parcel_id).

  • entity_key (str, optional) – Id column on the current entity holding its dominant reference id (default parcel_id); used only to link synthetic fallback rows, which are absent from the sidecar.

  • priority_column (str, optional) – Column holding each entity’s role on its reference (default priority_on_parcel); drives the secondary/principal rules.

  • dwelling_column (str, optional) – Column whose positive value marks an entity as dwelling-linked (default n_dwellings_overture); drives the suppression rule.

Raises:

FileNotFoundError – When the sidecar is missing: curation cannot recompute the overlay, so re-run harmonize with save_link: true first.

Materialize all linked reference ids per entity from a link sidecar.

Reads the n:m link sidecar persisted by the harmonize overlay (link_to_reference with save_link: true) and writes a pipe-joined id column onto the curated entity, ordered by descending intersection area — the dominant reference (e.g. the entity’s parcel_id) comes first. Harmonized spines stay link-only; this is where multi-reference membership becomes a canonical column.

Parameters:
  • entity_type (str, optional) – Auto-discover the reference recipe of this entity type for the current admin unit, the same way the harmonize overlay resolved it (so the sidecar filename matches per state/county). Ignored when link_recipe_id is given.

  • link_recipe_id (str, optional) – Explicit reference recipe ID of the link’s other side.

  • column (str, optional) – Output column written onto the curated entity (default parcel_id_all). Left missing where the collected value is identical to the base id column (e.g. parcel_id).

  • include_below_threshold (bool, optional) – When False (default), only link-labeled pairs (those kept by the harmonize crosswalk’s sliver thresholds) are collected. When True, every raw overlap in the sidecar counts.

Raises:

FileNotFoundError – When the sidecar is missing: curation cannot recompute the overlay, so re-run harmonize with save_link: true first.

openplaces.io.curator.evidence.merge_enrichments(state: openplaces.io.curator.CurateState, recipes: list[dict]) openplaces.io.curator.CurateState#

Merge enrichment evidence into canonical columns.

Each recipe specification must contain recipe_id and a columns mapping from evidence-column names to canonical-column names. Existing canonical values take precedence; enrichment fills missing values.

A recipe_spec whose enrichment output doesn’t exist yet for this admin is skipped rather than raising – e.g. an imagery-dependent enrichment intentionally left unrun this pass (see notebooks/examples/US_curate_footprints.ipynb’s --no_streetview/ --no_googlesatellite, which skip billing-costly image ingestion and the enrichment steps that depend on it). A recipe_spec whose evidence does exist but lacks a requested column still raises – that is a real recipe misconfiguration, not evidence that simply hasn’t been computed.