evidence#
Curation steps for incorporating enrichment evidence.
Functions#
|
Join another curated entity's attributes onto the current one by a shared id. |
Apportion a curated reference entity's values over the n:m link sidecar. |
|
|
Materialize all linked reference ids per entity from a link sidecar. |
|
Merge enrichment evidence into canonical columns. |
Module Contents#
- openplaces.io.curator.evidence.link_curated_entity(state: openplaces.io.curator.CurateState, recipe_id: str, columns: dict, entity_key: str = 'parcel_id', ref_key: str = 'parcel_id') openplaces.io.curator.CurateState#
Join another curated entity’s attributes onto the current one by a shared id.
Reads the output of a curate-stage recipe and maps each named column onto the current entity (1:many ref -> entity) on the id key. This is how, e.g., the footprint lane consumes the parcel curation lane’s land-use decisions (the refined
use_group_combinedand themanufactured_home_parkflag), so the referenced entity’s classification is finalized before the current one’s is inferred. Existing columns are overwritten — the referenced curated value supersedes the raw harmonized one.- Parameters:
recipe_id (str) – A stage
curaterecipe whose output supplies the attributes.columns (dict) – Mapping of
{ref_column: entity_column}. The entity column is the name written ontostate.curated(suffix it to mirror a harmonized evidence column, e.g._parcel).entity_key (str, optional) – Id column on the current entity (default
parcel_id). Unlike other cross-attributed columns, an id column keeps its bare name rather than the usual_parcel-style suffix (see_attributed_namein the harmonizer) — the entity’s ownparcel_idalready names the referenced row, so a suffix would be redundant. This is the referenced entity’s globally-uniqueparcel_id(geo_id), notparcel_id_local: the latter is only locally cross-comparable and can collide within a single admin unit, which would silently misattribute the joined columns to an unrelated parcel.ref_key (str, optional) – Id column on the referenced entity’s own curated output (default
parcel_id— that entity’s own key, not a cross-attributed one).
- openplaces.io.curator.evidence.apportion_curated_values(state: openplaces.io.curator.CurateState, recipe_id: str, columns: dict, entity_type: str = 'parcel', link_recipe_id: str | None = None, ref_key: str = 'parcel_id', entity_key: str = 'parcel_id', priority_column: str = 'priority_on_parcel', dwelling_column: str = 'n_dwellings_overture') openplaces.io.curator.CurateState#
Apportion a curated reference entity’s values over the n:m link sidecar.
The value-column counterpart of
link_curated_entity(): instead of mapping each entity’s dominant reference’s undivided value, this re-uses the n:m overlay persisted by the harmonize stage (link_to_referencewithsave_link: true) and the shared apportionment implementation (apportion_reference_values()) to distribute the curated reference’s values exactly the way the harmonize stage distributes raw reference values: overlap-area shares forimprovement_value/n_dwellings, dwelling-linked suppression,land_value/addresswhole on the dominant reference of principal entities only,year_builtas the linked references’ mean.Synthetic reference-derived fallback geometries (
geometry_sourcestarting with'{entity_type}.', added byinfer_spine_additionsafter the overlay was persisted, so never present in the sidecar) are appended as full-weight single links via theirentity_keycolumn.- Parameters:
recipe_id (str) – A stage
curaterecipe whose output supplies the values.columns (dict) – Mapping of
{ref_column: entity_column}. Every ref column must be one the shared apportionment knows (APPORTIONED_VALUE_COLUMNS); uselink_curated_entity()for plain dominant-reference attributes.entity_type (str, optional) – Reference entity type; resolves the link sidecar the same way the harmonize overlay did (default
parcel). Ignored when link_recipe_id is given.link_recipe_id (str, optional) – Explicit reference recipe id of the link’s other side.
ref_key (str, optional) – Id column on the referenced curated output (default
parcel_id).entity_key (str, optional) – Id column on the current entity holding its dominant reference id (default
parcel_id); used only to link synthetic fallback rows, which are absent from the sidecar.priority_column (str, optional) – Column holding each entity’s role on its reference (default
priority_on_parcel); drives the secondary/principal rules.dwelling_column (str, optional) – Column whose positive value marks an entity as dwelling-linked (default
n_dwellings_overture); drives the suppression rule.
- Raises:
FileNotFoundError – When the sidecar is missing: curation cannot recompute the overlay, so re-run harmonize with
save_link: truefirst.
- openplaces.io.curator.evidence.collect_link_ids(state: openplaces.io.curator.CurateState, entity_type: str | None = None, link_recipe_id: str | None = None, column: str = 'parcel_id_all', include_below_threshold: bool = False) openplaces.io.curator.CurateState#
Materialize all linked reference ids per entity from a link sidecar.
Reads the n:m link sidecar persisted by the harmonize overlay (
link_to_referencewithsave_link: true) and writes a pipe-joined id column onto the curated entity, ordered by descending intersection area — the dominant reference (e.g. the entity’sparcel_id) comes first. Harmonized spines stay link-only; this is where multi-reference membership becomes a canonical column.- Parameters:
entity_type (str, optional) – Auto-discover the reference recipe of this entity type for the current admin unit, the same way the harmonize overlay resolved it (so the sidecar filename matches per state/county). Ignored when
link_recipe_idis given.link_recipe_id (str, optional) – Explicit reference recipe ID of the link’s other side.
column (str, optional) – Output column written onto the curated entity (default
parcel_id_all). Left missing where the collected value is identical to the base id column (e.g.parcel_id).include_below_threshold (bool, optional) – When False (default), only link-labeled pairs (those kept by the harmonize crosswalk’s sliver thresholds) are collected. When True, every raw overlap in the sidecar counts.
- Raises:
FileNotFoundError – When the sidecar is missing: curation cannot recompute the overlay, so re-run harmonize with
save_link: truefirst.
- openplaces.io.curator.evidence.merge_enrichments(state: openplaces.io.curator.CurateState, recipes: list[dict]) openplaces.io.curator.CurateState#
Merge enrichment evidence into canonical columns.
Each recipe specification must contain
recipe_idand acolumnsmapping from evidence-column names to canonical-column names. Existing canonical values take precedence; enrichment fills missing values.A
recipe_specwhose enrichment output doesn’t exist yet for this admin is skipped rather than raising – e.g. an imagery-dependent enrichment intentionally left unrun this pass (seenotebooks/examples/US_curate_footprints.ipynb’s--no_streetview/--no_googlesatellite, which skip billing-costly image ingestion and the enrichment steps that depend on it). Arecipe_specwhose evidence does exist but lacks a requested column still raises – that is a real recipe misconfiguration, not evidence that simply hasn’t been computed.