links

Pipeline steps that create and use relationships between the spine and reference datasets: - link_to_reference: load a reference and build a spine ↔ reference crosswalk - infer_spine_additions: add new spine entries inferred from a reference - resolve_overlaps: remove remaining geometry overlaps from the spine

Functions

link_to_reference(...)	Load a reference dataset and build a spine ↔ reference crosswalk.
snap_chained_links(→ tuple[pandas.DataFrame, ...)	Snap chain-displaced multi-parcel links to their dominant parcel.
flag_duplicate_points(→ pandas.Series)	Flag colocated duplicate points from low-rank sources.
link_by_id(→ openplaces.io.harmonizer.HarmonizeState)	Link a reference entity to the spine by a precomputed id key (non-spatial).
infer_spine_additions(...)	Add spine entries inferred from a reference crosswalk.
resolve_overlaps(→ openplaces.io.harmonizer.HarmonizeState)	Resolve remaining geometry overlaps in the spine.

Module Contents

openplaces.io.harmonizer.links.link_to_reference(state: openplaces.io.harmonizer.HarmonizeState, join: str = 'spatial_overlay', entity_type: str | None = None, recipe_id: str | None = None, thresholds: dict | None = None, remap_id: str | None = None, source_geometry_type: str | None = None, aggregation_function=None, sort_by: str | None = None, list_columns: list[str] | None = None, save_link: bool = False) → openplaces.io.harmonizer.HarmonizeState

Load a reference dataset and build a spine ↔ reference crosswalk.

Populates state.references[recipe_id], state.crosswalks[recipe_id], state.overlays[recipe_id] (for spatial_overlay joins), state.reference_types[recipe_id], and state.source_geometry_types[recipe_id] (when source_geometry_type is provided).

Parameters:

• join (str) –

  How to join the reference to the spine:

  'spatial_overlay'

  Polygon-on-polygon identity overlay. Produces a crosswalk table with IoU and area-intersection columns. Populates state.overlays[recipe_id] with the full geometry-bearing overlay result for use by later steps.

  'spatial_point'

  Point-in-polygon sjoin. Joins reference points to the spine entities, and unlinked points to any polygon reference already in state.references matching the reference’s entity type.

• entity_type (str, optional) – Auto-discover the best ingest recipe of this entity type for the current admin_id. Ignored when recipe_id is given.

• recipe_id (str, optional) – Explicit reference recipe ID. Takes precedence over entity_type.

• thresholds (dict, optional) – For spatial_overlay: min_fraction_of_largest (float, default 1/6) — minimum fraction of the largest spine-reference intersection to keep a secondary link. area_intersection_m2_min (float, default 10) — minimum intersection area in m² to keep a link. For spatial_point: proximity_m (float, default 10) — radius for inner proximity pass. far_proximity_m (float, default 100) — radius for outer proximity pass (same-parcel constraint applied). Set to 0 to disable.

• remap_id (str, optional) – Recipe ID for a column-remap table applied to the reference after loading (see openplaces.io.transform.remap()).

• source_geometry_type (str, optional) – SourceGeometryType value describing what this source represents spatially (e.g. 'single_building_point'). Stored in state.source_geometry_types for use by downstream steps such as classify_footprint_priority.

• aggregation_function (None, callable, or dict, optional) – Controls how duplicate geo_id rows in the reference are reduced to one row before joining. None (default) applies the aggregation function from the attribute registry. A dict maps column names to callables; columns absent from the dict fall back to the registry default. Only used for spatial_overlay joins.

• sort_by (str, optional) – Column to sort reference rows by descending before aggregation. Falls back to geometry area when the column is absent and the reference is a GeoDataFrame. Only used for spatial_overlay joins.

• list_columns (list of str, optional) – Column names for which an extra {col}_list column is added to the aggregated reference, collecting all values per geo_id into a list. Normal scalar aggregation for each column still applies alongside. Only used for spatial_overlay joins.

• save_link (bool, optional) – Persist the full many-to-many identity overlay (geometry-free, every spine-reference pair including sub-threshold slivers, with the crosswalk’s link label joined on) as a sidecar parquet at the canonical entity-link path. On later runs the sidecar is reloaded instead of recomputing the overlay — the single most expensive harmonize step — iff its footer fingerprint (step config plus size/mtime of the ingest inputs) still matches; a deleted input with a tombstone receipt stays verifiable. After a reload, state.overlays[recipe_id] carries no geometry column (only the area/IoU columns are consumed downstream). Only used for spatial_overlay joins.

openplaces.io.harmonizer.links.snap_chained_links(crosswalk: pandas.DataFrame, spine_id_col: str, fraction_max: float = 0.75) → tuple[pandas.DataFrame, pandas.DataFrame]

Snap chain-displaced multi-parcel links to their dominant parcel.

A footprint layer displaced relative to the parcel layer makes each footprint straddle its own parcel and the next one over, chaining footprint-parcel-footprint-parcel down the block and inflating n_parcels_per_footprint for every home on it. Such a footprint is snapped to its dominant (largest-intersection) parcel when

• every minor link’s parcel is a different footprint’s dominant or unique parcel — the neighbor demonstrably has its own building. A genuine shared row-house footprint never satisfies this: the neighboring parcels’ only building is the shared footprint itself, so real multi-parcel buildings keep their multi links; and

• every minor link’s fraction_of_largest is at most fraction_max — a near-equal split leaves the dominant side genuinely ambiguous, so it is left alone.

Ownership is computed from the pre-snap crosswalk (dominants never move), so one pass resolves whole chains deterministically regardless of row order. Uses no geometry, so it works identically on a reloaded geometry-free link sidecar.

Returns:

The adjusted crosswalk — each snapped footprint collapses to a single link relabeled 'unique parcel (snapped from chain)' — and the removed minor rows (empty when nothing was snapped).

Return type:

tuple of (pandas.DataFrame, pandas.DataFrame)

openplaces.io.harmonizer.links.flag_duplicate_points(ref: pandas.DataFrame, key_col: str, ignore_sources: list[str]) → pandas.Series

Flag colocated duplicate points from low-rank sources.

Within groups of two or more points sharing key_col (e.g. NSI’s building_id_ubid, or the _olc location cell), rows whose source is in ignore_sources are labeled 'colocated low-rank source' when the group also contains at least one source outside that set — a higher-level record to defer to. A group made up entirely of ignorable sources stays unflagged (nothing better exists), as does any point at a unique location. Returns an object Series aligned to ref (null = kept); rows are never dropped here — the exclusion is applied where the evidence is merged onto a spine (reconcile_attributes()).

openplaces.io.harmonizer.links.link_by_id(state: openplaces.io.harmonizer.HarmonizeState, recipe_id: str | None = None, auto_discover: bool = False, entity_type: str = 'parcel', mode: str = 'attributes', spine_key: str = 'parcel_id_local', ref_key: str = 'parcel_id_local', columns: list[str] | None = None, suffix: str | None = None, count_as: str = 'n_transactions', flag_as: str = 'is_transacted', layer: str | None = None) → openplaces.io.harmonizer.HarmonizeState

Link a reference entity to the spine by a precomputed id key (non-spatial).

Joins on the standardized matching key (parcel_id_local) that data ingestion already computed on both sides, so no re-conversion happens here.

Column priority ('attributes' and 'aggregate' modes): when a column is already on the spine (from an earlier call, e.g. an earlier auto_discover match), the new source only overwrites it outright if the new source covers a majority of spine rows for that column; otherwise it only fills the existing column’s gaps (see _write_prioritized()). Combined with auto_discover’s oldest-to-newest join order, this makes the most recent source the default winner for each column, without letting a sparse recent source blank out a more complete older one.

Parameters:

• recipe_id (str, optional) – Reference entity recipe (e.g. an assessment roll or a transaction table). Required unless auto_discover is set.

• auto_discover (bool) – When set, ignore recipe_id and instead discover every ingest recipe of entity_type (plus any bundled additional_layers) whose admin scope covers the admin unit being processed (_discover_link_sources()), and recurse into this function once per match. Each match always joins via mode='aggregate' (a correct generalization of 'attributes' for 1:1 data too) on its resolved key (parcel_id_local for a standalone roll; a layer’s own layer_key for a bundled additional_layers entry), with columns defaulting to the attribute registry’s canonical columns for that entity type ( get_attributes()) — pass an explicit columns to override this default for every discovered match. Any *-remap.csv crosswalk found beside a matched source is applied automatically (see _apply_remap_csvs()). A standalone match that is also one of the spine’s own geometry sources has its resolve_spine keep_columns dropped from the join (already correct on the spine; re-deriving them via a non-unique local key would pool values across every row sharing it).

• entity_type (str) – Entity type to discover when auto_discover is set (default 'parcel').

• mode ({'attributes', 'count', 'aggregate'}) – 'attributes' joins columns from the reference onto the spine (1:1 on the key, keeping the first reference row per key). 'count' aggregates a 1:many reference into a per-spine count (count_as) and a boolean presence flag (flag_as) — used to track which parcels have been transacted. 'aggregate' reduces a 1:many reference onto the spine by grouping on the key and applying each column’s attribute- registry aggregation (e.g. sum land_value/n_dwellings, max year_built), falling back to the first non-null value for columns without a registry rule; it also emits a per-key record count. Use it when several reference rows share one spine key, such as MassGIS L3_ASSESS condominium records stacked on one parcel polygon.

• spine_key (str) – Key columns on the spine and reference (default parcel_id_local). Unlike an entity’s own index (unique by design), this key is not guaranteed unique; see _warn_if_duplicate_key(), which warns when either side turns out to have duplicates, so that risk is visible rather than silently assumed away.

• ref_key (str) – Key columns on the spine and reference (default parcel_id_local). Unlike an entity’s own index (unique by design), this key is not guaranteed unique; see _warn_if_duplicate_key(), which warns when either side turns out to have duplicates, so that risk is visible rather than silently assumed away.

• columns (list of str, optional) – Reference columns to attach in 'attributes' mode.

• suffix (str, optional) – Suffix appended to attached column names ('attributes' mode).

• count_as (str) – Output column names in 'count' mode.

• flag_as (str) – Output column names in 'count' mode.

• layer (str, optional) – Secondary layer (entity type or full entity string) of an additional_layers entity to load from recipe_id, e.g. the property assessor table bundled inside a MassGIS parcel recipe.

openplaces.io.harmonizer.links.infer_spine_additions(state: openplaces.io.harmonizer.HarmonizeState, entity_type: str | None = None, recipe_id: str | None = None, thresholds: dict | None = None) → openplaces.io.harmonizer.HarmonizeState

Add spine entries inferred from a reference crosswalk.

For each reference polygon that has no existing spine coverage and exceeds the improvement-value threshold, creates a new spine geometry equal to the reference polygon geometry.

The inferred GeoDataFrame is stored in state.metadata['inferred_from_<recipe_id>'] for use by reconcile_attributes.

Parameters:

• entity_type (str, optional) – Selects all crosswalks matching this type via state.reference_types.

• recipe_id (str, optional) – Explicit crosswalk key to use. Takes precedence over entity_type.

• thresholds (dict, optional) – n_per_group_min (float, default 0.2) — minimum mean spine count per purpose group to be eligible for inference. value_per_ha_quantile (float, default 0.05) — quantile of improvement_value_per_ha used as the lower inference bound.

openplaces.io.harmonizer.links.resolve_overlaps(state: openplaces.io.harmonizer.HarmonizeState, **_params) → openplaces.io.harmonizer.HarmonizeState

Resolve remaining geometry overlaps in the spine.

Calls resolve_overlapping_polygons() on state.spine (with keep=False).