image_ingester#

Image ingestion from Google Street View and Google Satellite for building footprints.

Called by Ingester._ingest_download_partition when recipe['image_scraper'] is set. Images are saved to the external data directory; a metadata parquet (entity_id → image_path + camera fields) is written alongside them.

Exceptions#

ImageScraperError

An image scraper could not be initialized (e.g. credentials/billing).

Functions#

fetch_images_by_admin(→ None)

Fetch Google images per building and write metadata parquets per admin unit.

load_image_metadata(→ pandas.DataFrame | None)

Load persisted image metadata for an admin unit, indexed by entity id.

Module Contents#

exception openplaces.io.ingester.image_ingester.ImageScraperError#

Bases: RuntimeError

An image scraper could not be initialized (e.g. credentials/billing).

Raised by _build_scraper so batch callers can skip a single image recipe (e.g. Street View when its Google Cloud project lacks billing) and continue with the rest of the run, rather than aborting.

openplaces.io.ingester.image_ingester.fetch_images_by_admin(ingester, n_sample: int | None = None, target_recipe_id: str | None = None, redownload: bool = False) → None#

Fetch Google images per building and write metadata parquets per admin unit.

Called by Ingester._ingest_download_partition when recipe['image_scraper'] is set. For each admin unit in ingester.admin_ids_to_process:

  1. Load buildings from target_recipe_id (falls back to recipe['entity_recipe']).

  2. Convert to AssetInventory.

  3. Run the appropriate scraper (google_streetview or google_satellite).

  4. Save images to the external directory under {admin_path}/{entity_path}/images/.

  5. Write a metadata parquet (entity_id, image_path, camera fields) to the standard external path.

Parameters:

  • ingester – openplaces.io.ingester.Ingester instance with resolved admin_ids_to_process and recipe.

  • n_sample – If set, cap the number of buildings processed per admin unit. Useful for test runs.

  • target_recipe_id – Recipe ID of the harmonized entity to photograph (e.g. 'US_building-nsi-2022' for NSI point buildings, 'US_footprint-spine-2026' for polygon footprints). When None, falls back to recipe['entity_recipe'].

  • redownload – Missing images are always fetched (first ingest downloads imagery). When True, images that already exist on disk are re-downloaded and overwritten rather than reused.

openplaces.io.ingester.image_ingester.load_image_metadata(image_recipe_id: str, admin_id) → pandas.DataFrame | None#

Load persisted image metadata for an admin unit, indexed by entity id.

Reads the metadata parquet(s) written by fetch_images_by_admin for the image recipe image_recipe_id. When the image recipe saves at a finer admin level than admin_id (e.g. images saved per town but admin_id is a county), the per-child frames are concatenated. Tolerant of missing files: children without imagery are skipped and None is returned when no metadata exists at all (e.g. a scraper whose ingest was skipped).

Parameters:

  • image_recipe_id (str) – Recipe ID of the image entity (e.g. 'image-googlesatellite-z20').

  • admin_id (str or AdminId) – Admin unit to load metadata for.

Returns:

Metadata indexed by entity (footprint) id with an image_path column (plus image_found and camera fields), or None if no metadata files are found.

Return type:

pandas.DataFrame or None