checkpoint

Incremental prediction checkpoints for interruptible detector runs.

Classes

PredictionCheckpoint

Persist detector predictions incrementally so interrupted runs resume.

Functions

local_checkpoint_path(→ pathlib.Path)	Return a stable, unsynced cache path for an output-side checkpoint.
prune_local_checkpoints(→ list[pathlib.Path])	Remove old local checkpoint files so the cache cannot grow forever.

Module Contents

openplaces.io.enricher.detectors.checkpoint.local_checkpoint_path(source_path: pathlib.Path | str) → pathlib.Path

Return a stable, unsynced cache path for an output-side checkpoint.

openplaces.io.enricher.detectors.checkpoint.prune_local_checkpoints(root: pathlib.Path | str | None = None, max_age_days: int = 14, max_total_mb: int = 1024, keep_latest: int = 8) → list[pathlib.Path]

Remove old local checkpoint files so the cache cannot grow forever.

class openplaces.io.enricher.detectors.checkpoint.PredictionCheckpoint(path: pathlib.Path | str, save_every: int = 500, legacy_paths=())

Persist detector predictions incrementally so interrupted runs resume.

A transient single-column parquet file stored in the local application cache. Keeping checkpoints outside synced data trees avoids file-lock conflicts with Dropbox and similar clients. Detectors load it on start to skip already-predicted keys, add each new prediction, and flush periodically; the Enricher deletes it once the evidence file is saved.

Parameters:

• path (Path or str) – Checkpoint parquet path.

• save_every (int) – Flush to disk after this many added predictions.

• legacy_paths (iterable of Path or str, optional) – Former checkpoint locations to migrate on first load.

load() → dict[Any, Any]

Return previously checkpointed predictions ({} when none).

None results (e.g. missing image files) count as done and are not recomputed on resume.

add(key: Any, value: Any) → None

Record one prediction; flush every save_every additions.

flush() → None

Write all recorded predictions to disk atomically.

Values are JSON-encoded per cell so mixed prediction types (class labels, counts, None) round-trip faithfully.

delete() → None

Remove the checkpoint file (call after evidence is saved).

prune_cache() → list[pathlib.Path]

Prune old local checkpoints once per checkpoint instance.