Skip to content

How it fits together

backscatter is a straight pipeline: pull a radar volume, decode it, draw it, save it, serve it. A background loop runs the pipeline on an interval; the API hands the saved frames to the browser.

flowchart LR
    S3[(NOAA S3\nLevel 2)] --> I[ingest]
    I --> D[decode]
    D --> R[render]
    R --> ST[store]
    ST --> A[api]
    A --> W[web map]
    C([collect loop]) -.drives.-> I

Each stage is its own package under src/backscatter/:

Stage Package What it does
ingest ingest/ Lists and downloads assembled Level 2 volumes from NOAA's public S3 bucket (anonymous, no credentials). Dedupes on (site, scan_time).
sites sites/ Bundled NEXRAD site table; resolves the nearest covering radar for a lat/lon.
decode decode/ Reads a volume with Py-ART and extracts the lowest-tilt reflectivity sweep.
render render/ Reprojects gate geometry to Web Mercator and applies the NWS dBZ color table → a georeferenced PNG + a small bounds sidecar.
store store/ SQLite index of every frame (+ the mutable locations table); raw volumes and PNGs on disk are the source of truth.
collect collect/ The long-running loop that walks ingest→render→index for each location, with failover and a throttled retention prune.
api api/ FastAPI app: serves rendered tiles + a small JSON timeline API.
web web/ A deliberately light MapLibre frontend (vanilla JS) — the map, timeline, and controls.

A few load-bearing ideas

  • The raw volumes on disk are the source of truth (the SQLite index just records what exists); pruning a frame deletes the volume, its render, and the row together.
  • Locations are mutable, persisted state. They live in SQLite, not code. The BACKSCATTER_LOCATIONS env var only seeds an empty store on first run — after that the database wins, and you manage locations in the UI.
  • Rendering correctness is the thing that will bite you. A wrong projection, flipped axis, or bad color mapping produces an image that looks plausible but is wrong — so anything touching geometry or color gets value-based tests and a visual check against a reference. See Contributing.
  • Everything configurable is read in one place (config.py). No module reads the environment directly.

Want the "why"?

The Design decisions (ADRs) record the real architectural choices — ingestion strategy, storage model, site selection, rendering geometry, retention, and more — with the alternatives that were weighed. The Roadmap shows how the app was built up one reviewable slice at a time.