Documents · Developer documentation
Reproducible visual documentation
An analytical example asks a question, displays executable code, shows its generated result, and explains what changed. A scientific result is not a decorative asset. Use maps for space, series for time, distributions for scale, before/after views for transformations, and a table for export verification.
Ownership and first-pass scope
scripts/visual_examples.py owns the exact code, prerequisites, captions and
interpretations. scripts/build_visual_docs.py executes that code on the
reviewed fixtures, saves results to docs/assets/generated/visual/, and updates
only delimited example regions in Learn and the source comparison page.
The existing reference generator inserts the same examples into noun/verb pages.
The six first-pass pages are Learn: verbs,
the grammar vignette,
temperature,
dataset comparison,
anomaly, and
threshold_state.
This remains the first-pass climate set. The main
elevation, roads, and
streamflow entries also have executable
real-data lessons and generated first-result figures. Their code lives in
scripts/source_lesson_content.py and scripts/build_streamflow_vignette.py;
scripts/build_source_vignettes.py regenerates/checks seven figures under
docs/assets/generated/nouns/. The same code executes in the notebooks.
The vignette builder copies the same code into native notebook cells. It uses
the existing run_vignettes.py / nbclient execution and MkDocs-Jupyter rendering,
not another notebook framework. Marked cells must emit their own PNG or HTML
table, and the runner checks decoded image content. Source notebooks remain
unexecuted in Git; executed copies may be retained as evidence.
Data and scientific scope
All first-pass results use REAL DATA: checksum-controlled PRISM Boulder January 2024 and gridMET Badlands July 2001 extracts. They run offline after installation. The loader rejects a checksum mismatch, synthetic marker, nonfinite values or unexpected orientation. Existing source QA owns the scientific fixture validation; the website does not certify a second copy.
The source panels deliberately retain native units and different bounds/dates. They are a support check and cannot estimate source bias; they must not be described as co-located or contemporaneous. A real paired comparison is a later, separately reviewed data acquisition. Daymet remains outside implemented sources.
Each result's provenance manifest records the exact executable code, fixture/provenance hashes, producing script, caption, interpretation, output hash, runtime-source hashes and generation environment/Git context. Input provenance retains provider URLs and source QA scope. No scientific figures are manually edited or pasted from screenshots.
First-pass result inventory
Every row is generated by scripts/build_visual_docs.py from the named entry
in scripts/visual_examples.py. Paths below are relative to
docs/assets/generated/visual/. PRISM denotes
tests/fixtures/real_data/prism_boulder_january_2024.nc; gridMET denotes
tests/fixtures/real_data/gridmet_badlands_july_2001.nc.
| Example key | Real input and transformation | Result file |
|---|---|---|
observed |
PRISM: select 16 January, north-up temperature map | observed.png |
subset |
PRISM: highlight 10–20 January against the full series | subset.png |
anomaly |
PRISM: subtract each pixel's selected-period mean | anomaly.png |
summary |
PRISM anomalies: unweighted spatial mean over y/x | summary.png |
standardize |
PRISM: direct-versus-piped z-score, before/after distributions | standardize.png |
threshold |
PRISM: maximum temperature ≤ 0°C becomes a state | threshold.png |
sources |
PRISM and gridMET: native-grid, native-unit support inspection | sources.png |
export |
PRISM summary: asserted NetCDF round-trip identity | export.json (rendered table) |
These are seven unique figures and one table, reused across the six pages. The Learn/vignette sequence has five figures and the table. No first-pass scientific result uses synthetic teaching data; negative-control images exist only inside tests.
Regenerate and verify
From the repository root:
python scripts/build_visual_docs.py
python scripts/build_vignette_notebooks.py
python scripts/build_reference_docs.py
python scripts/build_visual_docs.py --check
python scripts/build_reference_docs.py --check
python scripts/run_vignettes.py --output-dir artifacts/visual_docs/notebooks
python scripts/run_source_qa.py
mkdocs build --strict
python scripts/check_site_links.py site
pytest tests/browser -m browser --site-dir site --browser chromium -q
The generation step fails on execution errors. Freshness checks fail on changed code/inputs, missing, corrupt or empty output, changed image bytes, stale example Markdown or a stale supported notebook. Runtime Python changes invalidate the first-pass figure cache deliberately. Timestamps are not used as freshness proof.
Both documentation CI and Pages first check the committed results, then execute and regenerate them before building. Pixel hashes validate each generated asset, not cross-platform Matplotlib/font equivalence; numerical tests check the actual operations. Browser tests verify code/result/caption ordering, decoded images, native notebook results, and 390px presentation, not scientific correctness.
Safe updates
Edit the shared example definition, not a generated image or reference page. Retain the scientific question and record any changed baseline or unit convention. Run the commands above and review the figures at desktop and mobile sizes. If fixture bytes need changing, follow the reviewed-fixture acquisition policy before updating checksums. Keep negative controls in tests, not publication plots.
Next candidates are the remaining Learn lessons, states/events vignette and mean/zscore references. Expand only after reviewing this first pass. Existing legacy figures outside these six pages are not certified by this new manifest.