02 · From observations to a comparable signal¶
Context¶
Climate observations are often exchanged as one row per date and location. Here we flatten official PRISM minimum temperature into that familiar table, then recover the cube without changing a value.
Question¶
Can we compare departures through time at sites with different baseline temperatures?
Analysis story¶
We will prove the table-to-cube round trip, standardize each location through time, and compare the observed and standardized series at one grid cell.
Data used in this lesson¶
Every value comes from the PRISM Group at Oregon State University's AN91d daily 4 km climate product. This repository carries a small Boulder-region extract for 1–30 January 2024 so the lesson runs offline without replacing observations with generated values. The data validation page records source URLs, terms, checksums, bounds, units, and acceptance tests.
Prepare · Round-trip real observations through a tidy table¶
In [1]:
Copied!
from pathlib import Path
import xarray as xr
# Find the repository from either a root-level documentation build or a kernel
# started beside this notebook, then open the checksum-controlled PRISM extract.
data_path = next(
candidate / "tests" / "fixtures" / "real_data" / "prism_boulder_january_2024.nc"
for candidate in (Path.cwd(), *Path.cwd().parents)
if (candidate / "tests" / "fixtures" / "real_data" / "prism_boulder_january_2024.nc").exists()
)
prism = xr.open_dataset(data_path, engine="scipy").load()
# These assertions are part of the teaching contract: official source,
# canonical cube dimensions, complete daily time, and declared Celsius units.
assert prism.attrs["source"] == "PRISM Group, Oregon State University"
assert prism.attrs["is_synthetic"] == 0
assert prism.sizes == {"time": 30, "y": 24, "x": 24}
assert prism["tmax"].attrs["units"] == "degC"
import numpy as np
source = prism["tmin"].isel(y=slice(9, 14), x=slice(9, 15))
table = source.to_dataframe(name="tmin").reset_index()
cube = (
table.set_index(["time", "y", "x"])
.to_xarray()["tmin"]
.transpose("time", "y", "x")
.sel(time=source.time, y=source.y, x=source.x)
)
cube.attrs.update(source.attrs)
cube.attrs.update(source=prism.attrs["source"], is_synthetic=0)
np.testing.assert_array_equal(cube.values, source.values)
assert len(table) == source.size
cube
from pathlib import Path
import xarray as xr
# Find the repository from either a root-level documentation build or a kernel
# started beside this notebook, then open the checksum-controlled PRISM extract.
data_path = next(
candidate / "tests" / "fixtures" / "real_data" / "prism_boulder_january_2024.nc"
for candidate in (Path.cwd(), *Path.cwd().parents)
if (candidate / "tests" / "fixtures" / "real_data" / "prism_boulder_january_2024.nc").exists()
)
prism = xr.open_dataset(data_path, engine="scipy").load()
# These assertions are part of the teaching contract: official source,
# canonical cube dimensions, complete daily time, and declared Celsius units.
assert prism.attrs["source"] == "PRISM Group, Oregon State University"
assert prism.attrs["is_synthetic"] == 0
assert prism.sizes == {"time": 30, "y": 24, "x": 24}
assert prism["tmax"].attrs["units"] == "degC"
import numpy as np
source = prism["tmin"].isel(y=slice(9, 14), x=slice(9, 15))
table = source.to_dataframe(name="tmin").reset_index()
cube = (
table.set_index(["time", "y", "x"])
.to_xarray()["tmin"]
.transpose("time", "y", "x")
.sel(time=source.time, y=source.y, x=source.x)
)
cube.attrs.update(source.attrs)
cube.attrs.update(source=prism.attrs["source"], is_synthetic=0)
np.testing.assert_array_equal(cube.values, source.values)
assert len(table) == source.size
cube
Out[1]:
<xarray.DataArray 'tmin' (time: 30, y: 5, x: 6)> Size: 4kB
array([[[-4.6810e+00, -3.5300e+00, -4.0510e+00, -4.6580e+00,
-6.2680e+00, -7.2980e+00],
[-4.0190e+00, -2.8910e+00, -4.1510e+00, -5.6420e+00,
-6.1540e+00, -5.8930e+00],
[-2.6220e+00, -2.4120e+00, -4.3510e+00, -4.7710e+00,
-5.6820e+00, -6.6670e+00],
[-3.1740e+00, -3.2140e+00, -3.4460e+00, -4.5330e+00,
-5.0180e+00, -4.7310e+00],
[-5.2190e+00, -4.8940e+00, -4.2180e+00, -3.1950e+00,
-3.2860e+00, -3.5380e+00]],
[[-5.3960e+00, -4.0310e+00, -2.9770e+00, -3.4670e+00,
-5.6130e+00, -6.9230e+00],
[-4.9180e+00, -3.4670e+00, -3.1150e+00, -4.6210e+00,
-5.4930e+00, -5.2430e+00],
[-3.2590e+00, -2.6160e+00, -3.1850e+00, -3.7310e+00,
-4.9800e+00, -6.1360e+00],
[-3.5430e+00, -3.3990e+00, -2.9990e+00, -3.6000e+00,
-4.2170e+00, -3.9070e+00],
[-6.0730e+00, -5.4220e+00, -4.3500e+00, -2.0480e+00,
...
-2.6880e+00, -4.2090e+00],
[ 8.5600e-01, 2.3840e+00, 1.6790e+00, -1.4540e+00,
-2.8360e+00, -2.4390e+00],
[ 2.4760e+00, 3.0920e+00, 1.3030e+00, -1.0050e+00,
-2.6680e+00, -3.7970e+00],
[ 1.9400e+00, 2.0550e+00, 1.6320e+00, -8.4200e-01,
-1.5020e+00, -9.0300e-01],
[ 4.0000e-03, 7.8200e-01, 1.4300e+00, 1.1050e+00,
1.2200e+00, 8.2500e-01]],
[[-2.3000e+00, -5.3100e-01, 9.1000e-01, 3.7400e-01,
-1.9150e+00, -3.1600e+00],
[-1.5350e+00, 3.5600e-01, 1.0680e+00, -8.7700e-01,
-1.8440e+00, -1.4870e+00],
[ 5.8600e-01, 1.3720e+00, 8.1600e-01, -2.8400e-01,
-1.5530e+00, -2.4450e+00],
[ 9.3000e-02, 3.4000e-01, 7.8000e-01, 4.6000e-02,
-5.6700e-01, -2.2700e-01],
[-2.5880e+00, -1.8700e+00, -8.3100e-01, 1.6580e+00,
1.6670e+00, 1.1990e+00]]], dtype=float32)
Coordinates:
* time (time) datetime64[ns] 240B 2024-01-01 2024-01-02 ... 2024-01-30
* y (y) float64 40B 40.12 40.08 40.04 40.0 39.96
* x (x) float64 48B -105.4 -105.3 -105.3 -105.2 -105.2 -105.2
Attributes:
long_name: daily minimum air temperature
standard_name: air_temperature
units: degC
source_variable: PRISM tmin
is_synthetic: 0
source: PRISM Group, Oregon State UniversityPipe · Standardize each location through time¶
Preparation is complete. The analytical method is one verb.
In [2]:
Copied!
from cubedynamics import pipe, verbs as v
standardized = (
pipe(cube)
| v.zscore(dim="time")
).unwrap()
assert float(abs(standardized.mean("time")).max()) < 1e-5
standardized
from cubedynamics import pipe, verbs as v
standardized = (
pipe(cube)
| v.zscore(dim="time")
).unwrap()
assert float(abs(standardized.mean("time")).max()) < 1e-5
standardized
Out[2]:
<xarray.DataArray 'tmin_zscore' (time: 30, y: 5, x: 6)> Size: 4kB
array([[[ 6.27233863e-01, 6.17691338e-01, 4.82988298e-01,
4.47618753e-01, 4.09540296e-01, 3.74149263e-01],
[ 6.53532386e-01, 6.05838835e-01, 4.59063679e-01,
4.10811722e-01, 4.16493416e-01, 4.33902949e-01],
[ 6.61449671e-01, 6.13045871e-01, 4.54196155e-01,
4.43282902e-01, 4.30597723e-01, 4.08832520e-01],
[ 6.44476414e-01, 6.00199401e-01, 5.42647004e-01,
4.71403837e-01, 4.59263772e-01, 4.78086978e-01],
[ 5.84289134e-01, 5.14877677e-01, 5.35400748e-01,
5.20606995e-01, 5.34833431e-01, 5.33086181e-01]],
[[ 5.20817161e-01, 5.49260259e-01, 6.26188755e-01,
6.11016929e-01, 5.05264878e-01, 4.30105984e-01],
[ 5.27052224e-01, 5.33946991e-01, 5.93468130e-01,
5.56662917e-01, 5.12825787e-01, 5.27915239e-01],
[ 5.80574512e-01, 5.88292420e-01, 6.08614385e-01,
5.89982748e-01, 5.32727242e-01, 4.87292141e-01],
[ 5.96298277e-01, 5.76602519e-01, 6.00418508e-01,
6.01635933e-01, 5.73113203e-01, 5.94130218e-01],
[ 4.64903861e-01, 4.47008252e-01, 5.18290162e-01,
...
[ 1.33939540e+00, 1.26422381e+00, 1.21541274e+00,
1.00907290e+00, 9.00050163e-01, 9.33469594e-01],
[ 1.30870485e+00, 1.28090525e+00, 1.20297873e+00,
9.74505603e-01, 8.69085371e-01, 8.32898915e-01],
[ 1.31218028e+00, 1.27226520e+00, 1.19894147e+00,
9.86609221e-01, 9.59007382e-01, 1.01718116e+00],
[ 1.31444061e+00, 1.24447381e+00, 1.26752543e+00,
1.08742487e+00, 1.13065994e+00, 1.12067080e+00]],
[[ 9.81609166e-01, 1.02732146e+00, 1.14445722e+00,
1.13797951e+00, 1.04570663e+00, 9.91613150e-01],
[ 1.00300598e+00, 1.01110446e+00, 1.13614535e+00,
1.09149814e+00, 1.04462171e+00, 1.07116139e+00],
[ 1.06874561e+00, 1.07219923e+00, 1.13848329e+00,
1.07620811e+00, 1.03129971e+00, 1.03266823e+00],
[ 1.07102883e+00, 1.05351543e+00, 1.08882666e+00,
1.11055994e+00, 1.09190285e+00, 1.11238158e+00],
[ 9.52090859e-01, 9.03584361e-01, 9.74442184e-01,
1.16032028e+00, 1.18976665e+00, 1.17103899e+00]]],
dtype=float32)
Coordinates:
* time (time) datetime64[ns] 240B 2024-01-01 2024-01-02 ... 2024-01-30
* y (y) float64 40B 40.12 40.08 40.04 40.0 39.96
* x (x) float64 48B -105.4 -105.3 -105.3 -105.2 -105.2 -105.2
Attributes:
long_name: daily minimum air temperature
standard_name: air_temperature
units: degC
source_variable: PRISM tmin
is_synthetic: 0
source: PRISM Group, Oregon State UniversityFigure · Compare the observed and standardized records¶
In [3]:
Copied!
import matplotlib.pyplot as plt
site = {"y": float(cube.y[2]), "x": float(cube.x[3])}
fig, axes = plt.subplots(2, 1, figsize=(9, 5.5), sharex=True, constrained_layout=True)
cube.sel(**site).plot(ax=axes[0], marker="o", color="#8b543c")
axes[0].set_title("Observed PRISM daily minimum temperature")
axes[0].set_ylabel("Temperature (°C)")
standardized.sel(**site).plot(ax=axes[1], marker="o", color="#3f6f72")
axes[1].axhline(0, color="0.35", linewidth=0.8)
axes[1].set_title("The same grid cell after v.zscore(dim='time')")
axes[1].set_ylabel("Standard deviations")
plt.show()
import matplotlib.pyplot as plt
site = {"y": float(cube.y[2]), "x": float(cube.x[3])}
fig, axes = plt.subplots(2, 1, figsize=(9, 5.5), sharex=True, constrained_layout=True)
cube.sel(**site).plot(ax=axes[0], marker="o", color="#8b543c")
axes[0].set_title("Observed PRISM daily minimum temperature")
axes[0].set_ylabel("Temperature (°C)")
standardized.sel(**site).plot(ax=axes[1], marker="o", color="#3f6f72")
axes[1].axhline(0, color="0.35", linewidth=0.8)
axes[1].set_title("The same grid cell after v.zscore(dim='time')")
axes[1].set_ylabel("Standard deviations")
plt.show()
Data used¶
| Field | Frozen analysis input |
|---|---|
| Provider | PRISM Group, Oregon State University |
| Product | AN91d daily 4 km time series |
| Dates | 2024-01-01 to 2024-01-30 |
| Fixture | tests/fixtures/real_data/prism_boulder_january_2024.nc |
| Provenance record | tests/fixtures/real_data/prism_boulder_january_2024.provenance.json |
The PRISM source reference describes current catalog support; the fixture record above identifies the observations used here. Data validation documents checksums and acceptance checks. The analytical baseline and thresholds belong to this story, not the provider.
Reproduce¶
Clone the repository, then run these commands from its root:
python -m pip install -e ".[vignettes]"
python scripts/run_vignettes.py docs/vignettes/cube_from_tidy_table.ipynb
No network is needed after installation. Open the downloaded notebook in Jupyter and run all cells to see the same figures. The website executes these cells during its strict build. Environment setup and the vignette contract explain the workflow.