02 · From observations to a comparable signal¶
Context¶
Climate observations are often exchanged as one row per date and location. Here we flatten official PRISM minimum temperature into that familiar table, then recover the cube without changing a value.
Question¶
Can we compare departures through time at sites with different baseline temperatures?
Analysis story¶
We will prove the table-to-cube round trip, standardize each location through time, and compare the observed and standardized series at one grid cell.
Data used in this lesson¶
Every value comes from the PRISM Group at Oregon State University's AN91d daily 4 km climate product. This repository carries a small Boulder-region extract for 1–30 January 2024 so the lesson runs offline without replacing observations with generated values. The data validation page records source URLs, terms, checksums, bounds, units, and acceptance tests.
In [1]:
Copied!
# Record the exact code imported by this notebook kernel.
import cubedynamics as cd
print(cd.version_info())
# Record the exact code imported by this notebook kernel.
import cubedynamics as cd
print(cd.version_info())
CubeDynamics 0.1.0rc3 Artifact: development checkout Code: git:a852137944cabf238e9321f2eecf09e483c9b9f5 Imported from: /home/runner/work/cubedynamics/cubedynamics/src/cubedynamics Distribution: /opt/hostedtoolcache/Python/3.11.16/x64/lib/python3.11/site-packages
Prepare · Round-trip real observations through a tidy table¶
In [2]:
Copied!
from pathlib import Path
import xarray as xr
# Find the repository from either a root-level documentation build or a kernel
# started beside this notebook, then open the checksum-controlled PRISM extract.
data_path = next(
candidate / "tests" / "fixtures" / "real_data" / "prism_boulder_january_2024.nc"
for candidate in (Path.cwd(), *Path.cwd().parents)
if (candidate / "tests" / "fixtures" / "real_data" / "prism_boulder_january_2024.nc").exists()
)
prism = xr.open_dataset(data_path, engine="scipy").load()
# These assertions are part of the teaching contract: official source,
# canonical cube dimensions, complete daily time, and declared Celsius units.
assert prism.attrs["source"] == "PRISM Group, Oregon State University"
assert prism.attrs["is_synthetic"] == 0
assert prism.sizes == {"time": 30, "y": 24, "x": 24}
assert prism["tmax"].attrs["units"] == "degC"
import numpy as np
source = prism["tmin"].isel(y=slice(9, 14), x=slice(9, 15))
table = source.to_dataframe(name="tmin").reset_index()
cube = (
table.set_index(["time", "y", "x"])
.to_xarray()["tmin"]
.transpose("time", "y", "x")
.sel(time=source.time, y=source.y, x=source.x)
)
cube.attrs.update(source.attrs)
cube.attrs.update(source=prism.attrs["source"], is_synthetic=0)
np.testing.assert_array_equal(cube.values, source.values)
assert len(table) == source.size
cube
from pathlib import Path
import xarray as xr
# Find the repository from either a root-level documentation build or a kernel
# started beside this notebook, then open the checksum-controlled PRISM extract.
data_path = next(
candidate / "tests" / "fixtures" / "real_data" / "prism_boulder_january_2024.nc"
for candidate in (Path.cwd(), *Path.cwd().parents)
if (candidate / "tests" / "fixtures" / "real_data" / "prism_boulder_january_2024.nc").exists()
)
prism = xr.open_dataset(data_path, engine="scipy").load()
# These assertions are part of the teaching contract: official source,
# canonical cube dimensions, complete daily time, and declared Celsius units.
assert prism.attrs["source"] == "PRISM Group, Oregon State University"
assert prism.attrs["is_synthetic"] == 0
assert prism.sizes == {"time": 30, "y": 24, "x": 24}
assert prism["tmax"].attrs["units"] == "degC"
import numpy as np
source = prism["tmin"].isel(y=slice(9, 14), x=slice(9, 15))
table = source.to_dataframe(name="tmin").reset_index()
cube = (
table.set_index(["time", "y", "x"])
.to_xarray()["tmin"]
.transpose("time", "y", "x")
.sel(time=source.time, y=source.y, x=source.x)
)
cube.attrs.update(source.attrs)
cube.attrs.update(source=prism.attrs["source"], is_synthetic=0)
np.testing.assert_array_equal(cube.values, source.values)
assert len(table) == source.size
cube
Out[2]:
<xarray.DataArray 'tmin' (time: 30, y: 5, x: 6)> Size: 4kB
array([[[-4.6810e+00, -3.5300e+00, -4.0510e+00, -4.6580e+00,
-6.2680e+00, -7.2980e+00],
[-4.0190e+00, -2.8910e+00, -4.1510e+00, -5.6420e+00,
-6.1540e+00, -5.8930e+00],
[-2.6220e+00, -2.4120e+00, -4.3510e+00, -4.7710e+00,
-5.6820e+00, -6.6670e+00],
[-3.1740e+00, -3.2140e+00, -3.4460e+00, -4.5330e+00,
-5.0180e+00, -4.7310e+00],
[-5.2190e+00, -4.8940e+00, -4.2180e+00, -3.1950e+00,
-3.2860e+00, -3.5380e+00]],
[[-5.3960e+00, -4.0310e+00, -2.9770e+00, -3.4670e+00,
-5.6130e+00, -6.9230e+00],
[-4.9180e+00, -3.4670e+00, -3.1150e+00, -4.6210e+00,
-5.4930e+00, -5.2430e+00],
[-3.2590e+00, -2.6160e+00, -3.1850e+00, -3.7310e+00,
-4.9800e+00, -6.1360e+00],
[-3.5430e+00, -3.3990e+00, -2.9990e+00, -3.6000e+00,
-4.2170e+00, -3.9070e+00],
[-6.0730e+00, -5.4220e+00, -4.3500e+00, -2.0480e+00,
...
-2.6880e+00, -4.2090e+00],
[ 8.5600e-01, 2.3840e+00, 1.6790e+00, -1.4540e+00,
-2.8360e+00, -2.4390e+00],
[ 2.4760e+00, 3.0920e+00, 1.3030e+00, -1.0050e+00,
-2.6680e+00, -3.7970e+00],
[ 1.9400e+00, 2.0550e+00, 1.6320e+00, -8.4200e-01,
-1.5020e+00, -9.0300e-01],
[ 4.0000e-03, 7.8200e-01, 1.4300e+00, 1.1050e+00,
1.2200e+00, 8.2500e-01]],
[[-2.3000e+00, -5.3100e-01, 9.1000e-01, 3.7400e-01,
-1.9150e+00, -3.1600e+00],
[-1.5350e+00, 3.5600e-01, 1.0680e+00, -8.7700e-01,
-1.8440e+00, -1.4870e+00],
[ 5.8600e-01, 1.3720e+00, 8.1600e-01, -2.8400e-01,
-1.5530e+00, -2.4450e+00],
[ 9.3000e-02, 3.4000e-01, 7.8000e-01, 4.6000e-02,
-5.6700e-01, -2.2700e-01],
[-2.5880e+00, -1.8700e+00, -8.3100e-01, 1.6580e+00,
1.6670e+00, 1.1990e+00]]], dtype=float32)
Coordinates:
* time (time) datetime64[ns] 240B 2024-01-01 2024-01-02 ... 2024-01-30
* y (y) float64 40B 40.12 40.08 40.04 40.0 39.96
* x (x) float64 48B -105.4 -105.3 -105.3 -105.2 -105.2 -105.2
Attributes:
long_name: daily minimum air temperature
standard_name: air_temperature
units: degC
source_variable: PRISM tmin
is_synthetic: 0
source: PRISM Group, Oregon State UniversityPipe · Standardize each location through time¶
Preparation is complete. The analytical method is one verb.
In [3]:
Copied!
from cubedynamics import pipe, verbs as v
standardized = (
pipe(cube)
| v.zscore(dim="time")
).unwrap()
assert float(abs(standardized.mean("time")).max()) < 1e-5
standardized
from cubedynamics import pipe, verbs as v
standardized = (
pipe(cube)
| v.zscore(dim="time")
).unwrap()
assert float(abs(standardized.mean("time")).max()) < 1e-5
standardized
Out[3]:
<xarray.DataArray 'tmin_zscore' (time: 30, y: 5, x: 6)> Size: 4kB
array([[[ 6.27233863e-01, 6.17691338e-01, 4.82988298e-01,
4.47618753e-01, 4.09540296e-01, 3.74149263e-01],
[ 6.53532386e-01, 6.05838835e-01, 4.59063679e-01,
4.10811722e-01, 4.16493416e-01, 4.33902949e-01],
[ 6.61449671e-01, 6.13045871e-01, 4.54196155e-01,
4.43282902e-01, 4.30597723e-01, 4.08832520e-01],
[ 6.44476414e-01, 6.00199401e-01, 5.42647004e-01,
4.71403837e-01, 4.59263772e-01, 4.78086978e-01],
[ 5.84289134e-01, 5.14877677e-01, 5.35400748e-01,
5.20606995e-01, 5.34833431e-01, 5.33086181e-01]],
[[ 5.20817161e-01, 5.49260259e-01, 6.26188755e-01,
6.11016929e-01, 5.05264878e-01, 4.30105984e-01],
[ 5.27052224e-01, 5.33946991e-01, 5.93468130e-01,
5.56662917e-01, 5.12825787e-01, 5.27915239e-01],
[ 5.80574512e-01, 5.88292420e-01, 6.08614385e-01,
5.89982748e-01, 5.32727242e-01, 4.87292141e-01],
[ 5.96298277e-01, 5.76602519e-01, 6.00418508e-01,
6.01635933e-01, 5.73113203e-01, 5.94130218e-01],
[ 4.64903861e-01, 4.47008252e-01, 5.18290162e-01,
...
[ 1.33939540e+00, 1.26422381e+00, 1.21541274e+00,
1.00907290e+00, 9.00050163e-01, 9.33469594e-01],
[ 1.30870485e+00, 1.28090525e+00, 1.20297873e+00,
9.74505603e-01, 8.69085371e-01, 8.32898915e-01],
[ 1.31218028e+00, 1.27226520e+00, 1.19894147e+00,
9.86609221e-01, 9.59007382e-01, 1.01718116e+00],
[ 1.31444061e+00, 1.24447381e+00, 1.26752543e+00,
1.08742487e+00, 1.13065994e+00, 1.12067080e+00]],
[[ 9.81609166e-01, 1.02732146e+00, 1.14445722e+00,
1.13797951e+00, 1.04570663e+00, 9.91613150e-01],
[ 1.00300598e+00, 1.01110446e+00, 1.13614535e+00,
1.09149814e+00, 1.04462171e+00, 1.07116139e+00],
[ 1.06874561e+00, 1.07219923e+00, 1.13848329e+00,
1.07620811e+00, 1.03129971e+00, 1.03266823e+00],
[ 1.07102883e+00, 1.05351543e+00, 1.08882666e+00,
1.11055994e+00, 1.09190285e+00, 1.11238158e+00],
[ 9.52090859e-01, 9.03584361e-01, 9.74442184e-01,
1.16032028e+00, 1.18976665e+00, 1.17103899e+00]]],
dtype=float32)
Coordinates:
* time (time) datetime64[ns] 240B 2024-01-01 2024-01-02 ... 2024-01-30
* y (y) float64 40B 40.12 40.08 40.04 40.0 39.96
* x (x) float64 48B -105.4 -105.3 -105.3 -105.2 -105.2 -105.2
Attributes:
long_name: daily minimum air temperature
standard_name: air_temperature
units: 1
source_variable: PRISM tmin
is_synthetic: 0
source: PRISM Group, Oregon State University
analysis: zscore
semantic_name: zscore of tmin
semantic_kind: continuous_field
semantic_category: centered_transform
transform_operation: zscore
semantic_units: standard deviationsFigure · Compare the observed and standardized records¶
In [4]:
Copied!
import matplotlib.pyplot as plt
site = {"y": float(cube.y[2]), "x": float(cube.x[3])}
fig, axes = plt.subplots(2, 1, figsize=(9, 5.5), sharex=True, constrained_layout=True)
cube.sel(**site).plot(ax=axes[0], marker="o", color="#8b543c")
axes[0].set_title("Observed PRISM daily minimum temperature")
axes[0].set_ylabel("Temperature (°C)")
standardized.sel(**site).plot(ax=axes[1], marker="o", color="#3f6f72")
axes[1].axhline(0, color="0.35", linewidth=0.8)
axes[1].set_title("The same grid cell after v.zscore(dim='time')")
axes[1].set_ylabel("Standard deviations")
plt.show()
import matplotlib.pyplot as plt
site = {"y": float(cube.y[2]), "x": float(cube.x[3])}
fig, axes = plt.subplots(2, 1, figsize=(9, 5.5), sharex=True, constrained_layout=True)
cube.sel(**site).plot(ax=axes[0], marker="o", color="#8b543c")
axes[0].set_title("Observed PRISM daily minimum temperature")
axes[0].set_ylabel("Temperature (°C)")
standardized.sel(**site).plot(ax=axes[1], marker="o", color="#3f6f72")
axes[1].axhline(0, color="0.35", linewidth=0.8)
axes[1].set_title("The same grid cell after v.zscore(dim='time')")
axes[1].set_ylabel("Standard deviations")
plt.show()
Data used¶
| Field | Frozen analysis input |
|---|---|
| Provider | PRISM Group, Oregon State University |
| Product | AN91d daily 4 km time series |
| Dates | 2024-01-01 to 2024-01-30 |
| Fixture | tests/fixtures/real_data/prism_boulder_january_2024.nc |
| Provenance record | tests/fixtures/real_data/prism_boulder_january_2024.provenance.json |
The PRISM source reference describes current catalog support; the fixture record above identifies the observations used here. Data validation documents checksums and acceptance checks. The analytical baseline and thresholds belong to this story, not the provider.
Reproduce¶
Clone the repository, then run these commands from its root:
python -m pip install -e ".[vignettes]"
python scripts/run_vignettes.py docs/vignettes/cube_from_tidy_table.ipynb
No network is needed after installation. Open the downloaded notebook in
Jupyter and run all cells to see the same figures. The website executes these
cells during its strict build. Environment setup
and the vignette contract explain the workflow.
The first code cell prints cd.version_info() so a rendered result can be tied
to a package path and, for development checkouts, a Git commit.