Skip to content

Representation generation

representations.run(observations, encoder="scalar") accepts a batch or iterator of validated HarmonizedObservation contracts and yields immutable representations. batches(observations, size=64) supplies a bounded batching interface. Discovery and MESA are absent from this module.

Two deterministic local encoders demonstrate substitutability:

Config name Encoder identity Features Model preprocessing
scalar identity-scalar, version 1 [value] None; scientific units retained
squared scalar-and-square, version 1 [value, value²] Append square, no scaling

Every vector records observation identifiers, modality, encoder/version, preprocessing, dimensionality, track, and complete source/recipe/review lineage. Missing values, unknown encoders, malformed contracts, nonfinite vectors, and inconsistent dimensionality fail. Field and satellite vectors remain separate. The encoders do not change scientific units or perform concealed harmonization.

Rolf et al. (2021), MOSAIKS provides the precedent of computing reusable image features once for many tasks. The current scalar demo does not implement MOSAIKS or reproduce its results. Future encoder adapters may support random image features, remote-sensing foundation models, traits, tabular data, or text; scientific preprocessing still belongs in recipes and encoder preprocessing must remain explicit.

Metadata consensus and direct streaming

The streaming harmonization module adds metadata-only specialist dialogue, digest-bound unanimous review, documented Sentinel-2 scale/offset calibration, and bounded window-to-encoder execution without raw or harmonized data files. The existing fixture/snapshot commands retain their historical materialization behavior. Current streamed vectors are reference numeric features; model consensus still requires an actual human decision before execution.

MOSAIKS over harmonized outputs

The MOSAIKS module consumes bounded harmonized raster tiles, freezes train-only empirical convolution filters, writes dense feature datasets with scientific lineage, and verifies held-out ridge-model consumption against baselines. It does not encode raw source assets. Generated diagnostics exercise the complete mechanism; independent scientific validation and real-data review remain separate requirements.

Scientific meaning precedes embedding

Raw → semantically harmonized → aligned → embedded describes distinct responsibilities. Any alignment that changes scientific support belongs in an explicitly reviewed recipe; encoder layout and numerical preprocessing have separate identities. Embeddings do not perform or validate semantic harmonization.

MOSAIKS consumes harmonized raster outputs only. Canonical definitions, units, source/recipe/review identity, native support and window evidence accompany the vectors through compact lineage and metadata artifacts. A numeric feature can be usable by a downstream model while its scientific interpretation remains unvalidated. Verification limits · Semantic provenance

Deterministic backbone and model trust

VERDE interprets, selects, parameterizes, compares and proposes. It does not author scientific computation. New model plans select closed, source-bound capabilities; metadata coefficients and mathematical operations are resolved by tested code. Schema validation, scientific checks, existing vetoes and human decisions remain separate gates. Approved recipes execute without a model. See deterministic execution and model trust for the registry, IR contracts, examples, provenance and current limits.