Configuration

Acquisition configuration is expressed through the installed shyft.integration.met_no Python types. The service must already be running; Python connects remotely rather than constructing a local service. Detailed signatures come from the Installed Python API reference, not from manually maintained copies of the generated stubs.

Versions and activation

The lifecycle is: construct target/configuration, store a model/configuration version, activate that version, then let its service agent reconcile. ModelInfo.name is the logical configuration name; the returned mid is the persistent ID of one version. Multiple versions can share a name, but the manager runs the selected active version for that name. The installed met_no.Client exposes persistent active-version administration:

get_versions(name)

Return a Python list of existing ModelInfo entries with that exact logical name, ordered by created timestamp and then id as the tie-breaker. This uses the existing list conversion (ModelInfoVector is a list alias), not a new custom container. An unknown name returns an empty list.

get_active_version(name)

Return the persisted selected mid, or None if no version is active.

get_active_versions()

Return the service-wide persisted selection as a Python dict[str, int] mapping logical names to version IDs. This is not a runtime agent snapshot.

set_active_version(name, mid)

Synchronously persist the selection of a stored version belonging to that name. When the manager is running, start its agent or replace the previous version’s agent. Selecting the same already-active mid does not restart the agent. Acknowledgement is not proof of target readiness or acquisition completion; inspect status afterwards.

clear_active_version(name)

Synchronously stop, join, and remove the agent, if present, and remove only the persistent active selection. Repeated clearing is harmless. Stored versions, acquisition bookkeeping, fc/cc data, and catalog cache remain intact.

An active version is the persisted selection for its logical name; its agent may still be resolving or blocked. A passive version is stored but not selected and has no agent acquiring on its behalf. A deleted version has been removed through model CRUD; clearing activation does not delete a version.

Generic model CRUD remains separate: store_model does not activate or switch versions, and ModelInfo.json is metadata, not an activation command. Restarting the executable restores persisted active selections, not passive versions. The client exposes no service start/stop/restart operation; use the process supervisor described in Operations. Do not open the running service’s RocksDB directory concurrently for administration.

An active version cannot be overwritten with store_model: store a new version or clear its selection first. Replacing a passive version with changed acquisition configuration clears that version’s acquisition bookkeeping, not its fc/cc data or the shared catalog cache. Renaming a version through update_model_info clears its old active selection and stops its agent; the renamed version remains passive until explicitly selected under the new name.

Preparing, storing, and activating a configuration

This example uses the construction and CRUD patterns from python/test_suites/integration/test_met_no_types.py and python/test_suites/integration/test_met_no.py. Adjust the endpoints and database name to your deployment. The service listens on 127.0.0.1:20080; DTSS is a separate service, here on 127.0.0.1:20000. Before activation, meps_fc must already have a grid, CRS, one ensemble slot for this deterministic product, and the air_temperature_2m variable.

from shyft.integration import met_no
from shyft.hydrology import IDWParameter
from shyft.time_series import ModelInfo, UtcPeriod, max_utctime, time, utctime_now

client = met_no.Client(host_port='127.0.0.1:20080', timeout_ms=1000,
                       operation_timeout_ms=2000)
try:
    target = met_no.DtssGeoTarget()
    target.host_ports = ['127.0.0.1:20000']
    target.fc_db = 'meps_fc'

    temperature = met_no.VariableConfig()
    temperature.name = 'air_temperature_2m'
    temperature.offset = -273.15  # Kelvin to degrees Celsius for this target.

    cfg = met_no.Config()
    cfg.target = target
    cfg.period = UtcPeriod(time('2026-06-16T00:00:00Z'), max_utctime)
    cfg.extraction.variables = met_no.VariableConfigVector([temperature])
    cfg.source_buffer_distance = 15000.0
    cfg.interpolation = met_no.InterpolationConfig()
    cfg.interpolation.default_parameter = IDWParameter(
        max_members=7, max_distance=80000)

    info = ModelInfo(id=0, name='site-weather', created=utctime_now())
    mid = client.store_model(m=cfg, mi=info)
    cfg.id = mid
    print('Stored version:', mid)
    print(client.read_model(mid=mid))

    client.set_active_version(name=info.name, mid=mid)
    print('Active version:', client.get_active_version(name=info.name))
    agent = client.get_agent_status(name=info.name)
    if (agent is not None and agent.mid == mid
            and agent.state in (met_no.AgentState.READY,
                                met_no.AgentState.RECONCILING)):
        client.request_reconcile(name=info.name)
        print(client.get_agent_status(name=info.name))
    else:
        print('Inspect target resolution before requesting work:', agent)
finally:
    client.close()

Activation is separate from storage and starts the agent in the running service. Use the Service health and agent status and Request reconciliation now examples to observe and request work. The reconciliation request acknowledges scheduling, not completion.

For a new version, use a new Config with id == 0 (or reset a loaded configuration’s id to zero), together with ModelInfo(id=0, ...). To update an existing version, use matching nonzero configuration and metadata IDs. get_model_infos([]) lists all stored version metadata; read_model(mid) reads a version. Keeping stored versions separate from activation makes the desired version explicit.

Preparing a new version, then switching

After the example above, prepare a replacement under the same logical name. The current selection remains unchanged until the explicit switch:

client = met_no.Client(host_port='127.0.0.1:20080', timeout_ms=1000,
                       operation_timeout_ms=2000)
try:
    name = 'site-weather'
    current_mid = client.get_active_version(name)
    if current_mid is None:
        raise RuntimeError('Activate an initial version first.')
    replacement = client.read_model(current_mid)
    replacement.id = 0
    replacement.source_buffer_distance = 20000.0
    info = ModelInfo(id=0, name=name, created=utctime_now())
    new_mid = client.store_model(m=replacement, mi=info)
    print('Still active:', client.get_active_version(name))
    print('Stored versions:', [mi.id for mi in client.get_versions(name)])

    client.set_active_version(name=name, mid=new_mid)
    print('Selections:', client.get_active_versions())
    print('Replacement agent:', client.get_agent_status(name))
finally:
    client.close()

The old version remains stored and passive after the switch. Select its stored ID again to switch back; neither switching nor storing deletes forecast data.

Making a configuration passive

Clear the selection to stop acquisition for this name without deleting its versions or data:

client = met_no.Client(host_port='127.0.0.1:20080', timeout_ms=1000,
                       operation_timeout_ms=2000)
try:
    name = 'site-weather'
    client.clear_active_version(name=name)
    print(client.get_active_version(name))  # None
    print(client.get_agent_status(name))  # None
    print('Retained versions:', [mi.id for mi in client.get_versions(name)])
finally:
    client.close()

All retained versions for that name are now passive. Restarting the service does not reactivate them; explicitly select a retained mid to resume acquisition.

Forecast-reference period

Config.period selects forecast reference times, not the valid-time span inside each forecast. It uses Shyft’s normal [start, end) semantics: start is included, end is excluded. Discovery considers reference times before the current time, bounded by the configured end.

For a finite historical/backfill interval:

from shyft.time_series import UtcPeriod, time

cfg.period = UtcPeriod(time('2020-01-01T00:00:00Z'),
                       time('2020-02-01T00:00:00Z'))

For ongoing operation, max_utctime leaves the end open to future runs:

from shyft.time_series import UtcPeriod, max_utctime, time

cfg.period = UtcPeriod(time('2026-06-16T00:00:00Z'), max_utctime)

These examples change cfg locally; store the desired version and select it with set_active_version before expecting the service to follow it.

Forecast target and product

DtssGeoTarget.host_ports supplies the DTSS host:port endpoints. DtssGeoTarget.fc_db names the existing forecast geo database. Its CRS, grid, variables, and ensemble layout define/limit the destination representation. Every extracted variable must exist in fc.variables; member indices must fit its ensemble slots. Geometry must match directly when interpolation is disabled, or be produced on the authoritative target grid by interpolation. The service does not expand a target grid or add variable/ensemble definitions.

Config.product is an editable ForecastProduct. Its defaults are name='meps_det_sfc', archive_root='meps25epsarchive', and extension='.ncml'. The member product is meps_mbr_sfc. Product name, archive root, and extension describe catalog discovery; configure them for the MET archive/product being acquired, not as arbitrary display labels.

Source buffer

Config.source_buffer_distance is a fixed horizontal acquisition/source-domain buffer in metres, around the authoritative fc target. It determines how much MET source geometry is downloaded. Its default is zero.

The source buffer is deliberately independent of interpolation parameters. Changing an IDW max_distance does not implicitly enlarge the downloaded source window. Choose the buffer and interpolation neighborhood together for your target, but configure them separately.

Interpolation

InterpolationConfig.default_parameter is the fallback for variables without an exact override in the sparse parameters mapping. Keys are extracted MET variable names, not downstream model aliases. Supported parameter alternatives are the existing Shyft hydrology types: generic IDWParameter, temperature-aware IDWTemperatureParameter, and precipitation-aware IDWPrecipitationParameter. Their use here does not make weather integration hydrology-specific.

from shyft.integration import met_no
from shyft.hydrology import (
    IDWParameter, IDWTemperatureParameter, IDWPrecipitationParameter)

interpolation = met_no.InterpolationConfig()
interpolation.default_parameter = IDWParameter(max_members=7, max_distance=80000)
interpolation.parameters['air_temperature_2m'] = IDWTemperatureParameter(
    default_gradient=-0.004)
interpolation.parameters['precipitation_amount_acc'] = IDWPrecipitationParameter(
    scale_factor=1.08)
cfg.interpolation = interpolation

parameter_for(variable) returns that variable’s override or default_parameter. The temperature-aware variant accounts for elevation effects; the precipitation-aware variant supplies precipitation scaling. See Hydrology for the parameter details. To disable interpolation, set cfg.interpolation = None: source geometry must then already be compatible with fc. It does not mean that arbitrary source grids are accepted.

Extraction and variables

Config.extraction is an ExtractionPolicy. Select variables with a mutable VariableConfigVector. Each VariableConfig has:

• name: the MET variable name, also the interpolation override key and the target variable name;

• conversion: a SourceConversionPolicy;

• fill_value: source missing-value marker (default approximately 9.96921e36), or None to disable that explicit marker;

• scale: multiplier applied to valid source values (default 1.0);

• offset: added after scaling (default 0.0).

Values matching the configured fill marker become missing values rather than being scaled as observations. Unit conversion is explicit: for example, use an offset of -273.15 if the temperature source is Kelvin and the target expects Celsius.

SourceConversionPolicy.SAMPLE_AND_HOLD_TO_AVERAGE is the default: source samples become interval-average values held until the next step. ACCUMULATED_TO_RATE differences successive accumulated values and divides by the elapsed seconds; it requires at least two source samples. Choose any additional unit scaling for the target accordingly. NO_CONVERT retains instantaneous source values rather than converting them to interval averages/rates. Scale and offset still apply.

precipitation = met_no.VariableConfig()
precipitation.name = 'precipitation_amount_acc'
precipitation.conversion = met_no.SourceConversionPolicy.ACCUMULATED_TO_RATE
precipitation.fill_value = None
precipitation.scale = 1.0
precipitation.offset = 0.0
cfg.extraction.variables = met_no.VariableConfigVector([temperature, precipitation])

Retain the normal extraction defaults for service acquisition: GeometryMode.INCLUDE_SOURCE and ElevationMode.SOURCE_GEOPOTENTIAL. They supply source longitude/latitude and elevation from geopotential. GeometryMode.NONE omits source geometry and is not a substitute for service geometry mapping. Although the enum includes ElevationMode.EXTERNAL_LATER, external elevation supply is not implemented.

Nested configuration objects, the variable vector, and interpolation map can be edited in place. host_ports and ConcatConfig.ensembles are read as copied Python lists: assign a replacement list to change the configuration.

Concatenated forecast configuration

Config.concat = None disables concatenation. Otherwise, ConcatConfig.cc_db names the pre-existing derived database. forecast_offset skips the beginning of each forecast (often model warm-up); horizon selects the duration after that offset. Both are Shyft time durations expressed in seconds.

The selected valid-time interval is:

[forecast_reference_time + forecast_offset,
 forecast_reference_time + forecast_offset + horizon)

The interval must be nonempty, fit the stored forecast, and align with its time axis. The zero default horizon must therefore be replaced with a positive duration when enabling concat.

concat = met_no.ConcatConfig()
concat.cc_db = 'meps_cc'
concat.forecast_offset = time(3 * 3600)
concat.horizon = time(24 * 3600)
concat.ensembles = []
cfg.concat = concat

ensembles == [] means all available source members, in index order. A non-empty list selects fc member indices in the configured order: for example, [0, 2] maps source members 0 and 2 to destination slots 0 and 1. All selected indices must fit fc.n_ensembles and the selection length must match cc.n_ensembles. With an empty list the ensemble counts must match. The fc and cc grids and CRS/projection definitions must be identical, and selected variables must be represented by the target databases.

The cc anchor precondition

cc has exactly one base/anchor t0. Pre-create and seed it with a valid time axis according to the DTSS geo database requirements before enabling concat. For each incoming forecast:

cc_anchor_t0 <= forecast_reference_time

If historical backfill is expected, seed cc with a sufficiently early t0:

cc anchored in 2020
    -> can later process 2020...2026 forecasts in arbitrary fetch order

cc anchored in 2026
    -> cannot later merge a forecast whose reference time is in 2020

This permits out-of-order processing subject to the other concat conditions; it does not remove the anchor constraint. The service does not create or rebase cc automatically. A bad anchor can produce concat diagnostics even while acquisition into fc succeeds.

Concat reads verified fc data, not MET. It can be rerun safely for the same forecast under the normal DTSS merge/concat conditions. Reconciliation (including after restart) can repair/reconstruct derived cc content from discovered forecasts already complete in fc, without redownloading those forecasts. A temporary concat failure does not undo successful acquisition. See Operations for recovery limits and status counters.