headroomlabs-ai/headroom · error · RolloutConfigurationError

unsupported rollout worker schema version

Error message

unsupported rollout worker schema version

What it means

Raised by RolloutSnapshot.from_internal_dict() when the restored dict's 'schema_version' does not equal ROLLOUT_SCHEMA_VERSION. Snapshots are versioned so that handoffs between processes running different headroom builds fail loudly instead of misinterpreting fields.

Source

Thrown at headroom/rollout.py:292

            "registry_digest": self.registry_digest,
            "snapshot_digest": self.snapshot_digest,
            "channel": self.channel.value,
            "unsafe_allow_unstable": self.unsafe_allow_unstable,
            "explicit_requested": sorted(self.config.explicit_requested),
            "explicit_disabled": sorted(self.config.explicit_disabled),
            "legacy_requested": sorted(self.config.legacy_requested),
            "legacy_disabled": sorted(self.config.legacy_disabled),
        }

    @classmethod
    def from_internal_dict(cls, value: Mapping[str, object]) -> RolloutSnapshot:
        """Validate and restore a snapshot serialized for worker handoff."""

        if not isinstance(value, Mapping):
            raise RolloutConfigurationError("invalid rollout worker snapshot")
        try:
            if value.get("schema_version") != ROLLOUT_SCHEMA_VERSION:
                raise RolloutConfigurationError("unsupported rollout worker schema version")
            if value.get("policy_version") != ROLLOUT_POLICY_VERSION:
                raise RolloutConfigurationError("rollout worker policy version mismatch")
            channel = RolloutChannel.parse(str(value["channel"]), strict=True)
            unsafe = value["unsafe_allow_unstable"]
            if not isinstance(unsafe, bool):
                raise RolloutConfigurationError("invalid rollout worker unsafe override")

            def names(field: str) -> set[str]:
                raw = value[field]
                if not isinstance(raw, list) or not all(isinstance(item, str) for item in raw):
                    raise RolloutConfigurationError(f"invalid rollout worker field {field!r}")
                return set(_validate_names(set(raw), source=field, strict=True))

            snapshot = _resolve_snapshot(
                channel=channel,
                explicit_requested=names("explicit_requested"),
                explicit_disabled=names("explicit_disabled"),
                legacy_requested=names("legacy_requested"),

View on GitHub (pinned to 322425c43b)

Solutions

1. Align headroom versions across proxy and worker processes (same wheel version).
2. Drain or discard in-flight snapshots across an upgrade boundary instead of replaying them.
3. When constructing snapshots in code, emit them via to_internal_dict() rather than hand-writing the dict.

Example fix

# before
handoff = {"schema_version": 1, ...}  # hardcoded, drifts from library

# after
handoff = live_snapshot.to_internal_dict()  # always carries the right schema_version

from headroom.rollout import ROLLOUT_SCHEMA_VERSION

if payload.get('schema_version') != ROLLOUT_SCHEMA_VERSION:
    raise HandoffVersionError(f'snapshot schema {payload.get("schema_version")} != local {ROLLOUT_SCHEMA_VERSION}')
snapshot = RolloutSnapshot.from_internal_dict(payload)

def snapshot_schema_matches(payload: Mapping) -> bool:
    from headroom.rollout import ROLLOUT_SCHEMA_VERSION
    return payload.get('schema_version') == ROLLOUT_SCHEMA_VERSION

try:
    snapshot = RolloutSnapshot.from_internal_dict(payload)
except RolloutConfigurationError as e:
    if 'schema version' in str(e):
        snapshot = rebuild_snapshot_from_config()  # version skew: rebuild locally
    else:
        raise

When it happens

Trigger: A proxy running headroom X serializes a snapshot and a worker running headroom Y (different ROLLOUT_SCHEMA_VERSION) calls from_internal_dict() on it; or a hand-crafted dict omits or mutates 'schema_version'.

Common situations: Rolling deploys where proxy and worker versions skew; stale snapshots replayed from a queue after an upgrade; tests constructing snapshots by hand with a wrong constant.

Related errors

• invalid rollout worker snapshot
• rollout worker policy version mismatch
• invalid rollout worker unsafe override
• invalid rollout worker field {field!r}
• rollout worker registry digest mismatch

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/2c73d1f2bb141879. Report an issue: GitHub.