headroomlabs-ai/headroom · error · RolloutConfigurationError

invalid rollout worker snapshot

Error message

invalid rollout worker snapshot

What it means

Raised by RolloutSnapshot.from_internal_dict() when the value passed to restore a worker-handoff snapshot is not a Mapping at all. This is the coarsest guard: it rejects non-dict payloads (str, list, None) before any field validation; later guards produce more specific messages.

Source

Thrown at headroom/rollout.py:289

        return {
            "schema_version": self.schema_version,
            "policy_version": self.policy_version,
            "registry_digest": self.registry_digest,
            "snapshot_digest": self.snapshot_digest,
            "channel": self.channel.value,
            "unsafe_allow_unstable": self.unsafe_allow_unstable,
            "explicit_requested": sorted(self.config.explicit_requested),
            "explicit_disabled": sorted(self.config.explicit_disabled),
            "legacy_requested": sorted(self.config.legacy_requested),
            "legacy_disabled": sorted(self.config.legacy_disabled),
        }

    @classmethod
    def from_internal_dict(cls, value: Mapping[str, object]) -> RolloutSnapshot:
        """Validate and restore a snapshot serialized for worker handoff."""

        if not isinstance(value, Mapping):
            raise RolloutConfigurationError("invalid rollout worker snapshot")
        try:
            if value.get("schema_version") != ROLLOUT_SCHEMA_VERSION:
                raise RolloutConfigurationError("unsupported rollout worker schema version")
            if value.get("policy_version") != ROLLOUT_POLICY_VERSION:
                raise RolloutConfigurationError("rollout worker policy version mismatch")
            channel = RolloutChannel.parse(str(value["channel"]), strict=True)
            unsafe = value["unsafe_allow_unstable"]
            if not isinstance(unsafe, bool):
                raise RolloutConfigurationError("invalid rollout worker unsafe override")

            def names(field: str) -> set[str]:
                raw = value[field]
                if not isinstance(raw, list) or not all(isinstance(item, str) for item in raw):
                    raise RolloutConfigurationError(f"invalid rollout worker field {field!r}")
                return set(_validate_names(set(raw), source=field, strict=True))

            snapshot = _resolve_snapshot(
                channel=channel,

View on GitHub (pinned to 322425c43b)

Solutions

1. Pass the already-deserialized dict: from_internal_dict(json.loads(payload)).
2. Inspect for double encoding — if the value is a str, json.loads it once, then retry.
3. Type-check at the boundary that hands the snapshot to the worker.

Example fix

# before
snapshot = RolloutSnapshot.from_internal_dict(raw_body)  # raw_body is str

# after
import json
snapshot = RolloutSnapshot.from_internal_dict(json.loads(raw_body))

from collections.abc import Mapping
import json

if isinstance(payload, (str, bytes)):
    payload = json.loads(payload)
if not isinstance(payload, Mapping):
    raise ValueError(f'snapshot payload must be a mapping, got {type(payload).__name__}')
snapshot = RolloutSnapshot.from_internal_dict(payload)

from collections.abc import Mapping

def is_snapshot_mapping(value: object) -> bool:
    return isinstance(value, Mapping)

from headroom.rollout import RolloutConfigurationError

try:
    snapshot = RolloutSnapshot.from_internal_dict(payload)
except RolloutConfigurationError as e:
    logger.error('worker snapshot rejected: %s', e)
    request_fresh_snapshot()  # re-handoff instead of retrying bad payload

When it happens

Trigger: Calling from_internal_dict('encoded-snapshot'), from_internal_dict(None), or passing a JSON string that was never json.loads()-ed, or double-encoded JSON.

Common situations: Forgetting to deserialize a JSON payload before restore; passing raw bytes or a queue message body; double-serialization when a worker framework auto-encodes dicts to strings.

Related errors

• unsupported rollout worker schema version
• unknown rollout channel {value!r}
• rollout worker policy version mismatch
• invalid rollout worker unsafe override
• invalid rollout worker field {field!r}

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/01b7f7fdc9bee191. Report an issue: GitHub.