headroomlabs-ai/headroom · error · RolloutConfigurationError

rollout worker registry digest mismatch

Error message

rollout worker registry digest mismatch

What it means

Raised by from_internal_dict() when the restored dict's 'registry_digest' does not match the digest recomputed from the restored snapshot's feature registry. The digest ties the handoff payload to the exact registry state, so tampering or partial updates to feature names are detected even when all fields individually validate.

Source

Thrown at headroom/rollout.py:317

            def names(field: str) -> set[str]:
                raw = value[field]
                if not isinstance(raw, list) or not all(isinstance(item, str) for item in raw):
                    raise RolloutConfigurationError(f"invalid rollout worker field {field!r}")
                return set(_validate_names(set(raw), source=field, strict=True))

            snapshot = _resolve_snapshot(
                channel=channel,
                explicit_requested=names("explicit_requested"),
                explicit_disabled=names("explicit_disabled"),
                legacy_requested=names("legacy_requested"),
                legacy_disabled=names("legacy_disabled"),
                unsafe=unsafe,
            )
        except (KeyError, TypeError) as exc:
            raise RolloutConfigurationError("invalid rollout worker snapshot") from exc
        if value.get("registry_digest") != snapshot.registry_digest:
            raise RolloutConfigurationError("rollout worker registry digest mismatch")
        if value.get("snapshot_digest") != snapshot.snapshot_digest:
            raise RolloutConfigurationError("rollout worker snapshot digest mismatch")
        return snapshot

    def with_legacy_env(self, environ: Mapping[str, str]) -> RolloutSnapshot:
        """Return a new snapshot after applying supplied legacy alias values.

        This intentionally supports existing hot-reloadable aliases without
        re-reading ambient process state or weakening named disable precedence.
        Both the old and new snapshots remain immutable, so requests observe a
        complete policy rather than partially updated fields.
        """

        legacy_requested = set(self.config.legacy_requested)
        legacy_disabled = set(self.config.legacy_disabled)
        for spec in FEATURES.values():
            for alias in spec.legacy_env:
                if alias not in environ:

View on GitHub (pinned to 322425c43b)

Solutions

1. Never edit serialized snapshots; rebuild them via to_internal_dict() from a live snapshot.
2. If you must change feature sets, apply changes through the snapshot/config API and re-serialize.
3. Check transport integrity if payloads are edited by neither side (queue encoding, truncation).

Example fix

# before
payload = snapshot.to_internal_dict()
payload['explicit_requested'].append('new-feature')  # digest now stale

# after
new_snapshot = snapshot.with_feature_requested('new-feature')
payload = new_snapshot.to_internal_dict()

# There is no safe pre-check for digest equality except not mutating payloads.
# Verify provenance instead:
assert payload_source == 'RolloutSnapshot.to_internal_dict', 'handcrafted payload cannot match digests'

try:
    snapshot = RolloutSnapshot.from_internal_dict(payload)
except RolloutConfigurationError as e:
    if 'registry digest mismatch' in str(e):
        logger.error('payload tampered or corrupted; requesting fresh handoff')
        payload = request_fresh_handoff()
        snapshot = RolloutSnapshot.from_internal_dict(payload)
    else:
        raise

When it happens

Trigger: Restoring a snapshot whose feature-registry fields were edited (a feature name added/removed/renamed) without recomputing 'registry_digest', or a payload truncated/corrupted in transit through the queue.

Common situations: Middleware or ad-hoc scripts mutating handoff dicts; encoding corruption in queues; fixtures edited by hand to add a feature for testing.

Related errors

• rollout worker snapshot digest mismatch
• invalid rollout worker snapshot
• unsupported rollout worker schema version
• rollout worker policy version mismatch
• sha256 mismatch for {path.name}: expected {expected}, got {g

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/07f8320e2e6b98a0. Report an issue: GitHub.