headroomlabs-ai/headroom · error · RolloutConfigurationError

rollout worker snapshot digest mismatch

Error message

rollout worker snapshot digest mismatch

What it means

Raised by from_internal_dict() when the restored dict's 'snapshot_digest' does not match the digest computed over the fully resolved snapshot. It is the final integrity check (after registry_digest) and catches any mismatch between the serialized snapshot state and what the resolver reconstructs from it — including effects of policy/resolution changes between versions.

Source

Thrown at headroom/rollout.py:319

                raw = value[field]
                if not isinstance(raw, list) or not all(isinstance(item, str) for item in raw):
                    raise RolloutConfigurationError(f"invalid rollout worker field {field!r}")
                return set(_validate_names(set(raw), source=field, strict=True))

            snapshot = _resolve_snapshot(
                channel=channel,
                explicit_requested=names("explicit_requested"),
                explicit_disabled=names("explicit_disabled"),
                legacy_requested=names("legacy_requested"),
                legacy_disabled=names("legacy_disabled"),
                unsafe=unsafe,
            )
        except (KeyError, TypeError) as exc:
            raise RolloutConfigurationError("invalid rollout worker snapshot") from exc
        if value.get("registry_digest") != snapshot.registry_digest:
            raise RolloutConfigurationError("rollout worker registry digest mismatch")
        if value.get("snapshot_digest") != snapshot.snapshot_digest:
            raise RolloutConfigurationError("rollout worker snapshot digest mismatch")
        return snapshot

    def with_legacy_env(self, environ: Mapping[str, str]) -> RolloutSnapshot:
        """Return a new snapshot after applying supplied legacy alias values.

        This intentionally supports existing hot-reloadable aliases without
        re-reading ambient process state or weakening named disable precedence.
        Both the old and new snapshots remain immutable, so requests observe a
        complete policy rather than partially updated fields.
        """

        legacy_requested = set(self.config.legacy_requested)
        legacy_disabled = set(self.config.legacy_disabled)
        for spec in FEATURES.values():
            for alias in spec.legacy_env:
                if alias not in environ:
                    continue
                legacy_requested.discard(spec.name)

View on GitHub (pinned to 322425c43b)

Solutions

1. Pin the same headroom version on both sides of the handoff and re-emit snapshots after upgrading.
2. Regenerate the payload with to_internal_dict() instead of modifying fields in place.
3. Drop and rebuild stale in-flight snapshots after any rollout-policy upgrade rather than replaying them.

Example fix

# before
restore(old_queued_payload)  # serialized by previous version

# after
# after aligning versions, re-serialize and re-handoff
restore(current_snapshot.to_internal_dict())

# No meaningful caller-side pre-check: digest is recomputed by the library.
# Guard by version alignment instead:
from importlib.metadata import version
assert version('headroom') == expected_headroom_version, 'proxy/worker version skew'

try:
    snapshot = RolloutSnapshot.from_internal_dict(payload)
except RolloutConfigurationError as e:
    if 'snapshot digest mismatch' in str(e):
        snapshot = resolve_fresh_snapshot()  # rebuild locally, drop stale handoff
    else:
        raise

When it happens

Trigger: Restoring a snapshot serialized by a different headroom build whose resolution rules differ, or a payload where any snapshot-affecting field was altered without updating 'snapshot_digest'.

Common situations: Rolling upgrades with proxy/worker version skew; replayed stale queue messages after a deploy; hand-edited fixtures where one digest was updated but not the other.

Related errors

• rollout worker registry digest mismatch
• invalid rollout worker snapshot
• unsupported rollout worker schema version
• rollout worker policy version mismatch
• sha256 mismatch for {path.name}: expected {expected}, got {g

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/5a7922f493c86a2e. Report an issue: GitHub.