{"record":{"id":"5a7922f493c86a2e","repo":"headroomlabs-ai/headroom","slug":"rollout-worker-snapshot-digest-mismatch","errorCode":null,"errorMessage":"rollout worker snapshot digest mismatch","messagePattern":"rollout worker snapshot digest mismatch","errorType":"exception","errorClass":"RolloutConfigurationError","httpStatus":null,"severity":"error","filePath":"headroom/rollout.py","lineNumber":319,"sourceCode":"                raw = value[field]\n                if not isinstance(raw, list) or not all(isinstance(item, str) for item in raw):\n                    raise RolloutConfigurationError(f\"invalid rollout worker field {field!r}\")\n                return set(_validate_names(set(raw), source=field, strict=True))\n\n            snapshot = _resolve_snapshot(\n                channel=channel,\n                explicit_requested=names(\"explicit_requested\"),\n                explicit_disabled=names(\"explicit_disabled\"),\n                legacy_requested=names(\"legacy_requested\"),\n                legacy_disabled=names(\"legacy_disabled\"),\n                unsafe=unsafe,\n            )\n        except (KeyError, TypeError) as exc:\n            raise RolloutConfigurationError(\"invalid rollout worker snapshot\") from exc\n        if value.get(\"registry_digest\") != snapshot.registry_digest:\n            raise RolloutConfigurationError(\"rollout worker registry digest mismatch\")\n        if value.get(\"snapshot_digest\") != snapshot.snapshot_digest:\n            raise RolloutConfigurationError(\"rollout worker snapshot digest mismatch\")\n        return snapshot\n\n    def with_legacy_env(self, environ: Mapping[str, str]) -> RolloutSnapshot:\n        \"\"\"Return a new snapshot after applying supplied legacy alias values.\n\n        This intentionally supports existing hot-reloadable aliases without\n        re-reading ambient process state or weakening named disable precedence.\n        Both the old and new snapshots remain immutable, so requests observe a\n        complete policy rather than partially updated fields.\n        \"\"\"\n\n        legacy_requested = set(self.config.legacy_requested)\n        legacy_disabled = set(self.config.legacy_disabled)\n        for spec in FEATURES.values():\n            for alias in spec.legacy_env:\n                if alias not in environ:\n                    continue\n                legacy_requested.discard(spec.name)","sourceCodeStart":301,"sourceCodeEnd":337,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/rollout.py#L301-L337","documentation":"Raised by from_internal_dict() when the restored dict's 'snapshot_digest' does not match the digest computed over the fully resolved snapshot. It is the final integrity check (after registry_digest) and catches any mismatch between the serialized snapshot state and what the resolver reconstructs from it — including effects of policy/resolution changes between versions.","triggerScenarios":"Restoring a snapshot serialized by a different headroom build whose resolution rules differ, or a payload where any snapshot-affecting field was altered without updating 'snapshot_digest'.","commonSituations":"Rolling upgrades with proxy/worker version skew; replayed stale queue messages after a deploy; hand-edited fixtures where one digest was updated but not the other.","solutions":["Pin the same headroom version on both sides of the handoff and re-emit snapshots after upgrading.","Regenerate the payload with to_internal_dict() instead of modifying fields in place.","Drop and rebuild stale in-flight snapshots after any rollout-policy upgrade rather than replaying them."],"exampleFix":"# before\nrestore(old_queued_payload)  # serialized by previous version\n\n# after\n# after aligning versions, re-serialize and re-handoff\nrestore(current_snapshot.to_internal_dict())","handlingStrategy":"validation","validationCode":"# No meaningful caller-side pre-check: digest is recomputed by the library.\n# Guard by version alignment instead:\nfrom importlib.metadata import version\nassert version('headroom') == expected_headroom_version, 'proxy/worker version skew'","typeGuard":null,"tryCatchPattern":"try:\n    snapshot = RolloutSnapshot.from_internal_dict(payload)\nexcept RolloutConfigurationError as e:\n    if 'snapshot digest mismatch' in str(e):\n        snapshot = resolve_fresh_snapshot()  # rebuild locally, drop stale handoff\n    else:\n        raise","preventionTips":["Align headroom versions across all snapshot producers and consumers before deploy.","Rebuild snapshots from local config on digest errors rather than retrying the payload.","Use digest mismatches as a canary signal for incomplete rolling upgrades."],"tags":["integrity","rollout","digest","worker-handoff"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}