{"record":{"id":"4408d8b7bd51210e","repo":"stamparm/maltrail","slug":"not-a-maltrail-provenance-sidecar-bad-magic","errorCode":null,"errorMessage":"not a Maltrail provenance sidecar (bad magic)","messagePattern":"not a Maltrail provenance sidecar \\(bad magic\\)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"core/provenance.py","lineNumber":83,"sourceCode":"    os.replace(tmp, path)\n    return len(rows), len(pairs)\n\n\nclass Provenance(object):\n    \"\"\"An opened sidecar. Read-only, mmap'd, safe to share between request threads.\"\"\"\n\n    def __init__(self, path):\n        self._file = open(path, \"rb\")\n        try:\n            self._map = mmap.mmap(self._file.fileno(), 0, access=mmap.ACCESS_READ)\n        except Exception:\n            self._file.close()\n            raise\n\n        magic, count, table_len = _HEADER.unpack_from(self._map, 0)\n        if magic != _MAGIC:\n            self.close()\n            raise ValueError(\"not a Maltrail provenance sidecar (bad magic)\")\n\n        self.count = count\n        self._pairs = json.loads(self._map[_HEADER.size:_HEADER.size + table_len].decode(\"utf8\"))\n        self._base = _HEADER.size + table_len\n\n        if self._base + count * _ENTRY_SIZE > len(self._map):\n            self.close()\n            raise ValueError(\"provenance sidecar is truncated\")\n\n    def _hash_at(self, i):\n        offset = self._base + i * _ENTRY_SIZE\n        return struct.unpack_from(\"<Q\", self._map, offset)[0]\n\n    def lookup(self, trail):\n        \"\"\"(reference, source_path) for `trail`, or None.\n\n        The order matches what core/httpd.py's on-demand scan returned, so the caller does not care\n        which of the two answered.","sourceCodeStart":65,"sourceCodeEnd":101,"githubUrl":"https://github.com/stamparm/maltrail/blob/77cfb06d7606506d101bbcec0786c77166c4255e/core/provenance.py#L65-L101","documentation":"The provenance sidecar file's header magic does not match _MAGIC, so __init__ refuses to treat it as a Maltrail provenance sidecar. The file exists and mmaps fine, but is not the expected binary format (wrong file, old/corrupt format, or version mismatch).","triggerScenarios":"Opening a non-sidecar file (log, tarball, different bin format) via the sidecar reader; a sidecar written by an incompatible older/newer version with a changed magic.","commonSituations":"Wrong path in config pointing at trails.bin instead of its provenance sidecar; partially written/zero-length placeholder file; sidecar regenerated by a different tool version.","solutions":["Point the reader at the correct provenance sidecar file, not the trail bin","Regenerate the sidecar with the current library version","Check the file was fully transferred (compare size/checksum) and is not a placeholder","Verify no earlier writer produced the file with a different format"],"exampleFix":"// before\nprov = ProvenanceSidecar(\"trails.bin\")\n// after\nprov = ProvenanceSidecar(\"trails.bin.prov\")","handlingStrategy":"try-catch","validationCode":"def sidecar_looks_valid(path):\n    import struct, os\n    with open(path, \"rb\") as f:\n        magic = f.read(8)  # size of _MAGIC as packed\n    return magic == expected_magic_bytes","typeGuard":"def is_probable_sidecar(path):\n    import os\n    return os.path.isfile(path) and os.path.getsize(path) > 16","tryCatchPattern":"try:\n    prov = ProvenanceSidecar(path)\nexcept ValueError as e:\n    if \"bad magic\" in str(e):\n        prov = regenerate_sidecar(path)","preventionTips":["Keep sidecar and trail bin filenames clearly distinct","Regenerate sidecars with the same library version that reads them","Checksum sidecars after transfer and before use","Never hand-edit binary sidecars"],"tags":["python","file-format","binary-header"],"backgroundTag":"checksum-mismatch","analyzedSha":"77cfb06d7606506d101bbcec0786c77166c4255e","analyzedAt":"2026-09-13T03:50:16.010Z","contentChangedAt":"2026-09-13T03:50:16.010Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}