{"record":{"id":"f6799fa984a0e659","repo":"commaai/openpilot","slug":"data-must-begin-with-start-code","errorCode":null,"errorMessage":"data must begin with start code","messagePattern":"data must begin with start code","errorType":"exception","errorClass":"VideoFileInvalid","httpStatus":null,"severity":"error","filePath":"openpilot/tools/lib/vidindex.py","lineNumber":155,"sourceCode":"      else:\n        suffix_val = (suffix_val << 1) | ((dat[i] >> j) & 1)\n        suffix_len += 1\n      j -= 1\n\n      if prefix_val == 1 and prefix_len - 1 == suffix_len:\n        val = int(2**(prefix_len-1) - 1 + suffix_val)\n        size = prefix_len + suffix_len\n        return val, size\n    i += 1\n\n  raise VideoFileInvalid(\"invalid exponential-golomb code\")\n\ndef require_nal_unit_start(dat: bytes, nal_unit_start: int) -> None:\n  if nal_unit_start < 1:\n    raise ValueError(\"start index must be greater than zero\")\n\n  if dat[nal_unit_start:nal_unit_start + NAL_UNIT_START_CODE_SIZE] != NAL_UNIT_START_CODE:\n    raise VideoFileInvalid(\"data must begin with start code\")\n\ndef get_hevc_nal_unit_length(dat: bytes, nal_unit_start: int) -> int:\n  try:\n    pos = dat.index(NAL_UNIT_START_CODE, nal_unit_start + NAL_UNIT_START_CODE_SIZE)\n  except ValueError:\n    pos = -1\n\n  # length of NAL unit is byte count up to next NAL unit start index\n  nal_unit_len = (pos if pos != -1 else len(dat)) - nal_unit_start\n  if DEBUG:\n    print(\"  nal_unit_len:\", nal_unit_len)\n  return nal_unit_len\n\ndef get_hevc_nal_unit_type(dat: bytes, nal_unit_start: int) -> HevcNalUnitType:\n  # 7.3.1.2 NAL unit header syntax\n  # nal_unit_header( ) {    // descriptor\n  #   forbidden_zero_bit    f(1)\n  #   nal_unit_type         u(6)","sourceCodeStart":137,"sourceCodeEnd":173,"githubUrl":"https://github.com/commaai/openpilot/blob/516ec1e68203439a73f340f1d0b3b91eabc626ee/openpilot/tools/lib/vidindex.py#L137-L173","documentation":"Thrown by require_nal_unit_start() in openpilot's HEVC video indexing tool when the bytes at the expected NAL unit position do not equal the 4-byte Annex-B start code (00 00 00 01). The indexer (hevc_index) walks the file assuming each NAL unit begins exactly where the previous one ended, so any mismatch means the byte stream is not a sequence of well-formed Annex-B NAL units. It is raised as VideoFileInvalid, marking the input as corrupt or not raw HEVC.","triggerScenarios":"Calling hevc_index() (or require_nal_unit_start() directly) on data where dat[i:i+4] != b'\\x00\\x00\\x00\\x01'. Typical when the file is not raw HEVC/Annex-B (e.g. it is MP4, MKV, or fragmented), when a NAL length was miscomputed after a corrupt unit, or when passing a start index of trailing garbage. Also raised if nal_unit_start < 1 (ValueError variant: 'start index must be greater than zero').","commonSituations":"Indexing a camera route file that was truncated mid-write (loggerd crash), feeding an fMP4/AVCC-encapsulated stream without stripping length prefixes, or processing qcamera/resampled streams with emulation prevention edge cases that desync the scanner.","solutions":["Verify the input is raw Annex-B HEVC (.hevc stream extracted from a route), not an MP4 container — remux with ffmpeg -c copy -bsf:v hevc_mp4toannexb if needed.","If the file may be truncated/corrupt, call hevc_index with allow_corrupt=True so partial indexing is attempted instead of failing.","Check file integrity: compare size and sha against the route manifest; re-download the segment from the cloud if it came from a partially uploaded route.","Reproduce with a known-good route file to rule out a parsing bug, then report the file to openpilot devs if only that file fails."],"exampleFix":"// before\nindex = hevc_index('route/segment/camera.hevc')\n\n// after\nindex = hevc_index('route/segment/camera.hevc', allow_corrupt=True)  // tolerate truncated tail","handlingStrategy":"validation","validationCode":"def is_annexb_hevc(path, min_size=6):\n    with open(path, 'rb') as f:\n        head = f.read(4)\n        f.seek(-4, 2)\n    import os\n    if os.path.getsize(path) < min_size:\n        return False\n    return head == b'\\x00\\x00\\x00\\x01'","typeGuard":null,"tryCatchPattern":"from openpilot.tools.lib.vidindex import hevc_index, VideoFileInvalid\ntry:\n    frames, prefix, prefix_dat = hevc_index(path)\nexcept VideoFileInvalid as e:\n    if 'start code' in str(e):\n        frames, prefix, prefix_dat = hevc_index(path, allow_corrupt=True)\n    else:\n        raise","preventionTips":["Always extract the raw Annex-B .hevc stream from routes instead of passing container files.","Pre-check the first 4 bytes are 00 00 00 01 before calling hevc_index.","Use allow_corrupt=True when batch-processing older or partially uploaded routes."],"tags":["video","hevc","parsing","corruption","openpilot"],"backgroundTag":null,"analyzedSha":"516ec1e68203439a73f340f1d0b3b91eabc626ee","analyzedAt":"2026-08-15T00:17:37.461Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}