{"record":{"id":"a882c10fb395a7df","repo":"commaai/openpilot","slug":"data-is-too-short","errorCode":null,"errorMessage":"data is too short","messagePattern":"data is too short","errorType":"exception","errorClass":"VideoFileInvalid","httpStatus":null,"severity":"error","filePath":"openpilot/tools/lib/vidindex.py","lineNumber":266,"sourceCode":"  # Table 7-7 - Name association to slice_type\n  # slice_type | Name of slice_type\n  #     0      | B (B slice)\n  #     1      | P (P slice)\n  #     2      | I (I slice)\n  # unsigned integer 0-th order Exp-Golomb-coded syntax element with the left bit first\n  slice_type, _ = get_ue(dat, rbsp_start, skip_bits)\n  if DEBUG:\n    print(\"  slice_type:\", slice_type, f\"(first slice: {is_first_slice})\")\n  if slice_type > 2:\n    raise VideoFileInvalid(\"slice_type must be 0, 1, or 2\")\n  return slice_type, is_first_slice\n\ndef hevc_index(hevc_file_name: str, allow_corrupt: bool=False) -> tuple[list, int, bytes]:\n  with FileReader(hevc_file_name) as f:\n    dat = f.read()\n\n  if len(dat) < NAL_UNIT_START_CODE_SIZE + 1:\n    raise VideoFileInvalid(\"data is too short\")\n\n  if dat[0] != 0x00:\n    raise VideoFileInvalid(\"first byte must be 0x00\")\n\n  prefix_dat = b\"\"\n  frame_types = []\n\n  i = 1 # skip past first byte 0x00\n  try:\n    while i < len(dat):\n      require_nal_unit_start(dat, i)\n      nal_unit_len = get_hevc_nal_unit_length(dat, i)\n      nal_unit_type = get_hevc_nal_unit_type(dat, i)\n      if nal_unit_type in HEVC_PARAMETER_SET_NAL_UNITS:\n        prefix_dat += dat[i:i+nal_unit_len]\n      elif nal_unit_type in HEVC_CODED_SLICE_SEGMENT_NAL_UNITS:\n        slice_type, is_first_slice = get_hevc_slice_type(dat, i, nal_unit_type)\n        if is_first_slice:","sourceCodeStart":248,"sourceCodeEnd":284,"githubUrl":"https://github.com/commaai/openpilot/blob/516ec1e68203439a73f340f1d0b3b91eabc626ee/openpilot/tools/lib/vidindex.py#L248-L284","documentation":"Raised at the top of hevc_index() when the file is smaller than NAL_UNIT_START_CODE_SIZE + 1 bytes (5 bytes), i.e. too short to hold even one start code plus a NAL header. It is a fast-fail sanity check before any parsing begins, meaning the file is empty or near-empty rather than mis-parsed.","triggerScenarios":"Calling hevc_index() on a zero-byte or <5-byte file — typically an empty camera.ts / camera.hevc created by loggerd when a segment started but no encoder frames were written, or a failed/aborted download that created the file then wrote nothing.","commonSituations":"Empty segment files from a crashed or rebooted loggerd, routes where the camera stream never initialized (encoderd failure), or placeholder files created by a partial `scp`/download that was interrupted before any data moved.","solutions":["Check the file size before indexing and skip empty segments in batch tooling.","If the segment should have video, investigate why encoderd/loggerd produced no frames for that segment (check logs in the same route directory).","Re-download the segment from the cloud if the local copy is a truncated download.","Delete the empty file and any duplicate numbered segment so downstream tooling does not trip on it."],"exampleFix":"# before\nidx = hevc_index(f'{route_dir}/{seg}/camera.hevc')\n\n# after\nfrom pathlib import Path\np = Path(f'{route_dir}/{seg}/camera.hevc')\nif p.stat().st_size < 1024:\n    continue  # skip empty/no-video segments\nidx = hevc_index(str(p))","handlingStrategy":"validation","validationCode":"import os\nfrom openpilot.tools.lib.vidindex import hevc_index, NAL_UNIT_START_CODE_SIZE\n\nif os.path.getsize(path) >= NAL_UNIT_START_CODE_SIZE + 1:\n    frames, prefix, prefix_dat = hevc_index(path)\nelse:\n    skip_segment(path)  # empty/no-video segment","typeGuard":null,"tryCatchPattern":"try:\n    frames, prefix, prefix_dat = hevc_index(path)\nexcept VideoFileInvalid as e:\n    if 'too short' in str(e):\n        continue  # empty segment, expected in partially-logged routes\n    raise","preventionTips":["Filter segments by minimum file size before batch indexing.","Check encoderd health for routes that produce empty camera files.","Verify downloads completed (compare size to remote) before processing."],"tags":["video","hevc","empty-file","validation","openpilot"],"backgroundTag":null,"analyzedSha":"516ec1e68203439a73f340f1d0b3b91eabc626ee","analyzedAt":"2026-08-15T00:17:37.461Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}