commaai/openpilot · error · VideoFileInvalid

data is too short

Error message

data is too short

What it means

Raised at the top of hevc_index() when the file is smaller than NAL_UNIT_START_CODE_SIZE + 1 bytes (5 bytes), i.e. too short to hold even one start code plus a NAL header. It is a fast-fail sanity check before any parsing begins, meaning the file is empty or near-empty rather than mis-parsed.

Solutions

1. Check the file size before indexing and skip empty segments in batch tooling.
2. If the segment should have video, investigate why encoderd/loggerd produced no frames for that segment (check logs in the same route directory).
3. Re-download the segment from the cloud if the local copy is a truncated download.
4. Delete the empty file and any duplicate numbered segment so downstream tooling does not trip on it.

Example fix

# before
idx = hevc_index(f'{route_dir}/{seg}/camera.hevc')

# after
from pathlib import Path
p = Path(f'{route_dir}/{seg}/camera.hevc')
if p.stat().st_size < 1024:
    continue  # skip empty/no-video segments
idx = hevc_index(str(p))

import os
from openpilot.tools.lib.vidindex import hevc_index, NAL_UNIT_START_CODE_SIZE

if os.path.getsize(path) >= NAL_UNIT_START_CODE_SIZE + 1:
    frames, prefix, prefix_dat = hevc_index(path)
else:
    skip_segment(path)  # empty/no-video segment

try:
    frames, prefix, prefix_dat = hevc_index(path)
except VideoFileInvalid as e:
    if 'too short' in str(e):
        continue  # empty segment, expected in partially-logged routes
    raise

When it happens

Trigger: Calling hevc_index() on a zero-byte or <5-byte file — typically an empty camera.ts / camera.hevc created by loggerd when a segment started but no encoder frames were written, or a failed/aborted download that created the file then wrote nothing.

Common situations: Empty segment files from a crashed or rebooted loggerd, routes where the camera stream never initialized (encoderd failure), or placeholder files created by a partial `scp`/download that was interrupted before any data moved.

Related errors

• first byte must be 0x00
• data must begin with start code
• data to short to contain nal unit header
• slice_type must be 0, 1, or 2
• Custom series returned an empty result

AI-assisted analysis of commaai/openpilot@516ec1e682 (2026-08-15). Data as JSON: /api/errors/a882c10fb395a7df. Report an issue: GitHub.

Appendix: source

Thrown at openpilot/tools/lib/vidindex.py:266

  # Table 7-7 - Name association to slice_type
  # slice_type | Name of slice_type
  #     0      | B (B slice)
  #     1      | P (P slice)
  #     2      | I (I slice)
  # unsigned integer 0-th order Exp-Golomb-coded syntax element with the left bit first
  slice_type, _ = get_ue(dat, rbsp_start, skip_bits)
  if DEBUG:
    print("  slice_type:", slice_type, f"(first slice: {is_first_slice})")
  if slice_type > 2:
    raise VideoFileInvalid("slice_type must be 0, 1, or 2")
  return slice_type, is_first_slice

def hevc_index(hevc_file_name: str, allow_corrupt: bool=False) -> tuple[list, int, bytes]:
  with FileReader(hevc_file_name) as f:
    dat = f.read()

  if len(dat) < NAL_UNIT_START_CODE_SIZE + 1:
    raise VideoFileInvalid("data is too short")

  if dat[0] != 0x00:
    raise VideoFileInvalid("first byte must be 0x00")

  prefix_dat = b""
  frame_types = []

  i = 1 # skip past first byte 0x00
  try:
    while i < len(dat):
      require_nal_unit_start(dat, i)
      nal_unit_len = get_hevc_nal_unit_length(dat, i)
      nal_unit_type = get_hevc_nal_unit_type(dat, i)
      if nal_unit_type in HEVC_PARAMETER_SET_NAL_UNITS:
        prefix_dat += dat[i:i+nal_unit_len]
      elif nal_unit_type in HEVC_CODED_SLICE_SEGMENT_NAL_UNITS:
        slice_type, is_first_slice = get_hevc_slice_type(dat, i, nal_unit_type)
        if is_first_slice:

View on GitHub (pinned to 516ec1e682)