commaai/openpilot · error · VideoFileInvalid

data is too short

Error message

data is too short

What it means

Raised at the top of hevc_index() when the file is smaller than NAL_UNIT_START_CODE_SIZE + 1 bytes (5 bytes), i.e. too short to hold even one start code plus a NAL header. It is a fast-fail sanity check before any parsing begins, meaning the file is empty or near-empty rather than mis-parsed.

Source

Thrown at openpilot/tools/lib/vidindex.py:266

  # Table 7-7 - Name association to slice_type
  # slice_type | Name of slice_type
  #     0      | B (B slice)
  #     1      | P (P slice)
  #     2      | I (I slice)
  # unsigned integer 0-th order Exp-Golomb-coded syntax element with the left bit first
  slice_type, _ = get_ue(dat, rbsp_start, skip_bits)
  if DEBUG:
    print("  slice_type:", slice_type, f"(first slice: {is_first_slice})")
  if slice_type > 2:
    raise VideoFileInvalid("slice_type must be 0, 1, or 2")
  return slice_type, is_first_slice

def hevc_index(hevc_file_name: str, allow_corrupt: bool=False) -> tuple[list, int, bytes]:
  with FileReader(hevc_file_name) as f:
    dat = f.read()

  if len(dat) < NAL_UNIT_START_CODE_SIZE + 1:
    raise VideoFileInvalid("data is too short")

  if dat[0] != 0x00:
    raise VideoFileInvalid("first byte must be 0x00")

  prefix_dat = b""
  frame_types = []

  i = 1 # skip past first byte 0x00
  try:
    while i < len(dat):
      require_nal_unit_start(dat, i)
      nal_unit_len = get_hevc_nal_unit_length(dat, i)
      nal_unit_type = get_hevc_nal_unit_type(dat, i)
      if nal_unit_type in HEVC_PARAMETER_SET_NAL_UNITS:
        prefix_dat += dat[i:i+nal_unit_len]
      elif nal_unit_type in HEVC_CODED_SLICE_SEGMENT_NAL_UNITS:
        slice_type, is_first_slice = get_hevc_slice_type(dat, i, nal_unit_type)
        if is_first_slice:

View on GitHub (pinned to 516ec1e682)

Solutions

1. Check the file size before indexing and skip empty segments in batch tooling.
2. If the segment should have video, investigate why encoderd/loggerd produced no frames for that segment (check logs in the same route directory).
3. Re-download the segment from the cloud if the local copy is a truncated download.
4. Delete the empty file and any duplicate numbered segment so downstream tooling does not trip on it.

Example fix

# before
idx = hevc_index(f'{route_dir}/{seg}/camera.hevc')

# after
from pathlib import Path
p = Path(f'{route_dir}/{seg}/camera.hevc')
if p.stat().st_size < 1024:
    continue  # skip empty/no-video segments
idx = hevc_index(str(p))

import os
from openpilot.tools.lib.vidindex import hevc_index, NAL_UNIT_START_CODE_SIZE

if os.path.getsize(path) >= NAL_UNIT_START_CODE_SIZE + 1:
    frames, prefix, prefix_dat = hevc_index(path)
else:
    skip_segment(path)  # empty/no-video segment

try:
    frames, prefix, prefix_dat = hevc_index(path)
except VideoFileInvalid as e:
    if 'too short' in str(e):
        continue  # empty segment, expected in partially-logged routes
    raise

When it happens

Trigger: Calling hevc_index() on a zero-byte or <5-byte file — typically an empty camera.ts / camera.hevc created by loggerd when a segment started but no encoder frames were written, or a failed/aborted download that created the file then wrote nothing.

Common situations: Empty segment files from a crashed or rebooted loggerd, routes where the camera stream never initialized (encoderd failure), or placeholder files created by a partial `scp`/download that was interrupted before any data moved.

Related errors

• first byte must be 0x00
• data must begin with start code
• data to short to contain nal unit header
• slice_type must be 0, 1, or 2
• invalid config backup: {backup}

AI-assisted analysis of commaai/openpilot@516ec1e682 (2026-08-15). Data as JSON: /api/errors/a882c10fb395a7df. Report an issue: GitHub.