commaai/openpilot · error · VideoFileInvalid

data to short to contain nal unit header

Error message

data to short to contain nal unit header

What it means

Raised by get_hevc_nal_unit_type() in vidindex.py when fewer than 2 bytes remain after the NAL start code, so the 2-byte HEVC NAL unit header (forbidden_zero_bit, nal_unit_type, nuh_layer_id, nuh_temporal_id_plus1 per ITU-T H.265 7.3.1.2) cannot be read. A VideoFileInvalid, it almost always means the file ends with a bare start code or the NAL length computation ran past the end of a truncated file.

Source

Thrown at openpilot/tools/lib/vidindex.py:180

  # length of NAL unit is byte count up to next NAL unit start index
  nal_unit_len = (pos if pos != -1 else len(dat)) - nal_unit_start
  if DEBUG:
    print("  nal_unit_len:", nal_unit_len)
  return nal_unit_len

def get_hevc_nal_unit_type(dat: bytes, nal_unit_start: int) -> HevcNalUnitType:
  # 7.3.1.2 NAL unit header syntax
  # nal_unit_header( ) {    // descriptor
  #   forbidden_zero_bit    f(1)
  #   nal_unit_type         u(6)
  #   nuh_layer_id          u(6)
  #   nuh_temporal_id_plus1 u(3)
  # }
  header_start = nal_unit_start + NAL_UNIT_START_CODE_SIZE
  nal_unit_header = dat[header_start:header_start + NAL_UNIT_HEADER_SIZE]
  if len(nal_unit_header) != 2:
    raise VideoFileInvalid("data to short to contain nal unit header")
  nal_unit_type = HevcNalUnitType((nal_unit_header[0] >> 1) & 0x3F)
  if DEBUG:
    print("  nal_unit_type:", nal_unit_type.name, f"({nal_unit_type.value})")
  return nal_unit_type

def get_hevc_slice_type(dat: bytes, nal_unit_start: int, nal_unit_type: HevcNalUnitType) -> tuple[int, bool]:
  # 7.3.2.9 Slice segment layer RBSP syntax
  # slice_segment_layer_rbsp( ) {
  #   slice_segment_header( )
  #   slice_segment_data( )
  #   rbsp_slice_segment_trailing_bits( )
  # }
  # ...
  # 7.3.6.1 General slice segment header syntax
  # slice_segment_header( ) {                                             // descriptor
  #   first_slice_segment_in_pic_flag                                     u(1)
  #   if( nal_unit_type >= BLA_W_LP && nal_unit_type <= RSV_IRAP_VCL23 )
  #     no_output_of_prior_pics_flag                                      u(1)

View on GitHub (pinned to 516ec1e682)

Solutions

1. Pass allow_corrupt=True to hevc_index so the truncated tail is skipped instead of raising.
2. Confirm the file is complete (size matches the uploader log / rlog-derived camera path); re-pull the segment if truncated.
3. Strip trailing zero padding / dangling start codes before indexing if you control the pipeline.
4. As a diagnostic, hexdump the last 32 bytes of the file to confirm it ends with a dangling 00 00 00 01.

Example fix

// before
frames, prefix, prefix_dat = hevc_index(f'{seg}/camera.hevc')

// after
frames, prefix, prefix_dat = hevc_index(f'{seg}/camera.hevc', allow_corrupt=True)

import os
START_CODE = b'\x00\x00\x00\x01'
size = os.path.getsize(path)
with open(path, 'rb') as f:
    f.seek(size - 4)
    tail = f.read()
# file ending with a dangling start code will fail the header read
ends_with_dangling_sc = tail == START_CODE or (size >= 4 and tail.lstrip(b'\x00') == b'')

try:
    frames, prefix, prefix_dat = hevc_index(path)
except VideoFileInvalid as e:
    if 'nal unit header' in str(e):
        frames, prefix, prefix_dat = hevc_index(path, allow_corrupt=True)  # skip truncated tail
    else:
        raise

When it happens

Trigger: hevc_index() encounters a NAL unit at the very end of the buffer where dat[header_start:header_start+2] yields fewer than 2 bytes — i.e. the file ends with 00 00 00 01 and nothing after it, or a truncated final NAL. Also reachable by calling get_hevc_nal_unit_type(dat, i) directly with an i pointing into the last 1-5 bytes.

Common situations: Route files cut off mid-upload, segments whose loggerd write was interrupted, or trailing start-code padding bytes at the end of a stream. Anyone batch-indexing old routes with incomplete final segments hits this.

Related errors

• data must begin with start code
• slice_type must be 0, 1, or 2
• data is too short
• first byte must be 0x00
• invalid exponential-golomb code

AI-assisted analysis of commaai/openpilot@516ec1e682 (2026-08-15). Data as JSON: /api/errors/2057fa681f79504b. Report an issue: GitHub.