{"record":{"id":"993d3f980d4471ea","repo":"commaai/openpilot","slug":"failed-to-index-fn-r","errorCode":null,"errorMessage":"Failed to index {fn!r}","messagePattern":"Failed to index (.+?)","errorType":"exception","errorClass":"DataUnreadableError","httpStatus":null,"severity":"error","filePath":"openpilot/tools/lib/framereader.py","lineNumber":87,"sourceCode":"def ffprobe(fn, fmt=None):\n  fn = resolve_name(fn)\n  cmd = [\"ffprobe\", \"-v\", \"quiet\", \"-print_format\", \"json\", \"-show_format\", \"-show_streams\"]\n  if fmt:\n    cmd += [\"-f\", fmt]\n  cmd += [\"-i\", \"pipe:0\"]\n\n  try:\n    with FileReader(fn) as f:\n      ffprobe_output = subprocess.check_output(cmd, input=f.read(4096))\n  except subprocess.CalledProcessError as e:\n    raise DataUnreadableError(fn) from e\n  return json.loads(ffprobe_output)\n\ndef get_index_data(fn: str, index_data: dict|None = None):\n  if index_data is None:\n    index_data = get_video_index(fn)\n    if index_data is None:\n      raise DataUnreadableError(f\"Failed to index {fn!r}\")\n  stream = index_data[\"probe\"][\"streams\"][0]\n  return index_data[\"index\"], index_data[\"global_prefix\"], stream[\"width\"], stream[\"height\"]\n\ndef get_video_index(fn):\n  assert_hvec(fn)\n  frame_types, dat_len, prefix = hevc_index(fn)\n  index = np.array(frame_types + [(0xFFFFFFFF, dat_len)], dtype=np.uint32)\n  probe = ffprobe(fn, \"hevc\")\n  return {\n    'index': index,\n    'global_prefix': prefix,\n    'probe': probe\n  }\n\nclass FfmpegDecoder:\n  def __init__(self, fn: str, index_data: dict|None = None,\n               pix_fmt: str = \"rgb24\", hwaccel=\"auto\", loglevel=\"quiet\"):\n    self.fn = fn","sourceCodeStart":69,"sourceCodeEnd":105,"githubUrl":"https://github.com/commaai/openpilot/blob/516ec1e68203439a73f340f1d0b3b91eabc626ee/openpilot/tools/lib/framereader.py#L69-L105","documentation":"get_index_data() requires a video index; if get_video_index(fn) returns None (no cached index and indexing produced nothing usable), it raises DataUnreadableError('Failed to index ...'). In practice get_video_index rarely returns None — assert_hvec/ffprobe usually raise earlier — so this is the catch-all when indexing silently yields nothing, e.g. via the cached path.","triggerScenarios":"Calling FrameReader/get_index_data on a file that passes the header check but produces no index — typically an index-cache miss combined with a degenerate stream, or when index_data passed in as None and the underlying file became unreadable between calls.","commonSituations":"Corrupted .idx/cache files produced by a previous partial run; race between concurrent processes writing the video index cache.","solutions":["Delete the framereader index cache for that route and retry so it is rebuilt","Validate the file directly: ffprobe and hevc_index on the same path to see which stage yields nothing","If it reproduces, the camera file itself is bad — re-download or use another segment"],"exampleFix":"# before\nindex, prefix, w, h = get_index_data(fn)\n\n# after\nfrom openpilot.tools.lib.framereader import get_index_data, DataUnreadableError\ntry:\n    index, prefix, w, h = get_index_data(fn)\nexcept DataUnreadableError:\n    cache_clear_for(fn)  # or: os.remove(index_cache_path(fn))\n    index, prefix, w, h = get_index_data(fn)","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"from openpilot.tools.lib.exceptions import DataUnreadableError\ntry:\n    index, prefix, w, h = get_index_data(fn)\nexcept DataUnreadableError:\n    clear_framereader_cache(fn)\n    index, prefix, w, h = get_index_data(fn)  # one rebuild attempt","preventionTips":["Invalidate index caches when re-downloading camera files","Avoid concurrent processes indexing the same route simultaneously","Wrap whole-route processing so one bad segment does not kill a batch"],"tags":["video","index-cache","framereader","corrupt-data"],"backgroundTag":null,"analyzedSha":"516ec1e68203439a73f340f1d0b3b91eabc626ee","analyzedAt":"2026-08-15T00:17:37.461Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}