{"record":{"id":"e31f9a83df256e2d","repo":"commaai/openpilot","slug":"unstable-flash-read-at-0x-addr-05x","errorCode":null,"errorMessage":"unstable flash read at 0x{addr:05x}","messagePattern":"unstable flash read at 0x(.+?)","errorType":"console","errorClass":"RuntimeError","httpStatus":null,"severity":"warning","filePath":"openpilot/system/hardware/chestnut/flash.py","lineNumber":303,"sourceCode":"\ndef with_retries(flash, label, operation):\n  # on any transfer error, reconnect and restart the operation\n  attempt = 0\n  while True:\n    attempt += 1\n    try:\n      return operation()\n    except (OSError, TimeoutError, RuntimeError) as e:\n      check_budget()\n      print(f\"{label} attempt {attempt}: {e}\", flush=True)\n      reconnect(flash)\n\n\ndef stable_read(flash, addr, length, count=2):\n  def read():\n    reads = [flash.read(addr, length) for _ in range(count)]\n    if any(x != reads[0] for x in reads[1:]):\n      raise RuntimeError(f\"unstable flash read at 0x{addr:05x}\")\n    return reads[0]\n  return with_retries(flash, f\"read 0x{addr:05x}\", read)\n\n\ndef program_sector(flash, addr, target):\n  def program():\n    flash.erase_sector(addr)\n    if flash.read(addr, SECTOR) != bytes([0xFF]) * SECTOR:\n      raise RuntimeError(\"sector erase verification failed\")\n    for off in range(0, SECTOR, PAGE):\n      chunk = target[off:off + PAGE]\n      if chunk != bytes([0xFF]) * len(chunk):\n        flash.program(addr + off, chunk)\n        if flash.read(addr + off, len(chunk)) != chunk:\n          raise RuntimeError(f\"page verify failed at 0x{addr + off:05x}\")\n    if flash.read(addr, SECTOR) != target:\n      raise RuntimeError(\"sector verification failed\")\n  with_retries(flash, f\"sector 0x{addr:05x}\", program)","sourceCodeStart":285,"sourceCodeEnd":321,"githubUrl":"https://github.com/commaai/openpilot/blob/516ec1e68203439a73f340f1d0b3b91eabc626ee/openpilot/system/hardware/chestnut/flash.py#L285-L321","documentation":"RuntimeError raised inside stable_read(): the same address range was read count times (default 2) back-to-back and the results differed. SPI NOR reads are deterministic, so disagreement means the read path is unreliable - flaky USB link, a controller mid-reset, or a device that re-enumerated between reads. It is raised deliberately so with_retries catches it, reconnects, and retries the whole read set.","triggerScenarios":"stable_read(flash, addr, length) during read-back verification of the config (0x100 region) or firmware regions: two consecutive flash.read() calls return different bytes. Typical when the EP0 control-transfer link corrupts data or the device glitches - marginal USB-C connection, runtime PM transitions, or another process touching the device.","commonSituations":"Cheap or flaky USB-C cable or dock; EMI-heavy environment; the enclosure's port losing signal integrity; reads racing a device reset; symptoms are intermittent and move between addresses.","solutions":["Let with_retries do its job first - a single unstable read recovers via reconnect; only persistent instability matters","Harden the link: shorter/better USB-C cable, direct host-controller port, disable runtime PM for the device","Increase the read count (count=3) to distinguish one-off glitches from persistent corruption","If unstable reads persist across reconnects on every sector, stop flashing and fix the physical layer - continuing risks programming garbage and bricking into ROM mode"],"exampleFix":"# before\ndata = flash.read(addr, length)  # unverified\n\n# after\ndata = stable_read(flash, addr, length, count=2)","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"# stable_read already wraps this in with_retries; only handle the terminal case\ntry:\n    data = stable_read(flash, addr, length)\nexcept (TimeoutError, RuntimeError):\n    abort_session('bus unstable - fix physical layer')","preventionTips":["Always read critical regions through stable_read, never bare flash.read","Keep cables short and seated; avoid hubs during flashing","Treat any unstable read as a warning about the whole session, not one address"],"tags":["spi","data-integrity","usb","retry","read-verify"],"backgroundTag":null,"analyzedSha":"516ec1e68203439a73f340f1d0b3b91eabc626ee","analyzedAt":"2026-08-15T00:17:37.461Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}