{"record":{"id":"d866f31445647128","repo":"commaai/openpilot","slug":"final-full-image-verification-failed","errorCode":null,"errorMessage":"final full-image verification failed","messagePattern":"final full-image verification failed","errorType":"console","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"openpilot/system/hardware/chestnut/flash.py","lineNumber":553,"sourceCode":"    current = stable_read(flash, first_sector, span - first_sector)\n    target = bytearray(current)\n    target[:len(config)] = config\n    target[IMAGE_OFFSET - first_sector:image_end - first_sector] = image\n    target = bytes(target)\n    print(f\"target {len(image)} bytes at 0x{IMAGE_OFFSET:05x}, sha256={hashlib.sha256(image).hexdigest()}\", flush=True)\n\n    for addr in range(first_sector, span, SECTOR):\n      off = addr - first_sector\n      wanted = target[off:off + SECTOR]\n      if current[off:off + SECTOR] == wanted:\n        print(f\"sector 0x{addr:05x}: unchanged\", flush=True)\n      else:\n        print(f\"sector 0x{addr:05x}: programming\", flush=True)\n        program_sector(flash, addr, wanted)\n\n    verified = stable_read(flash, first_sector, span - first_sector, 3)\n    if verified != target:\n      raise RuntimeError(\"final full-image verification failed\")\n    print(f\"verified sha256={hashlib.sha256(verified).hexdigest()}\", flush=True)\n  finally:\n    flash.close()\n\n  activate(expected_product)\n\n\ndef main():\n  parser = argparse.ArgumentParser(description=\"check and flash the bundled chestnut firmware\")\n  parser.add_argument(\"version\", nargs=\"?\", help=\"expected firmware version hash\")\n  parser.add_argument(\"--force\", action=\"store_true\", help=\"reflash even when the version matches\")\n  args = parser.parse_args()\n  if os.geteuid() != 0:\n    raise RuntimeError(\"flash.py must run as root\")\n  flash_chestnut(expected_version=args.version, force=args.force)\n\n\nif __name__ == \"__main__\":","sourceCodeStart":535,"sourceCodeEnd":571,"githubUrl":"https://github.com/commaai/openpilot/blob/516ec1e68203439a73f340f1d0b3b91eabc626ee/openpilot/system/hardware/chestnut/flash.py#L535-L571","documentation":"After programming all changed sectors, write_image() does a stable_read() (three passes) of the full span and compares it to the target image. If the read-back still differs, the flash verify failed and the device is left unverified. This catches programming failures, flaky USB/serial links, and flash cells that do not hold what was written.","triggerScenarios":"A sector program silently corrupted data (bad flash block, voltage dip, loose cable); stable_read returned inconsistent data across its 3 attempts due to a noisy connection; the image changed on disk between the initial read and verification (rare re-entrancy).","commonSituations":"Marginal USB cable or powered hub during a long flash; failing flash chip on the chestnut; EMI or undervoltage on the device during programming.","solutions":["Check/replace the USB cable and port, power the device from a stable supply, then re-run flash.py --force","Re-run with --force to reprogram: single-bit programming glitches often clear on a second pass","Look at the per-sector 'programming' vs 'unchanged' output above the error to see if failures cluster at specific addresses - recurring failures at the same sector indicate a worn flash block","If verification keeps failing on the same sectors, the device hardware is suspect - escalate to comma support"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"for attempt in range(2):\n    try:\n        write_image(image, expected_product, product, force=True)\n        break\n    except RuntimeError as e:\n        if 'verification failed' in str(e) and attempt == 0:\n            print(\"verify failed once; re-seating connections and retrying\")\n            continue\n        raise","preventionTips":["Use a short, shielded USB cable plugged directly into the host (no hubs) for the entire flash","Watch the per-sector output: recurring 'programming' on the same addresses across runs indicates a bad flash block - stop and escalate instead of looping","Ensure the device power supply is stable; avoid flashing on battery-heavy load setups"],"tags":["firmware","flashing","verification","hardware","openpilot"],"backgroundTag":null,"analyzedSha":"516ec1e68203439a73f340f1d0b3b91eabc626ee","analyzedAt":"2026-08-15T00:17:37.461Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}