commaai/openpilot · error · TimeoutError

SPI flash WIP timeout

Error message

SPI flash WIP timeout

What it means

TimeoutError raised by Flash.wait_write_done() (default 10.0s): after a page program (0x02), sector erase (0x20), or write-status transaction, it polls the flash status register (command 0x05) and the Write-In-Progress bit (bit 0) never cleared within 10 seconds. Real SPI NOR erase/program takes milliseconds to a few hundred ms; 10s of continuous WIP means the flash is not acking writes or the status read is returning garbage.

Source

Thrown at openpilot/system/hardware/chestnut/flash.py:213

    for _ in range(4):
      self.reg_write(0xC8AD, 0)

  def write_enable(self):
    for reg, value in ((0xC8AD, 0), (0xC8AA, 0x06), (0xC8AC, 0x04), (0xC8A3, 0), (0xC8A4, 0), (0xC8A9, 1)):
      self.reg_write(reg, value)
    self.wait_controller()

  def status(self):
    self.transaction(0x05, length=1, addr_len=0x04)
    return self.reg_read(0x7000)[0]

  def wait_write_done(self, timeout=10.0):
    deadline = time.monotonic() + timeout
    while time.monotonic() < deadline:
      if not self.status() & 1:
        return
      time.sleep(0.005)
    raise TimeoutError("SPI flash WIP timeout")

  def init(self):
    self.reg_write(0xCC33, 0x04)
    self.reg_write(0xCA81, self.reg_read(0xCA81)[0] | 1)
    self.reg_write(0xC805, 0x02)
    self.reg_write(0xC8A6, 0x04)
    for _ in range(5):
      self.write_enable()
      self.write_buffer(bytes(4))
      self.transaction(0x01, length=1, addr_len=0x04, mode=1)
      time.sleep(0.01)
      if not self.status() & 0x1C:
        return
    raise RuntimeError("could not clear SPI block protection")

  def read(self, addr, length):
    out = bytearray()
    while len(out) < length:

View on GitHub (pinned to 516ec1e682)

Solutions

1. Ensure the reconnect path always re-runs flash.init() before any erase/program (the tool's reconnect() does this - use it, not bare connect())
2. Wrap erase/program in with_retries so a WIP timeout triggers reconnection and a fresh attempt
3. Check the status register for block-protection bits (status() & 0x1C) and clear them via init()'s write-status loop
4. If deterministic on one unit, suspect the SPI flash chip hardware and replace the enclosure

Example fix

# before
flash.erase_sector(addr); flash.wait_write_done()

# after: retry with reconnect on stall
with_retries(flash, f'sector 0x{addr:05x}', lambda: flash.erase_sector(addr))

def flash_ready(flash) -> bool:
    try:
        return (flash.status() & 0x1C) == 0  # protection clear implies writes can complete
    except Exception:
        return False

try:
    flash.wait_write_done()
except TimeoutError as e:
    if 'WIP' in str(e):
        reconnect(flash)  # re-inits controller
        retry_write()

When it happens

Trigger: wait_write_done() after program()/erase_sector()/init()'s write-status attempt. Happens when the device disconnected so reg_read returns stale data, when the SPI controller needs re-init (init() not called after reconnect), or when writes are silently rejected and the poll reads a corrupt status.

Common situations: Recovering after connect() without flash.init() so the controller is in the wrong mode; flash chip with persistent block protection (see error 65); marginal USB link causing corrupt status reads; end-of-life flash chip that programs/erases extremely slowly.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• flash controller timeout
• flash did not converge within {FLASH_BUDGET:g}s
• could not clear SPI block protection
• AT command timed out
• expected one chestnut, found {len(found)}

AI-assisted analysis of commaai/openpilot@516ec1e682 (2026-08-15). Data as JSON: /api/errors/c15858a8de705318. Report an issue: GitHub.