commaai/openpilot · critical · RuntimeError

Failed to open ISD-R after retries

Error message

Failed to open ISD-R after retries

What it means

open_isdr() tries to open a logical channel to the eUICC's ISD-R (ISDR_AID) up to OPEN_ISDR_RETRIES=10 times, sleeping 0.25s between attempts and resetting the modem at attempt 5. If every attempt fails with RuntimeError/TimeoutError/termios.error/SerialException, it gives up with this RuntimeError.

Source

Thrown at openpilot/common/esim/lpa.py:237

  def _reset_modem(self) -> None:
    if self._serial:
      try:
        self._serial.close()
      except Exception:
        pass
      self._serial = None
    subprocess.run(['/usr/comma/lte/lte.sh', 'start'], capture_output=True)

  def open_isdr(self) -> None:
    for attempt in range(OPEN_ISDR_RETRIES):
      try:
        self._open_isdr_once()
        return
      except (RuntimeError, TimeoutError, termios.error, SerialException):
        time.sleep(OPEN_ISDR_RETRY_DELAY_S)
        if attempt == OPEN_ISDR_RESET_ATTEMPT:
          self._reset_modem()
    raise RuntimeError("Failed to open ISD-R after retries")

  def send_apdu(self, apdu: bytes) -> tuple[bytes, int, int]:
    for attempt in range(SEND_APDU_RETRIES):
      try:
        if not self.channel:
          self.open_isdr()
        hex_payload = apdu.hex().upper()
        for line in self.query(f'AT+CGLA={self.channel},{len(hex_payload)},"{hex_payload}"'):
          if line.startswith("+CGLA:"):
            parts = line.split(":", 1)[1].split(",", 1)
            if len(parts) == 2:
              data = bytes.fromhex(parts[1].strip().strip('"'))
              if len(data) >= 2:
                return data[:-2], data[-2], data[-1]
        raise RuntimeError("Missing +CGLA response")
      except (RuntimeError, ValueError):
        self.channel = None
        if attempt == SEND_APDU_RETRIES - 1:

View on GitHub (pinned to 516ec1e682)

Solutions

1. Full power-cycle the device/modem (not just lte.sh start) — many ISD-R open failures need a cold reset
2. Verify the serial device exists and is readable/writable: ls -l /dev/modem_at0; run with DEBUG=1 to see the raw AT exchange
3. Confirm an eUICC is actually present (is_euicc()) before attempting ISD-R operations
4. Check modem firmware/health logs (dmesg, modem manager logs) for a crashed or rebooting modem; reflash/update firmware if it keeps crashing

Example fix

# before
client.open_isdr()  # RuntimeError: Failed to open ISD-R after retries

# after
import subprocess
client._reset_modem()                       # lte.sh restart
subprocess.run(['reboot'], check=False)      # or full device power cycle, then retry
client = AtClient()  # fresh client reconnects cleanly
client.open_isdr()

from openpilot.common.esim.lpa import AtClient

# ensure modem is responsive at all before attempting ISD-R
c = AtClient()
if not c.query('AT'):
  raise SystemExit('modem not answering basic AT; power-cycle first')

try:
  client.open_isdr()
except (RuntimeError, TimeoutError):
  client._reset_modem()
  client = AtClient()  # fresh connection after reset
  client.open_isdr()  # one full retry; escalate to power-cycle if this fails

When it happens

Trigger: Sustained modem failure: serial port unreadable (termios/SerialException), modem repeatedly answering ERROR/timeout to the channel-open AT sequence, or modem absent. Happens during heavy eSIM use or when the modem crashed earlier and lte.sh restart doesn't recover it.

Common situations: Modem firmware crash requiring full power cycle, not just lte.sh restart; /dev/modem_at0 permissions or udev breakage after update; no eUICC so the SELECT of ISDR AID always fails; hardware fault in the cellular module.

Related errors

• AT command timed out
• Missing +CGLA response
• no eUICC detected
• AT command failed: {line}
• send_apdu failed

AI-assisted analysis of commaai/openpilot@516ec1e682 (2026-08-15). Data as JSON: /api/errors/1f8bd3ed38c49c51. Report an issue: GitHub.