apache/iceberg · error · UncheckedIOException

Failed reading offset from: %s

Error message

Failed reading offset from: %s

What it means

When resuming or advancing a Spark micro-batch streaming query, SparkMicroBatchStream.readOffset opens the stored offset file and parses it via StreamingOffset.fromJson. Any IOException while reading is rethrown as UncheckedIOException with the offset file location, indicating the stream could not recover its previous progress.

Source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkMicroBatchStream.java:312

    }

    private void writeOffset(StreamingOffset offset, OutputFile file) {
      try (OutputStream outputStream = file.create()) {
        BufferedWriter writer =
            new BufferedWriter(new OutputStreamWriter(outputStream, StandardCharsets.UTF_8));
        writer.write(offset.json());
        writer.flush();
      } catch (IOException ioException) {
        throw new UncheckedIOException(
            String.format("Failed writing offset to: %s", initialOffsetLocation), ioException);
      }
    }

    private StreamingOffset readOffset(InputFile file) {
      try (InputStream in = file.newStream()) {
        return StreamingOffset.fromJson(in);
      } catch (IOException ioException) {
        throw new UncheckedIOException(
            String.format("Failed reading offset from: %s", initialOffsetLocation), ioException);
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Restore the offset file at initialOffsetLocation from backup, or reset the streaming checkpoint and restart the query with a fresh initial offset
2. Fix the underlying storage access error surfaced in the chained IOException (permissions, credentials, connectivity)
3. Recreate the offset file contents (StreamingOffset JSON) if the snapshot/version it pointed to is still valid; otherwise use Spark's checkpoint recovery

if (!file.exists()) {
  throw new IllegalStateException("Offset file missing: " + file.location());
}

try {
  StreamingOffset offset = readOffset(file);
} catch (UncheckedIOException e) {
  LOG.error("Cannot read offset; resetting checkpoint", e.getCause());
  // fall back to initial offset or restore from backup
}

When it happens

Trigger: Calling readOffset on a StreamingOffsetFileIO.InputFile whose stream cannot be opened or read — missing offset file, truncated/corrupt file, permission or network errors from the underlying FileIO.

Common situations: Checkpoint/offset files lost after table location changes or manual cleanup; partial writes from a previous crash; expired cloud credentials; offset location moved between runs.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed writing offset to: %s
• Table refresh failed
• Failed writing offset to: ${initialOffsetLocation}
• Failed reading offset from: ${initialOffsetLocation}
• Cannot load current offset at snapshot %d, the snapshot was

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/a443f01872470f91. Report an issue: GitHub.