apache/iceberg · critical · UncheckedIOException

Failed reading offset from: %s

Error message

Failed reading offset from: %s

What it means

readOffset deserializes the persisted StreamingOffset JSON via StreamingOffset.fromJson from the initial offset location file. Any IOException while opening or reading the stream is wrapped in this UncheckedIOException. It means the previously written offset could not be read back, typically because the file is missing, unreadable, or the storage backend failed mid-read.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkMicroBatchStream.java:310

    }

    private void writeOffset(StreamingOffset offset, OutputFile file) {
      try (OutputStream outputStream = file.create()) {
        BufferedWriter writer =
            new BufferedWriter(new OutputStreamWriter(outputStream, StandardCharsets.UTF_8));
        writer.write(offset.json());
        writer.flush();
      } catch (IOException ioException) {
        throw new UncheckedIOException(
            String.format("Failed writing offset to: %s", initialOffsetLocation), ioException);
      }
    }

    private StreamingOffset readOffset(InputFile file) {
      try (InputStream in = file.newStream()) {
        return StreamingOffset.fromJson(in);
      } catch (IOException ioException) {
        throw new UncheckedIOException(
            String.format("Failed reading offset from: %s", initialOffsetLocation), ioException);
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Verify the offset file exists at initialOffsetLocation and is readable by the query identity.
2. Restore/repair the checkpoint location (recreate from backup) or restart the query with a fresh checkpoint if offsets can be replayed.
3. Inspect the wrapped IOException cause for transient storage errors and add retry/throttling configuration for S3/HDFS.
4. Avoid external cleanup of checkpoint directories used by live streaming queries.

Example fix

// before
StreamingOffset offset = stream.initialOffset();
// after: guard with existence check and clear failure message
InputFile f = fileIO.newInputFile(initialOffsetLocation);
if (!f.exists()) { throw new IllegalStateException("Offset file missing, restart query or restore checkpoint: " + initialOffsetLocation); }
StreamingOffset offset = stream.initialOffset();

InputFile f = fileIO.newInputFile(initialOffsetLocation);
if (!f.exists()) {
  throw new IllegalStateException("Offset file missing at " + initialOffsetLocation);
}

try { offset = stream.initialOffset(); } catch (UncheckedIOException e) {
  if (e.getCause() instanceof FileNotFoundException) { restartQueryWithFreshCheckpoint(); }
  else { retryWithBackoff(e); }
}

When it happens

Trigger: initialOffset()/batch flow calls readOffset(file) and file.newStream() throws — offset file deleted from the checkpoint location, permissions revoked, or S3/HDFS read failure (throttling, connection reset).

Common situations: Checkpoint directory cleaned up by retention jobs while the query is live; S3 eventual-consistency or throttling during reads; IAM permission changes; offset file truncated by a concurrent writer.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed writing offset to: %s
• Failed to read StreamingOffset from json
• Failed writing offset to: %s
• Failed reading offset from: %s
• Failed to close equality delta writer

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/90ace5eb90505985. Report an issue: GitHub.