apache/iceberg · error · UncheckedIOException

Failed reading offset from: ${initialOffsetLocation}

Error message

Failed reading offset from: ${initialOffsetLocation}

What it means

readOffset streams the previously written offset JSON and parses it with StreamingOffset.fromJson. An IOException while reading is wrapped as UncheckedIOException naming initialOffsetLocation, so a corrupt/unreadable offset file stops the stream rather than silently resuming wrong.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkMicroBatchStream.java:310

    }

    private void writeOffset(StreamingOffset offset, OutputFile file) {
      try (OutputStream outputStream = file.create()) {
        BufferedWriter writer =
            new BufferedWriter(new OutputStreamWriter(outputStream, StandardCharsets.UTF_8));
        writer.write(offset.json());
        writer.flush();
      } catch (IOException ioException) {
        throw new UncheckedIOException(
            String.format("Failed writing offset to: %s", initialOffsetLocation), ioException);
      }
    }

    private StreamingOffset readOffset(InputFile file) {
      try (InputStream in = file.newStream()) {
        return StreamingOffset.fromJson(in);
      } catch (IOException ioException) {
        throw new UncheckedIOException(
            String.format("Failed reading offset from: %s", initialOffsetLocation), ioException);
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the cause for whether the file is missing or truncated; restore the checkpoint location from backup.
2. Verify the streaming checkpoint path was not removed by lifecycle/cleanup rules.
3. If the offset is unrecoverable, reset the stream with a new checkpoint dir or Spark's streaming checkpoint recovery options.

// check offset file exists and is non-empty before resume
if (table.io().newInputFile(initialOffsetLocation).getLength() == 0) { /* recover checkpoint */ }

try { offset = readOffset(file); } catch (UncheckedIOException e) { recoverOrResetCheckpoint(e.getCause()); }

When it happens

Trigger: initialOffset() (or batch resume) calling readOffset when the offset file at initialOffsetLocation cannot be opened or fully read (missing file, transient storage error, truncation).

Common situations: Checkpoint location on flaky object storage, files deleted by retention/cleanup policies, or partially written offsets after an abrupt job kill.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed writing offset to: ${initialOffsetLocation}
• Table refresh failed
• Queue filling failed
• Failed to read StreamingOffset from json
• Failed to write StreamingOffset to json

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/a06494e0571d5ef5. Report an issue: GitHub.