apache/iceberg · error · RuntimeIOException

Failed to get stream length

Error message

Failed to get stream length

What it means

AvroFileAppender.length() reports the stored length of the underlying output stream. If reading the stored length from the stream throws an IOException, it is rethrown as RuntimeIOException with 'Failed to get stream length'. This typically reflects an I/O failure while querying the stream position, distinct from having no stream at all.

Source

Thrown at core/src/main/java/org/apache/iceberg/avro/AvroFileAppender.java:85

    } catch (IOException e) {
      throw new RuntimeIOException(e);
    }
  }

  @Override
  public Metrics metrics() {
    Preconditions.checkState(isClosed, "Cannot return metrics while appending to an open file.");

    return AvroMetrics.fromWriter(datumWriter, icebergSchema, numRecords, metricsConfig);
  }

  @Override
  public long length() {
    if (stream != null) {
      try {
        return stream.storedLength();
      } catch (IOException e) {
        throw new RuntimeIOException(e, "Failed to get stream length");
      }
    }
    throw new RuntimeIOException("Failed to get stream length: no open stream");
  }

  @Override
  public void close() throws IOException {
    if (writer != null) {
      writer.close();
      this.writer = null;
      isClosed = true;
    }
  }

  @SuppressWarnings("unchecked")
  private static <D> DataFileWriter<D> newAvroWriter(
      Schema schema,
      PositionOutputStream stream,

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the cause chain for the underlying IOException and fix the storage access problem
2. Retry the write/commit operation after restoring connectivity to the storage system
3. Avoid calling length() after a prior write failure; treat the appender as failed and abort the write
4. Check storage backend health (HDFS NameNode, S3 endpoint) and credentials

Example fix

// before
long len = appender.length(); // throws after S3 connection drop

// after
if (!isHealthy(appender)) { abortWrite(); } else { long len = appender.length(); }

try { return appender.length(); } catch (RuntimeIOException e) { if (e.getCause() instanceof IOException) { /* retry after storage recovery or abort write */ } throw e; }

When it happens

Trigger: Calling length() on an appender whose OutputFile stream throws on storedLength() — e.g. the underlying file handle is broken, the remote storage call fails, or the stream is in an inconsistent state after an earlier write failure.

Common situations: HDFS/S3 connection failures while retrieving file size during manifest writing; a previously failed write leaving the stream unusable; flaky network storage mid-commit.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to get stream length: no open stream
• Failed to read header and fingerprint bytes
• Cannot read manifest list file: %s
• Failed to create snapshot list writer for path: %s
• Cannot read manifest list file: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/58245d5318367d20. Report an issue: GitHub.