{"record":{"id":"da76ba81019cdc6e","repo":"apache/iceberg","slug":"failed-to-get-file-length","errorCode":null,"errorMessage":"Failed to get file length","messagePattern":"Failed to get file length","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java","lineNumber":194,"sourceCode":"  @Override\n  public long length() {\n    try {\n      long length = 0L;\n\n      if (writer != null) {\n        length += writer.getPos();\n      }\n\n      if (!closed && recordCount > 0) {\n        // recordCount > 0 when there are records in the write store that have not been flushed to\n        // the Parquet file\n        length += writeStore.getBufferedSize();\n      }\n\n      return length;\n\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Failed to get file length\", e);\n    }\n  }\n\n  @Override\n  public List<Long> splitOffsets() {\n    if (writer != null) {\n      return ParquetUtil.getSplitOffsets(writer.getFooter());\n    }\n    return null;\n  }\n\n  private void checkSize() {\n    if (trackUncompressedSize) {\n      if (rowGroupUncompressedSize >= targetRowGroupSize) {\n        flushRowGroup(false);\n      } else if (recordCount >= nextCheckRecordCount) {\n        evaluateRowGroupSize(rowGroupUncompressedSize, false);\n      }","sourceCodeStart":176,"sourceCodeEnd":212,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java#L176-L212","documentation":"ParquetWriter.length() reports the bytes written so far plus the currently buffered size; it wraps any IOException from querying the underlying OutputFile/PositionOutputStream in an UncheckedIOException. This is called frequently (metrics, split-offset computation, commit-time file sizing), so a failure here means the underlying storage stream is broken or unreadable.","triggerScenarios":"Calling ParquetWriter.length() (directly or via checkSize/flushRowGroup/splitOffsets) while the underlying PositionOutputStream backed by the OutputFile throws IOException when queried for position/buffered size.","commonSituations":"Underlying local file handle closed or deleted mid-write; cloud storage (S3/GCS) stream failing on a position check due to network issues; HDFS lease recovery or client errors when querying the open file's length.","solutions":["Inspect e.getCause() for the root IOException from the output stream","Check network/connectivity to the storage backend (S3, GCS, HDFS NameNode)","Verify the output file/stream has not been closed or removed by another process","Retry the write job if the failure was transient network I/O","Validate FileIO/OutputFile implementation handles position queries correctly"],"exampleFix":"// before: transient S3 failure makes length() blow up mid-task\nlong len = writer.length();\n\n// after: treat as an I/O failure of the stream and fail the task with context\nlong len;\ntry {\n  len = writer.length();\n} catch (UncheckedIOException e) {\n  throw new RuntimeException(\"Output stream for data file is broken: \" + e.getCause().getMessage(), e);\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { long len = writer.length(); } catch (UncheckedIOException e) { throw new RuntimeException(\"Broken output stream: \" + e.getCause().getMessage(), e); }","preventionTips":["Ensure network stability to S3/GCS/HDFS during long write tasks; enable retries in the filesystem client","Never close or delete the output file from another thread while writing","Verify your FileIO's PositionOutputStream implements position queries correctly","Treat UncheckedIOException from length() as a stream-level failure, not a metrics bug"],"tags":["parquet","io","file-length"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}