What causes "Failed to get status for file: %s" in apache/iceberg?

Inspect the cause exception for the concrete FS error (permissions, throttling, token expiry).

apache/iceberg · error · RuntimeIOException

Failed to get status for file: %s

Error message

Failed to get status for file: %s

What it means

The non-FileNotFound branch of lazyStat(): any other IOException while fetching the FileStatus is wrapped in this RuntimeIOException. It signals the filesystem was reachable but status retrieval failed (network, permissions, throttling, corruption) rather than the file being absent.

Source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopInputFile.java:168

  }

  private HadoopInputFile(FileSystem fs, FileStatus stat, Configuration conf) {
    this.fs = fs;
    this.path = stat.getPath();
    this.location = path.toString();
    this.stat = stat;
    this.conf = conf;
    this.length = stat.getLen();
  }

  private FileStatus lazyStat() {
    if (stat == null) {
      try {
        this.stat = fs.getFileStatus(path);
      } catch (FileNotFoundException e) {
        throw new NotFoundException(e, "File does not exist: %s", path);
      } catch (IOException e) {
        throw new RuntimeIOException(e, "Failed to get status for file: %s", path);
      }
    }
    return stat;
  }

  @Override
  public long getLength() {
    if (length == null) {
      this.length = lazyStat().getLen();
    }
    return length;
  }

  @Override
  public SeekableInputStream newStream() {
    try {
      return HadoopStreams.wrap(fs.open(path));
    } catch (FileNotFoundException e) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

Inspect the cause exception for the concrete FS error (permissions, throttling, token expiry).
Retry with backoff for transient errors (throttling, network blips).
Refresh credentials/tokens if the job runs longer than the token lifetime.
Verify FS permissions and bucket policies allow HEAD/GET on the object.
Reduce parallelism if the object store is rate-limiting requests.

Example fix

// before
long len = io.newInputFile(path).getLength();
// after
try {
  long len = io.newInputFile(path).getLength();
} catch (RuntimeIOException e) {
  if (e.getCause() instanceof FileNotFoundException) { /* missing file path */ }
  else { /* transient FS error: retry with backoff */ }
}

Defensive patterns

Strategy: retry

Validate before calling

InputFile f = io.newInputFile(path);
if (!f.exists()) return; // distinguish missing (NotFoundException) from transient IO

Try / catch

try {
  long len = io.newInputFile(path).getLength();
} catch (RuntimeIOException e) {
  if (e.getCause() instanceof FileNotFoundException) throw e; // permanent
  // otherwise retry with backoff
}

Prevention

Use Tasks.foreach/retry with exponential backoff for FS reads in jobs.
Renew Kerberos/cloud credentials before they expire in long jobs.
Watch for object-store throttling and cap reader parallelism.

When it happens

Trigger: getLength/getStat/exists on a HadoopInputFile when fs.getFileStatus throws IOException other than FileNotFoundException — HDFS datanode failures, S3 503/throttling responses, Kerberos token expiry, permission denied.

Common situations: Expired Kerberos tokens in long-running jobs; S3 request-rate throttling during large scans; intermittent HDFS connectivity; bucket permissions changed mid-job.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/83e5901821bea6bf. Report an issue: GitHub.