What causes "Failed to get block locations for path: %s" in apache/iceberg?

Retry the call — HDFS metadata RPCs fail transiently under load (use Tasks.foreach with retry like the rest of Iceberg)

apache/iceberg · error · RuntimeIOException

Failed to get block locations for path

Error message

Failed to get block locations for path: %s

What it means

HadoopInputFile.getBlockLocations wraps Hadoop's FileSystem.getFileBlockLocations IOException in a RuntimeIOException. This is used for locality-aware planning; failure means the filesystem metadata call failed.

Solutions

Retry the call — HDFS metadata RPCs fail transiently under load (use Tasks.foreach with retry like the rest of Iceberg)
Check NameNode health / cluster connectivity
Verify the file still exists before requesting block locations
Disable or bypass locality optimization if block locations are not needed

Example fix

// before
String[] hosts = inputFile.getBlockLocations(0, len);
// after
String[] hosts = Tasks.foreach(inputFile)
    .retry(3).exponentialBackoff(100, 1000, 10000)
    .throwFailureWhenFinished()
    .run(file -> file.getBlockLocations(0, len));

Defensive patterns

Strategy: retry

Validate before calling

if (!inputFile.exists()) {
  throw new NotFoundException("Cannot get block locations, missing: %s", inputFile.location());
}

Try / catch

try { return file.getBlockLocations(start, len); }
catch (RuntimeIOException e) {
  LOG.warn("Block location lookup failed, skipping locality optimization", e);
  return new String[0]; // proceed without locality
}

Prevention

Wrap metadata calls in Tasks.foreach retry with backoff
Monitor NameNode health and RPC latency
Fall back to no-locality planning instead of failing the whole job

When it happens

Trigger: Calling getBlockLocations (directly or via Iceberg's locality/planning code) when the underlying FileSystem call throws IOException — NameNode unavailable, RPC timeout, path concurrently deleted.

Common situations: Busy/unreachable NameNode or HDFS in safe mode; S3-backed paths where block-location metadata is unavailable or calls fail intermittently; permissions issues fetching block info.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/c8b7c2f3763c1284. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopInputFile.java:226

  public FileStatus getStat() {
    return lazyStat();
  }

  public Path getPath() {
    return path;
  }

  public String[] getBlockLocations(long start, long end) {
    List<String> hosts = Lists.newArrayList();
    try {
      for (BlockLocation bl : fs.getFileBlockLocations(path, start, end)) {
        Collections.addAll(hosts, bl.getHosts());
      }

      return hosts.toArray(NO_LOCATION_PREFERENCE);

    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to get block locations for path: %s", path);
    }
  }

  @Override
  public String location() {
    return location;
  }

  @Override
  public boolean exists() {
    try {
      return lazyStat() != null;
    } catch (NotFoundException e) {
      return false;
    }
  }

  @Override

View on GitHub (pinned to 86d9c8fc54)