apache/iceberg · error · RuntimeIOException

Failed to create file: %s

Error message

Failed to create file: %s

What it means

HadoopOutputFile.create() wraps any non-already-exists IOException from fs.create into a RuntimeIOException with 'Failed to create file'. This covers everything except the duplicate-path case handled above.

Source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopOutputFile.java:78

  public static OutputFile fromPath(Path path, FileSystem fs, Configuration conf) {
    return new HadoopOutputFile(fs, path, conf);
  }

  private HadoopOutputFile(FileSystem fs, Path path, Configuration conf) {
    this.fs = fs;
    this.path = path;
    this.conf = conf;
  }

  @Override
  public PositionOutputStream create() {
    try {
      return HadoopStreams.wrap(fs.create(path, false /* createOrOverwrite */));
    } catch (FileAlreadyExistsException e) {
      throw new AlreadyExistsException(e, "Path already exists: %s", path);
    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to create file: %s", path);
    }
  }

  @Override
  public PositionOutputStream createOrOverwrite() {
    try {
      return HadoopStreams.wrap(fs.create(path, true /* createOrOverwrite */));
    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to create file: %s", path);
    }
  }

  public Path getPath() {
    return path;
  }

  public Configuration getConf() {
    return conf;

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Ensure parent directories exist (fs.mkdirs) or write to a location that does
2. Verify write permissions/credentials for the target path
3. Check quotas and free space on the filesystem
4. Retry transient network/RPC failures with backoff

Example fix

// before
PositionOutputStream s = io.newOutputFile("hdfs://nn/data/part-0.parquet").create();
// after
FileSystem fs = FileSystem.get(conf);
Path p = new Path("hdfs://nn/data/part-0.parquet");
fs.mkdirs(p.getParent());
PositionOutputStream s = io.newOutputFile(p.toString()).create();

When it happens

Trigger: Calling create() when fs.create throws IOException for reasons other than file-exists: permission denied, parent directory missing, NameNode unreachable, quota exceeded, S3 access denied/throttling.

Common situations: Missing HDFS directory (no mkdirs beforehand); write permissions missing for the job user; HDFS quota/space exceeded; bad S3 credentials or missing bucket.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to create Parquet output file for %s
• Failed to write json to file: %s
• Failed to list tables under: %s
• Failed to delete file: %s
• Create namespace failed: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/7bffe83db2403a66. Report an issue: GitHub.