What causes "Failed to create output file: ${path.toString()}" in prestodb/presto?

Create the missing parent directory and grant write permission to the Presto user (hdfs dfs -mkdir -p / -chown / -chmod).

prestodb/presto · error · PrestoException

ICEBERG_FILESYSTEM_ERROR

ICEBERG_FILESYSTEM_ERROR

Error message

Failed to create output file: ${path.toString()}

What it means

HdfsOutputFile's constructor creates a HadoopOutputFile delegate for the target path. An IOException there is wrapped in a PrestoException with code ICEBERG_FILESYSTEM_ERROR ('Failed to create output file'), signaling the filesystem could not prepare the destination — typically permissions, missing parent directory, or storage connectivity.

Source

Thrown at presto-iceberg/src/main/java/com/facebook/presto/iceberg/HdfsOutputFile.java:48

public class HdfsOutputFile
        implements OutputFile
{
    private final OutputFile delegate;
    private final Path path;
    private final HdfsEnvironment environment;
    private final HdfsContext context;
    private final String user;

    public HdfsOutputFile(Path path, HdfsEnvironment environment, HdfsContext context)
    {
        this.path = requireNonNull(path, "path is null");
        this.environment = requireNonNull(environment, "environment is null");
        this.context = requireNonNull(context, "context is null");
        try {
            this.delegate = HadoopOutputFile.fromPath(path, environment.getFileSystem(context, path), environment.getConfiguration(context, path));
        }
        catch (IOException e) {
            throw new PrestoException(ICEBERG_FILESYSTEM_ERROR, "Failed to create output file: " + path.toString(), e);
        }
        this.user = context.getIdentity().getUser();
    }

    @Override
    public PositionOutputStream create()
    {
        return environment.doAs(user, delegate::create);
    }

    @Override
    public PositionOutputStream createOrOverwrite()
    {
        return environment.doAs(user, delegate::createOrOverwrite);
    }

    @Override
    public String location()

View on GitHub (pinned to 55bb57d202)

Solutions

Create the missing parent directory and grant write permission to the Presto user (hdfs dfs -mkdir -p / -chown / -chmod).
Check HDFS quota (hdfs dfs -count -q) and free space or raise the quota.
Verify object-store credentials/bucket policy allow writes at the target location.
Inspect the wrapped IOException cause to distinguish not-found vs permission vs quota vs connectivity.

Example fix

// before: writing into a location whose directory was removed
// after: ensure directory exists before write
Path dir = new Path(tableLocation);
FileSystem fs = environment.getFileSystem(context, dir);
if (!fs.exists(dir)) { fs.mkdirs(dir); }
OutputFile out = new HdfsOutputFile(new Path(tableLocation, "metadata.json"), environment, context);

Defensive patterns

Strategy: validation

Validate before calling

Path dir = new Path(tableLocation);
FileSystem fs = environment.getFileSystem(context, dir);
if (!fs.exists(dir)) fs.mkdirs(dir);
// and verify writability
Path probe = new Path(dir, ".presto_probe"); fs.create(probe).close(); fs.delete(probe, false);

Type guard

boolean canWrite(HdfsContext ctx, Path dir) { try { FileSystem fs = environment.getFileSystem(ctx, dir); return fs.exists(dir) && fs.mkdirs(dir); } catch (IOException e) { return false; } }

Try / catch

try { OutputFile out = new HdfsOutputFile(path, env, ctx); } catch (PrestoException e) { if (e.getCause() instanceof IOException) { ensureParentDir(path); /* retry once */ } throw e; }

Prevention

Pre-create table/base directories with correct ownership for the Presto user
Check HDFS quotas and object-store write policies before bulk loads
Keep filesystem configs (credentials, endpoints) valid for all workers
Run write-path smoke tests after storage or permission changes

When it happens

Trigger: Constructing HdfsOutputFile during Iceberg writes (commit metadata, new data files) when HadoopOutputFile.fromPath throws IOException: parent directory missing, permission denied, quota exceeded, or object-store put/create failing due to credentials.

Common situations: Table location directory deleted or not pre-created (HDFS does not auto-create parents), write permission missing for Presto user, HDFS disk quota exceeded, S3 bucket policy denying PutObject.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/ff6702c0e7f62076. Report an issue: GitHub.