prestodb/presto · error · PrestoException

ICEBERG_WRITER_OPEN_ERROR

ICEBERG_WRITER_OPEN_ERROR

Error message

Error creating Parquet file

What it means

Wraps an IOException thrown while opening/creating a Parquet writer in IcebergFileWriterFactory.createParquetWriter. The failure happens during file creation (path setup, filesystem access, Parquet schema/message-type construction or writer init), before any rows are written, and is rethrown as ICEBERG_WRITER_OPEN_ERROR.

Source

Thrown at presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergFileWriterFactory.java:178

            return new IcebergParquetFileWriter(
                    hdfsEnvironment.doAs(session.getUser(), () -> fileSystem.create(outputPath)),
                    rollbackAction,
                    fileColumnNames,
                    fileColumnTypes,
                    convert(icebergSchema, "table"),
                    makeTypeMap(fileColumnTypes, fileColumnNames),
                    parquetWriterOptions,
                    IntStream.range(0, fileColumnNames.size()).toArray(),
                    getCompressionCodec(session).getParquetCompressionCodec(),
                    outputPath,
                    hdfsEnvironment,
                    hdfsContext,
                    metricsConfig,
                    writerTimezone,
                    nodeVersion.toString());
        }
        catch (IOException e) {
            throw new PrestoException(ICEBERG_WRITER_OPEN_ERROR, "Error creating Parquet file", e);
        }
    }

    private IcebergFileWriter createOrcWriter(
            Path outputPath,
            Schema icebergSchema,
            JobConf jobConf,
            ConnectorSession session)
    {
        try {
            FileSystem fileSystem = hdfsEnvironment.getFileSystem(session.getUser(), outputPath, jobConf);
            DataSink orcDataSink = hdfsEnvironment.doAs(session.getUser(), () -> new OutputStreamDataSink(fileSystem.create(outputPath)));
            Callable<Void> rollbackAction = () -> {
                hdfsEnvironment.doAs(session.getUser(), () -> fileSystem.delete(outputPath, false));
                return null;
            };

            List<Types.NestedField> columnFields = icebergSchema.columns();

View on GitHub (pinned to 55bb57d202)

Solutions

  1. Check the wrapped cause and target output path permissions/existence; ensure the write location is accessible
  2. Verify filesystem credentials (Kerberos ticket, S3/ABFS keys or instance role) are valid and not expired
  3. Confirm the target directory exists or the connector can create it, and storage quota/disk is not exhausted
  4. Retry the query after fixing transient connectivity to the storage system

Example fix

// before
-- writing to s3://bucket/missing-prefix/ with no s3 credentials configured
// after
-- configure hive.s3.aws-access-key/secret (or instance profile) and ensure the target directory is writable
CREATE TABLE ... WITH (format='PARQUET', location='s3://bucket/valid-prefix/');
Defensive patterns

Strategy: try-catch

Validate before calling

// verify the target location is writable before writing
boolean ok = fs.exists(dir) ? fs.isDirectory(dir) && fs.access(dir, WRITE) : fs.mkdirs(dir);
if (!ok) throw new IOException("Cannot write to " + dir);

Try / catch

try { /* run write query */ } catch (PrestoException e) { if ("ICEBERG_WRITER_OPEN_ERROR".equals(e.getErrorCode().getName())) { /* check e.getCause() IOException: fix path/permissions/credentials, then retry */ } else throw e; }

Prevention

When it happens

Trigger: createFileWriter dispatches to createParquetWriter for a PARQUET table and the underlying HDFS/Parquet writer constructor throws IOException — bad output path, permission denied, missing filesystem credentials, or invalid schema for Parquet.

Common situations: Insufficient permissions or non-existent target directory in HDFS/S3, expired Kerberos/credentials, wrong writer timezone or node version config, or quota/disk-full conditions.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/dae731421e9c2ad9. Report an issue: GitHub.