What causes "Create parquet reader for this file [%s] failed" in apache/seatunnel?

Check the chained cause `e` for the root IOException; confirm the file exists and is reachable (`hdfs dfs -ls` / `aws s3 ls`).

apache/seatunnel · error · FileConnectorException

COMMON_ERROR_CODE-14

COMMON_ERROR_CODE-14

Error message

Create parquet reader for this file [%s] failed

What it means

ParquetReadStrategy opens the file via ParquetFileReader.open inside doWithHadoopAuth to read the footer; any IOException while creating the reader is wrapped as READER_OPERATION_FAILED with the file path, chaining the original cause. This means the parquet reader could not be constructed at all — typically I/O, auth, or file-access problems, not schema issues.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/ParquetReadStrategy.java:761

        return getSeaTunnelRowTypeInfoWithUserConfigRowType(path, null);
    }

    @Override
    public SeaTunnelRowType getSeaTunnelRowTypeInfoWithUserConfigRowType(
            String path, SeaTunnelRowType configRowType) throws FileConnectorException {
        ParquetMetadata metadata;
        try (ParquetFileReader reader =
                hadoopFileSystemProxy.doWithHadoopAuth(
                        ((configuration, userGroupInformation) -> {
                            HadoopInputFile hadoopInputFile =
                                    HadoopInputFile.fromPath(new Path(path), configuration);
                            return ParquetFileReader.open(hadoopInputFile);
                        }))) {
            metadata = reader.getFooter();
        } catch (IOException e) {
            String errorMsg =
                    String.format("Create parquet reader for this file [%s] failed", path);
            throw new FileConnectorException(
                    CommonErrorCodeDeprecated.READER_OPERATION_FAILED, errorMsg, e);
        }

        FileMetaData fileMetaData = metadata.getFileMetaData();
        MessageType originalSchema = fileMetaData.getSchema();
        if (readColumns.isEmpty()) {
            for (int i = 0; i < originalSchema.getFieldCount(); i++) {
                readColumns.add(originalSchema.getFieldName(i));
            }
        }
        String[] fields = new String[readColumns.size()];
        SeaTunnelDataType<?>[] types = new SeaTunnelDataType[readColumns.size()];
        buildColumnsWithErrorCheck(
                TablePath.DEFAULT,
                IntStream.range(0, readColumns.size()).iterator(),
                i -> {
                    fields[i] = readColumns.get(i);
                    Type type = originalSchema.getType(fields[i]);

View on GitHub (pinned to cf67b549a7)

Solutions

Check the chained cause `e` for the root IOException; confirm the file exists and is reachable (`hdfs dfs -ls` / `aws s3 ls`).
Verify Hadoop auth configuration (kerberos principal/keytab, or storage credentials) used by doWithHadoopAuth.
Validate the file isn't truncated/corrupt (footer readable via parquet-tools).
Re-run after fixing storage/network access; if transient (HDFS flapping), retry the read.

Example fix

// before: no auth config
FaT = { source = { file = { path = "hdfs://nn/data/x.parquet" } } }

// after: provide kerberos config
FaT = { source = { file = { path = "hdfs://nn/data/x.parquet" }, hadoop_security_kerberos_principal = "user@REALM", hadoop_security_kerberos_keytab_path = "/etc/keytabs/user.keytab" } }

Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-flight: confirm the path is readable through Hadoop before submitting the job
org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
org.apache.hadoop.fs.Path p = new org.apache.hadoop.fs.Path(path);
org.apache.hadoop.fs.FileSystem fs = p.getFileSystem(conf);
if (!fs.exists(p) || !fs.open(p).read(new byte[4]) .equals(4)) {
    throw new IllegalStateException("Cannot read parquet file: " + path);
}

Try / catch

try {
    rows = parquetSource.read();
} catch (FileConnectorException e) {
    if (e.getMessage().startsWith("Create parquet reader for this file")) {
        Throwable root = ExceptionUtils.getRootCause(e);
        // inspect root (IOException: missing file, auth, hdfs) and retry if transient
    } else throw e;
}

Prevention

Always inspect the chained root cause for the real IOException
Verify Kerberos/keytab or storage credentials before running
Check file existence and stability (not being overwritten) before reads
Retry on transient storage/network errors with backoff

When it happens

Trigger: readWithAvro calls hadoopFileSystemProxy.doWithHadoopAuth to open HadoopInputFile and ParquetFileReader.open throws IOException (missing file, HDFS/DFS access failure, permission error, Kerberos/auth failure, corrupt footer).

Common situations: File deleted/moved between listing and reading; HDFS NameNode unreachable; missing Kerberos credentials or wrong fs.defaultFS; S3/OSS credentials invalid; truncated parquet file.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/e69c1013df596795. Report an issue: GitHub.