apache/flink · error · IllegalStateException

Truncation is not available in hadoop version < 2.7 , You ar

Error message

Truncation is not available in hadoop version < 2.7 , You are on Hadoop ${VersionInfo.getVersion()}

What it means

Flink only attempts truncate-based resume on Hadoop 2.7+, and truncate() explicitly guards with HadoopUtils.isMinHadoopVersion(2,7). On older Hadoop runtimes this IllegalStateException is thrown, naming the detected VersionInfo — resume-by-truncate is simply unavailable there.

Source

Thrown at flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java:186

                truncateMethod = FileSystem.class.getMethod("truncate", Path.class, long.class);
            } catch (NoSuchMethodException e) {
                throw new FlinkRuntimeException(
                        "Could not find a public truncate method on the Hadoop File System.");
            }

            if (!Modifier.isPublic(truncateMethod.getModifiers())) {
                throw new FlinkRuntimeException(
                        "Could not find a public truncate method on the Hadoop File System.");
            }

            truncateHandle = truncateMethod;
        }
    }

    private static boolean truncate(final FileSystem hadoopFs, final Path file, final long length)
            throws IOException {
        if (!HadoopUtils.isMinHadoopVersion(2, 7)) {
            throw new IllegalStateException(
                    "Truncation is not available in hadoop version < 2.7 , You are on Hadoop "
                            + VersionInfo.getVersion());
        }

        if (truncateHandle != null) {
            try {
                return (Boolean) truncateHandle.invoke(hadoopFs, file, length);
            } catch (InvocationTargetException e) {
                ExceptionUtils.rethrowIOException(e.getTargetException());
            } catch (Throwable t) {
                throw new IOException(
                        "Truncation of file failed because of access/linking problems with Hadoop's truncate call. "
                                + "This is most likely a dependency conflict or class loading problem.");
            }
        } else {
            throw new IllegalStateException("Truncation handle has not been initialized");
        }
        return false;

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Upgrade Hadoop (client jars and/or cluster) to 2.7 or newer
2. Remove old hadoop-common artifacts so VersionInfo reports the actual cluster version
3. On old Hadoop, disable file-sink resume for that path (accept rewrite from last checkpoint instead of resume)

if (!HadoopUtils.isMinHadoopVersion(2, 7)) {
    // choose a non-resuming sink strategy for this cluster instead of attempting truncate
}

When it happens

Trigger: Calling HadoopRecoverableFsDataOutputStream.resume/truncate paths with hadoop-common < 2.7 on the classpath; HadoopUtils.isMinHadoopVersion(2,7) returns false.

Common situations: Legacy clusters on Hadoop 1.x/2.0-2.6; old shaded Hadoop jars bundled in the application; environments where VersionInfo resolves to an ancient version due to jar conflicts.

Related errors

• Could not find a public truncate method on the Hadoop File S
• Truncate failed: ${tempFile} (requested=${recoverable.offset
• Problem while truncating file: ${path}
• Truncation of file failed because of access/linking problems
• Can't deploy a standalone cluster.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/9a4af1d5de5952e2. Report an issue: GitHub.