{"record":{"id":"9a4af1d5de5952e2","repo":"apache/flink","slug":"truncation-is-not-available-in-hadoop-version-2","errorCode":null,"errorMessage":"Truncation is not available in hadoop version < 2.7 , You are on Hadoop ${VersionInfo.getVersion()}","messagePattern":"Truncation is not available in hadoop version < 2\\.7 , You are on Hadoop (.+?)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java","lineNumber":186,"sourceCode":"                truncateMethod = FileSystem.class.getMethod(\"truncate\", Path.class, long.class);\n            } catch (NoSuchMethodException e) {\n                throw new FlinkRuntimeException(\n                        \"Could not find a public truncate method on the Hadoop File System.\");\n            }\n\n            if (!Modifier.isPublic(truncateMethod.getModifiers())) {\n                throw new FlinkRuntimeException(\n                        \"Could not find a public truncate method on the Hadoop File System.\");\n            }\n\n            truncateHandle = truncateMethod;\n        }\n    }\n\n    private static boolean truncate(final FileSystem hadoopFs, final Path file, final long length)\n            throws IOException {\n        if (!HadoopUtils.isMinHadoopVersion(2, 7)) {\n            throw new IllegalStateException(\n                    \"Truncation is not available in hadoop version < 2.7 , You are on Hadoop \"\n                            + VersionInfo.getVersion());\n        }\n\n        if (truncateHandle != null) {\n            try {\n                return (Boolean) truncateHandle.invoke(hadoopFs, file, length);\n            } catch (InvocationTargetException e) {\n                ExceptionUtils.rethrowIOException(e.getTargetException());\n            } catch (Throwable t) {\n                throw new IOException(\n                        \"Truncation of file failed because of access/linking problems with Hadoop's truncate call. \"\n                                + \"This is most likely a dependency conflict or class loading problem.\");\n            }\n        } else {\n            throw new IllegalStateException(\"Truncation handle has not been initialized\");\n        }\n        return false;","sourceCodeStart":168,"sourceCodeEnd":204,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java#L168-L204","documentation":"Flink only attempts truncate-based resume on Hadoop 2.7+, and truncate() explicitly guards with HadoopUtils.isMinHadoopVersion(2,7). On older Hadoop runtimes this IllegalStateException is thrown, naming the detected VersionInfo — resume-by-truncate is simply unavailable there.","triggerScenarios":"Calling HadoopRecoverableFsDataOutputStream.resume/truncate paths with hadoop-common < 2.7 on the classpath; HadoopUtils.isMinHadoopVersion(2,7) returns false.","commonSituations":"Legacy clusters on Hadoop 1.x/2.0-2.6; old shaded Hadoop jars bundled in the application; environments where VersionInfo resolves to an ancient version due to jar conflicts.","solutions":["Upgrade Hadoop (client jars and/or cluster) to 2.7 or newer","Remove old hadoop-common artifacts so VersionInfo reports the actual cluster version","On old Hadoop, disable file-sink resume for that path (accept rewrite from last checkpoint instead of resume)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if (!HadoopUtils.isMinHadoopVersion(2, 7)) {\n    // choose a non-resuming sink strategy for this cluster instead of attempting truncate\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check Hadoop version at deployment time and gate resume features on it","Upgrade legacy clusters before enabling recoverable HDFS sinks","Log org.apache.hadoop.util.VersionInfo.getVersion() during setup to catch jar conflicts"],"tags":["hdfs","hadoop-version","truncate","unsupported-operation"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}