{"record":{"id":"cb77a63db79b1560","repo":"prestodb/presto","slug":"generic-internal-error-cb77a6","errorCode":"GENERIC_INTERNAL_ERROR","errorMessage":"Failed to find compressionCodec for inputFormat: %s","messagePattern":"Failed to find compressionCodec for inputFormat: (.+?)","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"critical","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/HiveUtil.java","lineNumber":347,"sourceCode":"                    e);\n        }\n    }\n\n    public static void setReadColumns(Configuration configuration, List<Integer> readHiveColumnIndexes)\n    {\n        configuration.set(READ_COLUMN_IDS_CONF_STR, Joiner.on(',').join(readHiveColumnIndexes));\n        configuration.setBoolean(READ_ALL_COLUMNS, false);\n    }\n\n    public static Optional<CompressionCodec> getCompressionCodec(TextInputFormat inputFormat, Path file)\n    {\n        CompressionCodecFactory compressionCodecFactory;\n\n        try {\n            compressionCodecFactory = (CompressionCodecFactory) COMPRESSION_CODECS_FIELD.get(inputFormat);\n        }\n        catch (IllegalAccessException e) {\n            throw new PrestoException(GENERIC_INTERNAL_ERROR, \"Failed to find compressionCodec for inputFormat: \" + inputFormat.getClass().getName(), e);\n        }\n\n        if (compressionCodecFactory == null) {\n            return Optional.empty();\n        }\n\n        return Optional.ofNullable(compressionCodecFactory.getCodec(file));\n    }\n\n    public static InputFormat<?, ?> getInputFormat(Configuration configuration, String inputFormatName, String serDe, boolean symlinkTarget)\n    {\n        try {\n            JobConf jobConf = toJobConf(configuration);\n\n            Class<? extends InputFormat<?, ?>> inputFormatClass = getInputFormatClass(jobConf, inputFormatName);\n            if (symlinkTarget && (inputFormatClass == SymlinkTextInputFormat.class)) {\n                if (serDe == null) {\n                    throw new PrestoException(HIVE_UNSUPPORTED_FORMAT, \"Missing SerDe for SymlinkTextInputFormat\");","sourceCodeStart":329,"sourceCodeEnd":365,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/HiveUtil.java#L329-L365","documentation":"getCompressionCodec reads the private COMPRESSION_CODECS_FIELD reflectively from an input format instance (e.g. for compressed text input formats); if reflection is blocked (field renamed/removed in the Hadoop version on the classpath) it throws GENERIC_INTERNAL_ERROR. This is an internal compatibility failure, not user data corruption.","triggerScenarios":"Calling isUncompressed/getCompressionCodec on an input format class that lacks the expected private 'compressionCodec' factory field, or under a Hadoop/Hive version where the field name changed, or where a SecurityManager/JPMS restricts setAccessible.","commonSituations":"Upgrading Hadoop/Hive so HiveInputFormat internals no longer match the reflection assumption; shaded/relocated Hadoop classes on the classpath; restrictive JVM module settings.","solutions":["Align the Hadoop/Hive dependency versions with those the connector was built for","Remove conflicting shaded Hadoop classes from the classpath","If on a supported version still failing, report/patch: replace the reflective lookup for the new field name"],"exampleFix":"// before: classpath mixes hadoop 2.x and 3.x jars\n// after\nensure single hadoop-common version matching the connector's pom in the plugin classpath","handlingStrategy":"try-catch","validationCode":"boolean hasCodecField(Class<?> fmt) {\n    try { fmt.getDeclaredField(\"compressionCodecs\"); return true; } catch (NoSuchFieldException e) { return false; }\n}\n// call before invoking the codec path; if false, upgrade/align Hadoop versions","typeGuard":null,"tryCatchPattern":"try { codec = getCompressionCodec(inputFormat); } catch (PrestoException e) { if (e.getErrorCode().getCode() == StandardErrorCode.GENERIC_INTERNAL_ERROR.getCode()) { /* fall back to extension-based compression detection */ } else throw e; }","preventionTips":["Pin Hadoop/Hive dependency versions to those the connector was built against","Avoid shaded Hadoop jars on the connector plugin classpath","Add an integration test that exercises the compressed-input-format path after dependency upgrades"],"tags":["hive","reflection","internal-error","version-compatibility"],"backgroundTag":"reflective-access-failed","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}