prestodb/presto · critical · PrestoException

GENERIC_INTERNAL_ERROR

GENERIC_INTERNAL_ERROR

Error message

Failed to find compressionCodec for inputFormat: %s

What it means

getCompressionCodec reads the private COMPRESSION_CODECS_FIELD reflectively from an input format instance (e.g. for compressed text input formats); if reflection is blocked (field renamed/removed in the Hadoop version on the classpath) it throws GENERIC_INTERNAL_ERROR. This is an internal compatibility failure, not user data corruption.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/HiveUtil.java:347

                    e);
        }
    }

    public static void setReadColumns(Configuration configuration, List<Integer> readHiveColumnIndexes)
    {
        configuration.set(READ_COLUMN_IDS_CONF_STR, Joiner.on(',').join(readHiveColumnIndexes));
        configuration.setBoolean(READ_ALL_COLUMNS, false);
    }

    public static Optional<CompressionCodec> getCompressionCodec(TextInputFormat inputFormat, Path file)
    {
        CompressionCodecFactory compressionCodecFactory;

        try {
            compressionCodecFactory = (CompressionCodecFactory) COMPRESSION_CODECS_FIELD.get(inputFormat);
        }
        catch (IllegalAccessException e) {
            throw new PrestoException(GENERIC_INTERNAL_ERROR, "Failed to find compressionCodec for inputFormat: " + inputFormat.getClass().getName(), e);
        }

        if (compressionCodecFactory == null) {
            return Optional.empty();
        }

        return Optional.ofNullable(compressionCodecFactory.getCodec(file));
    }

    public static InputFormat<?, ?> getInputFormat(Configuration configuration, String inputFormatName, String serDe, boolean symlinkTarget)
    {
        try {
            JobConf jobConf = toJobConf(configuration);

            Class<? extends InputFormat<?, ?>> inputFormatClass = getInputFormatClass(jobConf, inputFormatName);
            if (symlinkTarget && (inputFormatClass == SymlinkTextInputFormat.class)) {
                if (serDe == null) {
                    throw new PrestoException(HIVE_UNSUPPORTED_FORMAT, "Missing SerDe for SymlinkTextInputFormat");

View on GitHub (pinned to 55bb57d202)

Solutions

1. Align the Hadoop/Hive dependency versions with those the connector was built for
2. Remove conflicting shaded Hadoop classes from the classpath
3. If on a supported version still failing, report/patch: replace the reflective lookup for the new field name

Example fix

// before: classpath mixes hadoop 2.x and 3.x jars
// after
ensure single hadoop-common version matching the connector's pom in the plugin classpath

boolean hasCodecField(Class<?> fmt) {
    try { fmt.getDeclaredField("compressionCodecs"); return true; } catch (NoSuchFieldException e) { return false; }
}
// call before invoking the codec path; if false, upgrade/align Hadoop versions

try { codec = getCompressionCodec(inputFormat); } catch (PrestoException e) { if (e.getErrorCode().getCode() == StandardErrorCode.GENERIC_INTERNAL_ERROR.getCode()) { /* fall back to extension-based compression detection */ } else throw e; }

When it happens

Trigger: Calling isUncompressed/getCompressionCodec on an input format class that lacks the expected private 'compressionCodec' factory field, or under a Hadoop/Hive version where the field name changed, or where a SecurityManager/JPMS restricts setAccessible.

Common situations: Upgrading Hadoop/Hive so HiveInputFormat internals no longer match the reflection assumption; shaded/relocated Hadoop classes on the classpath; restrictive JVM module settings.

Related errors

• unexpected update mode:
• Unsupported bucket function type
• GENERIC_INTERNAL_ERROR
• HIVE_UNKNOWN_ERROR
• GENERIC_INTERNAL_ERROR

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/cb77a63db79b1560. Report an issue: GitHub.