prestodb/presto · error · PrestoException

HIVE_INVALID_FILE_NAMES

HIVE_INVALID_FILE_NAMES

Error message

Hive table '%s' is corrupt. Some of the filenames in the partition: %s are not integers

What it means

Thrown when sorting legacy bucket files by numeric filename fails. File names are integers only when written with file_renaming_enabled=true; if the first file matches \d+ but other files in the partition cannot be parsed as integers, the partition is treated as corrupt.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/StoragePartitionLoader.java:630

                // legacy mode requires exactly one file per bucket
                if (fileInfos.size() != partitionBucketCount) {
                    throw new PrestoException(
                            HIVE_INVALID_BUCKET_FILES,
                            format("Hive table '%s' is corrupt. File '%s' does not match the standard naming pattern, and the number " +
                                            "of files in the directory (%s) does not match the declared bucket count (%s) for partition: %s",
                                    table.getSchemaTableName(),
                                    fileName,
                                    fileInfos.size(),
                                    partitionBucketCount,
                                    partitionName));
                }
                if (fileInfos.get(0).getFileName().matches("\\d+")) {
                    try {
                        // File names are integer if they are created when file_renaming_enabled is set to true
                        fileInfos.sort(Comparator.comparingInt(fileInfo -> Integer.parseInt(fileInfo.getFileName())));
                    }
                    catch (NumberFormatException e) {
                        throw new PrestoException(
                                HIVE_INVALID_FILE_NAMES,
                                format("Hive table '%s' is corrupt. Some of the filenames in the partition: %s are not integers",
                                        new SchemaTableName(table.getDatabaseName(), table.getTableName()),
                                        partitionName));
                    }
                }
                else {
                    // Sort FileStatus objects (instead of, e.g., fileStatus.getPath().toString). This matches org.apache.hadoop.hive.ql.metadata.Table.getSortedPaths
                    fileInfos.sort(null);
                }

                // Use position in sorted list as the bucket number
                bucketToFileInfo.clear();
                for (int i = 0; i < fileInfos.size(); i++) {
                    bucketToFileInfo.put(i, fileInfos.get(i));
                }
                break;
            }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Remove or rename non-integer-named files in the partition so all files match the integer naming scheme
2. Make naming consistent: re-run the write with a single file_renaming_enabled setting for the whole partition
3. Exclude stray files (_SUCCESS, .crc) from the partition directory
4. If bucketing is not required, unbucket the table/partition to bypass this path

Example fix

// before: mixed names
1  2  000000_0.bak
// after
hdfs dfs -rm .../000000_0.bak   # or rename to '3'

// before querying: all files must be integer-named if any is
List<String> names = listPartitionFileNames(partitionDir);
boolean anyDigits = names.stream().anyMatch(n -> n.matches("\\d+"));
if (anyDigits && names.stream().anyMatch(n -> !n.matches("\\d+"))) {
    throw new IllegalStateException("Mixed file naming in partition: " + names);
}

static boolean hasConsistentIntegerNames(List<String> names) {
    return !names.stream().anyMatch(n -> n.matches("\\d+"))
        || names.stream().allMatch(n -> n.matches("\\d+"));
}

try {
    session.execute(query);
} catch (PrestoException e) {
    if (HiveErrorCode.HIVE_INVALID_FILE_NAMES.toErrorCode().equals(e.getErrorCode())) {
        // inspect partition for mixed naming and repair
    } else throw e;
}

When it happens

Trigger: computeBucketToFileInfoMapping (from bucketToFileInfo) sees the first file name is all digits, sorts all files with Integer.parseInt, and one file name (e.g. '000000_0_copy' or a CRC file) throws NumberFormatException.

Common situations: Mixed naming in one partition: some files from a file_renaming_enabled writer, others from a legacy writer; hidden files like .filename.crc or _SUCCESS picked up by the listing; manually added files with suffixes.

Related errors

• unknown java type
• HIVE_INVALID_BUCKET_FILES
• HIVE_INVALID_METADATA
• HIVE_INVALID_METADATA
• Unsupported bucket function type

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/18368d6506aebede. Report an issue: GitHub.