prestodb/presto · error · PrestoException

HIVE_INVALID_BUCKET_FILES

HIVE_INVALID_BUCKET_FILES

Error message

A row that is supposed to be in bucket %s is encountered. Only rows in bucket %s (modulo %s) are expected

What it means

HiveBucketAdapterRecordCursor wraps a RecordCursor to re-bucket rows: it recomputes each row's original bucket via HiveBucketing.getHiveBucket and keeps only rows that map to the requested bucket (directly or modulo partitionBucketCount, the multiply-bucketed split case). This PrestoException (HIVE_INVALID_BUCKET_FILES) is thrown in advanceNextPosition when a row's computed bucket is not congruent to bucketToKeep modulo partitionBucketCount, meaning the underlying file contains rows that do not belong to the bucket this file was supposed to hold — the bucketing of the file's data does not match its declared bucket.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/HiveBucketAdapterRecordCursor.java:129

                else if (javaType == long.class) {
                    scratch[i] = delegate.getLong(index);
                }
                else if (javaType == double.class) {
                    scratch[i] = delegate.getDouble(index);
                }
                else if (javaType == Slice.class) {
                    scratch[i] = delegate.getSlice(index);
                }
                else if (javaType == Block.class) {
                    scratch[i] = (Block) delegate.getObject(index);
                }
                else {
                    throw new UnsupportedOperationException("unknown java type");
                }
            }
            int bucket = HiveBucketing.getHiveBucket(tableBucketCount, typeInfoList, scratch, useLegacyTimestampBucketing);
            if ((bucket - bucketToKeep) % partitionBucketCount != 0) {
                throw new PrestoException(HIVE_INVALID_BUCKET_FILES, format(
                        "A row that is supposed to be in bucket %s is encountered. Only rows in bucket %s (modulo %s) are expected",
                        bucket, bucketToKeep % partitionBucketCount, partitionBucketCount));
            }
            if (bucket == bucketToKeep) {
                return true;
            }
        }
    }

    @Override
    public boolean getBoolean(int field)
    {
        return delegate.getBoolean(field);
    }

    @Override
    public long getLong(int field)
    {

View on GitHub (pinned to 55bb57d202)

Solutions

Verify the table/partition's bucketing metadata (bucket columns, bucket count, and any hive.bucketing.* session/property settings, especially useLegacyTimestampBucketing) matches how the files were actually written, and fix the mismatch.
Use the correct legacy-mode session property (e.g. set the legacy timestamp bucketing flag) when reading data written by older Hive/Presto versions.
Rebuild/rewrite the affected partition (INSERT OVERWRITE or CTAS) so bucket files are regenerated by the engine from the declared bucketing spec.
Check for manually copied, renamed, or concatenated bucket files and restore files so each file number matches the bucket its rows hash into.

Example fix

// before: reading data written by legacy Hive bucketing fails
SELECT * FROM bucketed_table;

// after: match the writer's bucketing mode for the session
SET SESSION use_legacy_timestamp_bucketing = true;
SELECT * FROM bucketed_table;

Defensive patterns

Strategy: validation

Validate before calling

-- before querying: confirm bucketing metadata matches the data on disk
SHOW CREATE TABLE bucketed_table;
-- check: bucketed-by columns exist, bucket count per partition matches file layout,
-- and legacy timestamp bucketing mode matches how data was written.
-- Optionally verify a bucket file's rows re-hash to its bucket number in a scratch query.

Try / catch

try { ResultSet rs = stmt.executeQuery("SELECT * FROM bucketed_table"); ... }
catch (SQLException e) {
  if (e.getMessage() != null && e.getMessage().contains("HIVE_INVALID_BUCKET_FILES")) {
    // re-write the partition or fix legacy-bucketing session settings before retrying
  } else throw e;
}

Prevention

Never manually copy, rename, or concatenate bucket files; always rewrite partitions via INSERT OVERWRITE.
Keep hive bucketing session properties (legacy timestamp bucketing) consistent across the cluster that wrote and the cluster that reads.
Verify partition bucket counts match the table spec when using multiply-bucketed tables.
After any external Hive-CLI schema/layout change, run a validation query against one partition before production reads.

When it happens

Trigger: Reading a Hive table/partition whose bucket files were produced with a different bucketing hash, a different bucket column ordering, mismatched tableBucketCount (e.g. table-level vs partition-level bucket count in a multiply-bucketed table), legacy vs non-legacy timestamp bucketing (useLegacyTimestampBucketing mismatch), or files written/copied into the wrong bucket number.

Common situations: Data was written by Hive or an older Presto version with legacy timestamp bucketing and then read with the current setting; a user manually copied/renamed bucket files (000000_0 vs 000001_0) or concatenated files; a partition was created with bucket count N but files came from a table with a different bucket count; inserting into a bucketed table with a misconfigured hive.bucketing properties.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/dc7a2c7a61375dd7. Report an issue: GitHub.