{"record":{"id":"dc7a2c7a61375dd7","repo":"prestodb/presto","slug":"hive-invalid-bucket-files","errorCode":"HIVE_INVALID_BUCKET_FILES","errorMessage":"A row that is supposed to be in bucket %s is encountered. Only rows in bucket %s (modulo %s) are expected","messagePattern":"A row that is supposed to be in bucket (.+?) is encountered\\. Only rows in bucket (.+?) \\(modulo (.+?)\\) are expected","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/HiveBucketAdapterRecordCursor.java","lineNumber":129,"sourceCode":"                else if (javaType == long.class) {\n                    scratch[i] = delegate.getLong(index);\n                }\n                else if (javaType == double.class) {\n                    scratch[i] = delegate.getDouble(index);\n                }\n                else if (javaType == Slice.class) {\n                    scratch[i] = delegate.getSlice(index);\n                }\n                else if (javaType == Block.class) {\n                    scratch[i] = (Block) delegate.getObject(index);\n                }\n                else {\n                    throw new UnsupportedOperationException(\"unknown java type\");\n                }\n            }\n            int bucket = HiveBucketing.getHiveBucket(tableBucketCount, typeInfoList, scratch, useLegacyTimestampBucketing);\n            if ((bucket - bucketToKeep) % partitionBucketCount != 0) {\n                throw new PrestoException(HIVE_INVALID_BUCKET_FILES, format(\n                        \"A row that is supposed to be in bucket %s is encountered. Only rows in bucket %s (modulo %s) are expected\",\n                        bucket, bucketToKeep % partitionBucketCount, partitionBucketCount));\n            }\n            if (bucket == bucketToKeep) {\n                return true;\n            }\n        }\n    }\n\n    @Override\n    public boolean getBoolean(int field)\n    {\n        return delegate.getBoolean(field);\n    }\n\n    @Override\n    public long getLong(int field)\n    {","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/HiveBucketAdapterRecordCursor.java#L111-L147","documentation":"HiveBucketAdapterRecordCursor wraps a RecordCursor to re-bucket rows: it recomputes each row's original bucket via HiveBucketing.getHiveBucket and keeps only rows that map to the requested bucket (directly or modulo partitionBucketCount, the multiply-bucketed split case). This PrestoException (HIVE_INVALID_BUCKET_FILES) is thrown in advanceNextPosition when a row's computed bucket is not congruent to bucketToKeep modulo partitionBucketCount, meaning the underlying file contains rows that do not belong to the bucket this file was supposed to hold — the bucketing of the file's data does not match its declared bucket.","triggerScenarios":"Reading a Hive table/partition whose bucket files were produced with a different bucketing hash, a different bucket column ordering, mismatched tableBucketCount (e.g. table-level vs partition-level bucket count in a multiply-bucketed table), legacy vs non-legacy timestamp bucketing (useLegacyTimestampBucketing mismatch), or files written/copied into the wrong bucket number.","commonSituations":"Data was written by Hive or an older Presto version with legacy timestamp bucketing and then read with the current setting; a user manually copied/renamed bucket files (000000_0 vs 000001_0) or concatenated files; a partition was created with bucket count N but files came from a table with a different bucket count; inserting into a bucketed table with a misconfigured hive.bucketing properties.","solutions":["Verify the table/partition's bucketing metadata (bucket columns, bucket count, and any hive.bucketing.* session/property settings, especially useLegacyTimestampBucketing) matches how the files were actually written, and fix the mismatch.","Use the correct legacy-mode session property (e.g. set the legacy timestamp bucketing flag) when reading data written by older Hive/Presto versions.","Rebuild/rewrite the affected partition (INSERT OVERWRITE or CTAS) so bucket files are regenerated by the engine from the declared bucketing spec.","Check for manually copied, renamed, or concatenated bucket files and restore files so each file number matches the bucket its rows hash into."],"exampleFix":"// before: reading data written by legacy Hive bucketing fails\nSELECT * FROM bucketed_table;\n\n// after: match the writer's bucketing mode for the session\nSET SESSION use_legacy_timestamp_bucketing = true;\nSELECT * FROM bucketed_table;","handlingStrategy":"validation","validationCode":"-- before querying: confirm bucketing metadata matches the data on disk\nSHOW CREATE TABLE bucketed_table;\n-- check: bucketed-by columns exist, bucket count per partition matches file layout,\n-- and legacy timestamp bucketing mode matches how data was written.\n-- Optionally verify a bucket file's rows re-hash to its bucket number in a scratch query.","typeGuard":null,"tryCatchPattern":"try { ResultSet rs = stmt.executeQuery(\"SELECT * FROM bucketed_table\"); ... }\ncatch (SQLException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"HIVE_INVALID_BUCKET_FILES\")) {\n    // re-write the partition or fix legacy-bucketing session settings before retrying\n  } else throw e;\n}","preventionTips":["Never manually copy, rename, or concatenate bucket files; always rewrite partitions via INSERT OVERWRITE.","Keep hive bucketing session properties (legacy timestamp bucketing) consistent across the cluster that wrote and the cluster that reads.","Verify partition bucket counts match the table spec when using multiply-bucketed tables.","After any external Hive-CLI schema/layout change, run a validation query against one partition before production reads."],"tags":["hive","bucketing","data-corruption","presto"],"backgroundTag":"bucket-file-mismatch","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}