{"record":{"id":"43434163ae4f4b8e","repo":"prestodb/presto","slug":"hive-invalid-bucket-files-434341","errorCode":"HIVE_INVALID_BUCKET_FILES","errorMessage":"A row that is supposed to be in bucket %s is encountered. Only rows in bucket %s (modulo %s) are expected","messagePattern":"A row that is supposed to be in bucket (.+?) is encountered\\. Only rows in bucket (.+?) \\(modulo (.+?)\\) are expected","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/HivePageSource.java","lineNumber":325,"sourceCode":"            this.bucketColumns = bucketAdaptation.getBucketColumnIndices();\n            this.bucketToKeep = bucketAdaptation.getBucketToKeep();\n            this.typeInfoList = bucketAdaptation.getBucketColumnHiveTypes().stream()\n                    .map(HiveType::getTypeInfo)\n                    .collect(toImmutableList());\n            this.tableBucketCount = bucketAdaptation.getTableBucketCount();\n            this.partitionBucketCount = bucketAdaptation.getPartitionBucketCount();\n            this.useLegacyTimestampBucketing = bucketAdaptation.useLegacyTimestampBucketing();\n        }\n\n        @Nullable\n        public Page filterPageToEligibleRowsOrDiscard(Page page)\n        {\n            IntArrayList ids = new IntArrayList(page.getPositionCount());\n            Page bucketColumnsPage = page.extractChannels(bucketColumns);\n            for (int position = 0; position < page.getPositionCount(); position++) {\n                int bucket = getHiveBucket(tableBucketCount, typeInfoList, bucketColumnsPage, position, useLegacyTimestampBucketing);\n                if ((bucket - bucketToKeep) % partitionBucketCount != 0) {\n                    throw new PrestoException(HIVE_INVALID_BUCKET_FILES, format(\n                            \"A row that is supposed to be in bucket %s is encountered. Only rows in bucket %s (modulo %s) are expected\",\n                            bucket, bucketToKeep % partitionBucketCount, partitionBucketCount));\n                }\n                if (bucket == bucketToKeep) {\n                    ids.add(position);\n                }\n            }\n            int retainedRowCount = ids.size();\n            if (retainedRowCount == 0) {\n                return null; // Empty page after filtering\n            }\n            if (retainedRowCount == page.getPositionCount()) {\n                return page; // Unchanged after filtering\n            }\n            Block[] adaptedBlocks = new Block[page.getChannelCount()];\n            for (int i = 0; i < adaptedBlocks.length; i++) {\n                Block block = page.getBlock(i);\n                if (block instanceof LazyBlock && !((LazyBlock) block).isLoaded()) {","sourceCodeStart":307,"sourceCodeEnd":343,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/HivePageSource.java#L307-L343","documentation":"When reading a bucketed table, HivePageSource filters pages to only rows of the bucket this split should keep. If a row's computed bucket is not congruent to bucketToKeep modulo partitionBucketCount, the split contains files from wrong buckets, so it throws HIVE_INVALID_BUCKET_FILES. This indicates corrupted or mis-bucketed files in the table.","triggerScenarios":"filterPageToEligibleRowsOrDiscard computes getHiveBucket(...) for a row and (bucket - bucketToKeep) % partitionBucketCount != 0 — i.e. the reader received a page whose rows belong to another bucket than the split expects.","commonSituations":"Buckets written with a different bucketing version (legacy vs modern Hive bucketing, hive.legacy bucketing flag mismatch), files copied/misnamed after writes, manual file operations corrupting bucket-to-file mapping, table migrated between Hive versions.","solutions":["Align the bucketing mode: set hive.legacy bucketing / bucketing_version session or table property to match how the data was written.","Check the table's bucketing_version in metastore and rewrite the affected partitions with the current engine.","Identify and fix misnamed/misplaced bucket files in the listed partition.","If the mismatch is only from a connector bug, upgrade Presto/Hive connector version."],"exampleFix":"-- before (data written with old bucketing, read with new)\nSELECT * FROM bucketed_table WHERE ...;\n-- after\nSET SESSION hive.legacy_bucketing = true;\nSELECT * FROM bucketed_table WHERE ...;","handlingStrategy":"validation","validationCode":"-- verify bucketing metadata matches data layout\nSHOW CREATE TABLE bucketed_table; -- check bucketing_version / bucket_count\n-- confirm files per partition match bucket count\nSELECT \"$path\" FROM bucketed_table WHERE dt = '2026-09-04';","typeGuard":null,"tryCatchPattern":"catch (PrestoException e) {\n    if (\"HIVE_INVALID_BUCKET_FILES\".equals(e.getErrorCode().getName())) {\n        // retry with legacy bucketing session flag, or repair the partition\n    }\n}","preventionTips":["Never manually add/rename files in bucketed tables.","Keep bucketing_version consistent when migrating tables between Hive versions.","Always write bucketed tables through the engine, never external ETL that ignores bucket numbering.","Test bucketed reads after any table migration or compaction."],"tags":["hive","bucketing","corrupt-data","reader"],"backgroundTag":"invalid-bucket-files","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}