{"record":{"id":"d24dd66c6c650a0c","repo":"prestodb/presto","slug":"hive-exceeded-partition-limit","errorCode":"HIVE_EXCEEDED_PARTITION_LIMIT","errorMessage":"Query over table '%s' can potentially read more than %s partitions","messagePattern":"Query over table '(.+?)' can potentially read more than (.+?) partitions","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/HivePartitionManager.java","lineNumber":319,"sourceCode":"            return partitionPredicateBuilder.build();\n        }\n        else {\n            return ImmutableMap.of();\n        }\n    }\n\n    public HivePartitionResult getPartitions(SemiTransactionalHiveMetastore metastore, ConnectorTableHandle tableHandle, Constraint<ColumnHandle> constraint, ConnectorSession session)\n    {\n        HiveTableHandle hiveTableHandle = (HiveTableHandle) tableHandle;\n        TupleDomain<ColumnHandle> effectivePredicate = constraint.getSummary();\n\n        Table table = getTable(session, metastore, hiveTableHandle, isOfflineDataDebugModeEnabled(session));\n\n        List<HiveColumnHandle> partitionColumns = getPartitionKeyColumnHandles(table);\n\n        List<HivePartition> partitions = getPartitionsList(metastore, tableHandle, constraint, session);\n        if (partitions.size() > maxPartitionsPerScan) {\n            throw new PrestoException(HIVE_EXCEEDED_PARTITION_LIMIT, format(\n                    \"Query over table '%s' can potentially read more than %s partitions\",\n                    hiveTableHandle.getSchemaTableName().toString(),\n                    maxPartitionsPerScan));\n        }\n\n        Optional<HiveBucketHandle> hiveBucketHandle = getBucketHandle(table, session, effectivePredicate);\n        Optional<HiveBucketFilter> bucketFilter = hiveBucketHandle.flatMap(value -> getHiveBucketFilter(table, effectivePredicate, isLegacyTimestampBucketing(session)));\n\n        if (!queryUsesHiveBucketColumn(effectivePredicate)\n                && hiveBucketHandle.isPresent()\n                && queryAccessesTooManyBuckets(hiveBucketHandle.get(), bucketFilter, partitions, session)) {\n            hiveBucketHandle = Optional.empty();\n            bucketFilter = Optional.empty();\n        }\n\n        if (effectivePredicate.isNone()) {\n            return new HivePartitionResult(\n                    ImmutableList.copyOf(partitionColumns),","sourceCodeStart":301,"sourceCodeEnd":337,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/HivePartitionManager.java#L301-L337","documentation":"HivePartitionManager.getPartitions enumerates partitions matching the query constraint and enforces hive.max-partitions-per-scan (default 100). If the predicate resolves to more partitions than allowed, it throws HIVE_EXCEEDED_PARTITION_LIMIT to prevent runaway scans.","triggerScenarios":"getPartitions called during query planning; getPartitionsList(...) returns more partitions than maxPartitionsPerScan — e.g. no filter or an overly broad filter on partition columns.","commonSituations":"SELECT without a partition-key predicate on a large partitioned table, filter that doesn't prune partitions (function on partition column, non-partition column filter), querying many days/hours of a time-partitioned table.","solutions":["Add a selective predicate on partition columns (e.g. WHERE dt = '2026-09-04').","Raise the limit: set session property <catalog>.max_partitions_per_scan (or hive.max-partitions-per-scan in config) to a larger value.","Restructure the table to coarser partitions so fewer partitions match.","Split the query into multiple queries each covering fewer partitions."],"exampleFix":"-- before\nSELECT * FROM events; -- scans > 100 partitions\n-- after\nSET SESSION hive.max_partitions_per_scan = 10000;\n-- better: prune partitions\nSELECT * FROM events WHERE dt = '2026-09-04';","handlingStrategy":"validation","validationCode":"-- check partition count a query would touch before running\nSELECT count(DISTINCT dt) FROM events WHERE <your predicate>;\n-- compare against max partitions per scan\nSHOW SESSION LIKE '%max_partitions_per_scan%';","typeGuard":null,"tryCatchPattern":"catch (PrestoException e) {\n    if (\"HIVE_EXCEEDED_PARTITION_LIMIT\".equals(e.getErrorCode().getName())) {\n        // add partition predicate or raise max_partitions_per_scan, then retry\n    }\n}","preventionTips":["Always filter on partition columns in queries against partitioned tables.","Avoid wrapping partition columns in functions that defeat pruning.","Raise max-partitions-per-scan consciously; it exists to bound memory and metastore load.","Design partition schemes so typical queries touch fewer than the limit."],"tags":["hive","partitions","limit-exceeded","query-planning"],"backgroundTag":"max-partitions-per-scan-exceeded","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}