prestodb/presto · error · PrestoException

HIVE_EXCEEDED_PARTITION_LIMIT

HIVE_EXCEEDED_PARTITION_LIMIT

Error message

Query over table '%s' can potentially read more than %s partitions

What it means

HivePartitionManager.getPartitions enumerates partitions matching the query constraint and enforces hive.max-partitions-per-scan (default 100). If the predicate resolves to more partitions than allowed, it throws HIVE_EXCEEDED_PARTITION_LIMIT to prevent runaway scans.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/HivePartitionManager.java:319

            return partitionPredicateBuilder.build();
        }
        else {
            return ImmutableMap.of();
        }
    }

    public HivePartitionResult getPartitions(SemiTransactionalHiveMetastore metastore, ConnectorTableHandle tableHandle, Constraint<ColumnHandle> constraint, ConnectorSession session)
    {
        HiveTableHandle hiveTableHandle = (HiveTableHandle) tableHandle;
        TupleDomain<ColumnHandle> effectivePredicate = constraint.getSummary();

        Table table = getTable(session, metastore, hiveTableHandle, isOfflineDataDebugModeEnabled(session));

        List<HiveColumnHandle> partitionColumns = getPartitionKeyColumnHandles(table);

        List<HivePartition> partitions = getPartitionsList(metastore, tableHandle, constraint, session);
        if (partitions.size() > maxPartitionsPerScan) {
            throw new PrestoException(HIVE_EXCEEDED_PARTITION_LIMIT, format(
                    "Query over table '%s' can potentially read more than %s partitions",
                    hiveTableHandle.getSchemaTableName().toString(),
                    maxPartitionsPerScan));
        }

        Optional<HiveBucketHandle> hiveBucketHandle = getBucketHandle(table, session, effectivePredicate);
        Optional<HiveBucketFilter> bucketFilter = hiveBucketHandle.flatMap(value -> getHiveBucketFilter(table, effectivePredicate, isLegacyTimestampBucketing(session)));

        if (!queryUsesHiveBucketColumn(effectivePredicate)
                && hiveBucketHandle.isPresent()
                && queryAccessesTooManyBuckets(hiveBucketHandle.get(), bucketFilter, partitions, session)) {
            hiveBucketHandle = Optional.empty();
            bucketFilter = Optional.empty();
        }

        if (effectivePredicate.isNone()) {
            return new HivePartitionResult(
                    ImmutableList.copyOf(partitionColumns),

View on GitHub (pinned to 55bb57d202)

Solutions

Add a selective predicate on partition columns (e.g. WHERE dt = '2026-09-04').
Raise the limit: set session property <catalog>.max_partitions_per_scan (or hive.max-partitions-per-scan in config) to a larger value.
Restructure the table to coarser partitions so fewer partitions match.
Split the query into multiple queries each covering fewer partitions.

Example fix

-- before
SELECT * FROM events; -- scans > 100 partitions
-- after
SET SESSION hive.max_partitions_per_scan = 10000;
-- better: prune partitions
SELECT * FROM events WHERE dt = '2026-09-04';

Defensive patterns

Strategy: validation

Validate before calling

-- check partition count a query would touch before running
SELECT count(DISTINCT dt) FROM events WHERE <your predicate>;
-- compare against max partitions per scan
SHOW SESSION LIKE '%max_partitions_per_scan%';

Try / catch

catch (PrestoException e) {
    if ("HIVE_EXCEEDED_PARTITION_LIMIT".equals(e.getErrorCode().getName())) {
        // add partition predicate or raise max_partitions_per_scan, then retry
    }
}

Prevention

Always filter on partition columns in queries against partitioned tables.
Avoid wrapping partition columns in functions that defeat pruning.
Raise max-partitions-per-scan consciously; it exists to bound memory and metastore load.
Design partition schemes so typical queries touch fewer than the limit.

When it happens

Trigger: getPartitions called during query planning; getPartitionsList(...) returns more partitions than maxPartitionsPerScan — e.g. no filter or an overly broad filter on partition columns.

Common situations: SELECT without a partition-key predicate on a large partitioned table, filter that doesn't prune partitions (function on partition column, non-partition column filter), querying many days/hours of a time-partitioned table.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/d24dd66c6c650a0c. Report an issue: GitHub.