prestodb/presto · error · PrestoException

HIVE_METASTORE_ERROR

HIVE_METASTORE_ERROR

Error message

Failed to fetch partitions from Glue Data Catalog

What it means

GlueHiveMetastore wraps ExecutionException/InterruptedException raised by the parallel (executor-based) GetPartitions calls into HIVE_METASTORE_ERROR 'Failed to fetch partitions from Glue Data Catalog'. It re-establishes the interrupt flag when interrupted and preserves the cause. This indicates the concurrent partition-listing task failed — usually a Glue throttling/ServiceUnavailable error, credential problem, or executor shutdown.

Source

Thrown at presto-hive-metastore/src/main/java/com/facebook/presto/hive/metastore/glue/GlueHiveMetastore.java:1030

        // Do parallel partition fetch.
        CompletionService<List<Partition>> completionService = new ExecutorCompletionService<>(partitionsReadExecutor);
        for (int i = 0; i < partitionSegments; i++) {
            Segment segment = Segment.builder().segmentNumber(i).totalSegments(partitionSegments).build();
            completionService.submit(() -> getPartitions(databaseName, tableName, expression, segment));
        }

        List<Partition> partitions = new ArrayList<>();
        try {
            for (int i = 0; i < partitionSegments; i++) {
                Future<List<Partition>> futurePartitions = completionService.take();
                partitions.addAll(futurePartitions.get());
            }
        }
        catch (ExecutionException | InterruptedException e) {
            if (e instanceof InterruptedException) {
                Thread.currentThread().interrupt();
            }
            throw new PrestoException(HIVE_METASTORE_ERROR, "Failed to fetch partitions from Glue Data Catalog", e);
        }

        partitions.sort(PARTITION_COMPARATOR);
        return partitions;
    }

    private List<Partition> getPartitions(String databaseName, String tableName, String expression, @Nullable Segment segment)
    {
        try {
            GluePartitionConverter converter = new GluePartitionConverter(databaseName, tableName);

            ImmutableList.Builder<Partition> partitionBuilder = ImmutableList.builder();

            GetPartitionsRequest partitionsRequest = GetPartitionsRequest.builder()
                    .catalogId(catalogId)
                    .databaseName(databaseName)
                    .tableName(tableName)
                    .expression(expression)

View on GitHub (pinned to 55bb57d202)

Solutions

1. Inspect the cause: if it's Glue throttling, reduce partition fetch concurrency (glue.partition-generator concurrency settings) or request a Glue quota increase.
2. Refresh/fix AWS credentials and verify IAM policy grants glue:GetPartitions (and glue:GetTables).
3. Retry the query after transient AWS errors; enable the connector's retry/backoff settings for Glue clients.
4. Filter partitions (partition predicates) or use partition pruning metadata caching to reduce the number of GetPartitions calls.

Example fix

// before (diagnosis)
List<String> names = metastore.getPartitionNames(context, db, table).orElseThrow(...);
// after
try {
    List<String> names = metastore.getPartitionNames(context, db, table).orElseThrow(...);
}
catch (PrestoException e) {
    Throwable cause = e.getCause();
    if (cause instanceof ExecutionException && isThrottling(cause)) { backoffAndRetry(); } else { throw e; }
}

// Verify Glue access before fetching:
// awsCredentials valid, region correct, IAM policy includes glue:GetPartitions.

try {
    partitions = metastore.getPartitions(ctx, table, columnNames, parts, domainCompactionThreshold);
}
catch (PrestoException e) {
    if (e.getErrorCode().getName().equals("HIVE_METASTORE_ERROR") && isThrottlingOrTransient(e.getCause())) {
        partitions = retryWithBackoff();
    } else { throw e; }
}

When it happens

Trigger: getPartitions/getAllPartitions on a table with many partitions where parallel Glue GetPartitions calls fail — Glue throttling (ThrottlingException/TooManyRequests), AWS credentials expired or IAM glue:GetPartitions denied, task executor rejected/cancelled, or thread interruption during shutdown.

Common situations: Large partitioned tables hitting Glue API rate limits; short-lived assumed-role credentials expiring mid-query; Presto shutdown or query cancellation interrupting the fetch; AWS region outage.

Related errors

• HIVE_PARTITION_NOT_FOUND
• NOT_SUPPORTED
• ALREADY_EXISTS
• ARROW_FLIGHT_CLIENT_ERROR
• Failed to resolve host:

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/1fde7df3ff8fde42. Report an issue: GitHub.