{"record":{"id":"1fde7df3ff8fde42","repo":"prestodb/presto","slug":"hive-metastore-error-1fde7d","errorCode":"HIVE_METASTORE_ERROR","errorMessage":"Failed to fetch partitions from Glue Data Catalog","messagePattern":"Failed to fetch partitions from Glue Data Catalog","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive-metastore/src/main/java/com/facebook/presto/hive/metastore/glue/GlueHiveMetastore.java","lineNumber":1030,"sourceCode":"        // Do parallel partition fetch.\n        CompletionService<List<Partition>> completionService = new ExecutorCompletionService<>(partitionsReadExecutor);\n        for (int i = 0; i < partitionSegments; i++) {\n            Segment segment = Segment.builder().segmentNumber(i).totalSegments(partitionSegments).build();\n            completionService.submit(() -> getPartitions(databaseName, tableName, expression, segment));\n        }\n\n        List<Partition> partitions = new ArrayList<>();\n        try {\n            for (int i = 0; i < partitionSegments; i++) {\n                Future<List<Partition>> futurePartitions = completionService.take();\n                partitions.addAll(futurePartitions.get());\n            }\n        }\n        catch (ExecutionException | InterruptedException e) {\n            if (e instanceof InterruptedException) {\n                Thread.currentThread().interrupt();\n            }\n            throw new PrestoException(HIVE_METASTORE_ERROR, \"Failed to fetch partitions from Glue Data Catalog\", e);\n        }\n\n        partitions.sort(PARTITION_COMPARATOR);\n        return partitions;\n    }\n\n    private List<Partition> getPartitions(String databaseName, String tableName, String expression, @Nullable Segment segment)\n    {\n        try {\n            GluePartitionConverter converter = new GluePartitionConverter(databaseName, tableName);\n\n            ImmutableList.Builder<Partition> partitionBuilder = ImmutableList.builder();\n\n            GetPartitionsRequest partitionsRequest = GetPartitionsRequest.builder()\n                    .catalogId(catalogId)\n                    .databaseName(databaseName)\n                    .tableName(tableName)\n                    .expression(expression)","sourceCodeStart":1012,"sourceCodeEnd":1048,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive-metastore/src/main/java/com/facebook/presto/hive/metastore/glue/GlueHiveMetastore.java#L1012-L1048","documentation":"GlueHiveMetastore wraps ExecutionException/InterruptedException raised by the parallel (executor-based) GetPartitions calls into HIVE_METASTORE_ERROR 'Failed to fetch partitions from Glue Data Catalog'. It re-establishes the interrupt flag when interrupted and preserves the cause. This indicates the concurrent partition-listing task failed — usually a Glue throttling/ServiceUnavailable error, credential problem, or executor shutdown.","triggerScenarios":"getPartitions/getAllPartitions on a table with many partitions where parallel Glue GetPartitions calls fail — Glue throttling (ThrottlingException/TooManyRequests), AWS credentials expired or IAM glue:GetPartitions denied, task executor rejected/cancelled, or thread interruption during shutdown.","commonSituations":"Large partitioned tables hitting Glue API rate limits; short-lived assumed-role credentials expiring mid-query; Presto shutdown or query cancellation interrupting the fetch; AWS region outage.","solutions":["Inspect the cause: if it's Glue throttling, reduce partition fetch concurrency (glue.partition-generator concurrency settings) or request a Glue quota increase.","Refresh/fix AWS credentials and verify IAM policy grants glue:GetPartitions (and glue:GetTables).","Retry the query after transient AWS errors; enable the connector's retry/backoff settings for Glue clients.","Filter partitions (partition predicates) or use partition pruning metadata caching to reduce the number of GetPartitions calls."],"exampleFix":"// before (diagnosis)\nList<String> names = metastore.getPartitionNames(context, db, table).orElseThrow(...);\n// after\ntry {\n    List<String> names = metastore.getPartitionNames(context, db, table).orElseThrow(...);\n}\ncatch (PrestoException e) {\n    Throwable cause = e.getCause();\n    if (cause instanceof ExecutionException && isThrottling(cause)) { backoffAndRetry(); } else { throw e; }\n}","handlingStrategy":"retry","validationCode":"// Verify Glue access before fetching:\n// awsCredentials valid, region correct, IAM policy includes glue:GetPartitions.","typeGuard":null,"tryCatchPattern":"try {\n    partitions = metastore.getPartitions(ctx, table, columnNames, parts, domainCompactionThreshold);\n}\ncatch (PrestoException e) {\n    if (e.getErrorCode().getName().equals(\"HIVE_METASTORE_ERROR\") && isThrottlingOrTransient(e.getCause())) {\n        partitions = retryWithBackoff();\n    } else { throw e; }\n}","preventionTips":["Reduce Glue fetch concurrency or raise API rate limits for large partitioned tables.","Use credential refresh/short-lived role assumption with adequate session duration.","Cache partition metadata and apply partition filters to limit GetPartitions calls.","Retry on transient AWS errors with exponential backoff."],"tags":["glue","aws","partitions","throttling","network"],"backgroundTag":"aws-throttling","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}