{"record":{"id":"bbb21e6ec8bd16ab","repo":"prestodb/presto","slug":"hive-partition-dropped-during-query-bbb21e","errorCode":"HIVE_PARTITION_DROPPED_DURING_QUERY","errorMessage":"Statistics result does not contain entry for partition: ${partitionName}","messagePattern":"Statistics result does not contain entry for partition: (.+?)","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"warning","filePath":"presto-hive-metastore/src/main/java/com/facebook/presto/hive/metastore/file/FileHiveMetastore.java","lineNumber":367,"sourceCode":"\n        Path tableMetadataDirectory = getTableMetadataDirectory(databaseName, tableName);\n        TableMetadata tableMetadata = readSchemaFile(\"table\", tableMetadataDirectory, tableCodec)\n                .orElseThrow(() -> new TableNotFoundException(new SchemaTableName(databaseName, tableName)));\n\n        TableMetadata updatedMetadata = tableMetadata\n                .withParameters(updateStatisticsParameters(tableMetadata.getParameters(), updatedStatistics.getBasicStatistics()))\n                .withColumnStatistics(updatedStatistics.getColumnStatistics());\n\n        writeSchemaFile(\"table\", tableMetadataDirectory, tableCodec, updatedMetadata, true);\n    }\n\n    @Override\n    public synchronized void updatePartitionStatistics(MetastoreContext metastoreContext, String databaseName, String tableName, Map<String, Function<PartitionStatistics, PartitionStatistics>> updates)\n    {\n        updates.forEach((partitionName, update) -> {\n            PartitionStatistics originalStatistics = getPartitionStatistics(metastoreContext, databaseName, tableName, ImmutableSet.of(partitionName)).get(partitionName);\n            if (originalStatistics == null) {\n                throw new PrestoException(HIVE_PARTITION_DROPPED_DURING_QUERY, \"Statistics result does not contain entry for partition: \" + partitionName);\n            }\n            PartitionStatistics updatedStatistics = update.apply(originalStatistics);\n\n            Table table = getRequiredTable(metastoreContext, databaseName, tableName);\n            List<String> partitionValues = extractPartitionValues(partitionName);\n            Path partitionDirectory = getPartitionMetadataDirectory(table, partitionValues);\n            PartitionMetadata partitionMetadata = readSchemaFile(\"partition\", partitionDirectory, partitionCodec)\n                    .orElseThrow(() -> new PartitionNotFoundException(new SchemaTableName(databaseName, tableName), partitionValues));\n\n            PartitionMetadata updatedMetadata = partitionMetadata\n                    .withParameters(updateStatisticsParameters(partitionMetadata.getParameters(), updatedStatistics.getBasicStatistics()))\n                    .withColumnStatistics(updatedStatistics.getColumnStatistics());\n\n            writeSchemaFile(\"partition\", partitionDirectory, partitionCodec, updatedMetadata, true);\n        });\n    }\n\n    @Override","sourceCodeStart":349,"sourceCodeEnd":385,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive-metastore/src/main/java/com/facebook/presto/hive/metastore/file/FileHiveMetastore.java#L349-L385","documentation":"updatePartitionStatistics re-reads current partition statistics and updates them; if a partition's statistics entry cannot be found (the partition was dropped concurrently or never had stats), it throws HIVE_PARTITION_DROPPED_DURING_QUERY. This signals the partition vanished between the query plan and the stats update.","triggerScenarios":"Calling updatePartitionStatistics for a partitionName not returned by getPartitionStatistics — typically because another session dropped the partition (or table) while the query was running.","commonSituations":"Long-running INSERT/ANALYZE queries on partitions concurrently deleted by ETL cleanup; DROP PARTITION racing a query that computes stats; stale partition names after repartitioning.","solutions":["Retry the operation; if the partition was intentionally dropped, the lost stats update can be ignored.","Coordinate ETL jobs so partition drops don't race stats-updating queries (locks or scheduling).","Verify the partition still exists (getPartitionNames/getPartition) before updating its statistics."],"exampleFix":"// before\nmetastore.updatePartitionStatistics(metastoreContext, db, table, updateFunctions); // throws if partition gone\n// after\nSet<String> existing = metastore.getPartitionNames(metastoreContext, db, table)\n        .orElse(ImmutableSet.of());\nMap<String, Function<PartitionStatistics, PartitionStatistics>> live = updateFunctions.keySet().stream()\n        .filter(existing::contains)\n        .collect(toImmutableMap(fn -> fn, updateFunctions::get));\nif (!live.isEmpty()) {\n    metastore.updatePartitionStatistics(metastoreContext, db, table, live);\n}","handlingStrategy":"try-catch","validationCode":"Set<String> existing = metastore.getPartitionNames(metastoreContext, db, table).orElse(ImmutableSet.of());\n// only include partitions still present in 'existing' in the update map","typeGuard":null,"tryCatchPattern":"try {\n    metastore.updatePartitionStatistics(metastoreContext, db, table, updates);\n} catch (PrestoException e) {\n    if (e.getErrorCode() == HIVE_PARTITION_DROPPED_DURING_QUERY.toErrorCode()) {\n        // partition dropped concurrently: drop its update entry and retry for the rest\n    } else { throw e; }\n}","preventionTips":["Filter update keys against currently existing partitions.","Avoid concurrent DROP PARTITION and ANALYZE/INSERT on the same partitions.","Treat this error as benign when the drop was intentional."],"tags":["hive","metastore","partition","statistics","concurrency"],"backgroundTag":"partition-dropped-during-query","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}