What causes "Partition %s was added or modified during INSERT" in prestodb/presto?

Serialize writers to the same partition: ensure only one INSERT writes a given partition at a time (scheduling/locking).

prestodb/presto · error · PrestoException

HIVE_CONCURRENT_MODIFICATION_DETECTED

HIVE_CONCURRENT_MODIFICATION_DETECTED

Error message

Partition %s was added or modified during INSERT

What it means

When finishing a Hive INSERT, Presto merges per-worker PartitionUpdate records grouped by partition name. Each update in a group must agree on update mode, write path, and target path. A mismatch means the partition's layout changed while the query was writing — typically another writer added or modified the same partition concurrently — so Presto aborts with HIVE_CONCURRENT_MODIFICATION_DETECTED to avoid committing inconsistent data.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/PartitionUpdate.java:197

    public static List<PartitionUpdate> mergePartitionUpdates(Iterable<PartitionUpdate> unMergedUpdates)
    {
        ImmutableList.Builder<PartitionUpdate> partitionUpdates = ImmutableList.builder();
        for (Collection<PartitionUpdate> partitionGroup : Multimaps.index(unMergedUpdates, PartitionUpdate::getName).asMap().values()) {
            PartitionUpdate firstPartition = partitionGroup.iterator().next();

            ImmutableList.Builder<FileWriteInfo> allFileWriterInfos = ImmutableList.builder();
            long totalRowCount = 0;
            long totalInMemoryDataSizeInBytes = 0;
            long totalOnDiskDataSizeInBytes = 0;
            boolean containsNumberedFileNames = true;
            for (PartitionUpdate partition : partitionGroup) {
                // verify partitions have the same new flag, write path and target path
                // this shouldn't happen but could if another user added a partition during the write
                if (partition.getUpdateMode() != firstPartition.getUpdateMode() ||
                        !partition.getWritePath().equals(firstPartition.getWritePath()) ||
                        !partition.getTargetPath().equals(firstPartition.getTargetPath())) {
                    throw new PrestoException(HIVE_CONCURRENT_MODIFICATION_DETECTED, format("Partition %s was added or modified during INSERT", firstPartition.getName()));
                }
                allFileWriterInfos.addAll(partition.getFileWriteInfos());
                totalRowCount += partition.getRowCount();
                totalInMemoryDataSizeInBytes += partition.getInMemoryDataSizeInBytes();
                totalOnDiskDataSizeInBytes += partition.getOnDiskDataSizeInBytes();
                containsNumberedFileNames &= partition.containsNumberedFileNames();
            }

            partitionUpdates.add(new PartitionUpdate(firstPartition.getName(),
                    firstPartition.getUpdateMode(),
                    firstPartition.getWritePath(),
                    firstPartition.getTargetPath(),
                    allFileWriterInfos.build(),
                    totalRowCount,
                    totalInMemoryDataSizeInBytes,
                    totalOnDiskDataSizeInBytes,
                    containsNumberedFileNames));
        }

View on GitHub (pinned to 55bb57d202)

Solutions

Serialize writers to the same partition: ensure only one INSERT writes a given partition at a time (scheduling/locking).
Re-run the failed INSERT after the concurrent writer completes.
Use INSERT OVERWRITE or write to distinct partitions/table versions to avoid overlap.
Check for duplicate/overlapping scheduled jobs and stagger or partition their targets.

Example fix

// before: two jobs both run at 00:00 writing dt=today
jobA: INSERT INTO events PARTITION (dt='2026-09-04') ...
jobB: INSERT INTO events PARTITION (dt='2026-09-04') ...
// after: stagger or partition differently
jobB: INSERT INTO events PARTITION (dt='2026-09-04', src='jobB') ...  // or run jobB after jobA completes

Defensive patterns

Strategy: retry

Try / catch

try {
    executeInsert(sql);
} catch (PrestoException e) {
    if (e.getErrorCode() == HiveErrorCode.HIVE_CONCURRENT_MODIFICATION_DETECTED.toErrorCode().getId()) {
        // wait for the competing writer, then retry the insert
        retryAfterDelay(sql);
    }
}

Prevention

Serialize writes to the same partition via job scheduling or an external lock.
Ensure only one INSERT targets a given partition at a time in your ETL orchestration.
Use distinct partitions or INSERT OVERWRITE when parallel writers are unavoidable.

When it happens

Trigger: Two queries (or jobs) writing the same partition of the same table simultaneously so their PartitionUpdate entries for one partition disagree on updateMode/writePath/targetPath; a concurrent INSERT/ALTER/ADD PARTITION against the target partition during a long-running INSERT.

Common situations: Scheduled ETL jobs overlapping; multiple analysts INSERTing into the same daily partition at the same time; a backfill job racing a streaming write to one partition; stale metastore state letting two writers pick the same target partition.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/0d1bbd18bb23fb27. Report an issue: GitHub.