apache/seatunnel · warning

Partition already exists

Error message

Partition {} already exists

What it means

This is a warning logged by HiveMetaStoreCatalog.addPartitions when the Hive Metastore reports that a partition being added already exists (AlreadyExistsException from HMS appendPartition). The catalog treats this as a benign, idempotent outcome: the partition is already registered, so the loop continues with the next partition instead of failing the operation. It is not thrown to the caller; it only signals redundant partition registration.

Solutions

1. Treat this as informational: the partition exists and the operation is effectively idempotent; no code change is required.
2. If the warning is noisy, filter partitions before calling addPartitions by checking existing partitions via the catalog and only adding missing ones.
3. Deduplicate the partition list passed to addPartitions so each partition is appended only once per call.
4. If the partition exists but has wrong location/stats, drop it first with dropPartitions and re-add it rather than relying on appendPartition.

Example fix

// before
catalog.addPartitions(dbName, tableName, partitions);

// after
List<String> existing = catalog.listPartitionNames(dbName, tableName);
List<String> missing = partitions.stream()
        .filter(p -> !existing.contains(p))
        .collect(Collectors.toList());
catalog.addPartitions(dbName, tableName, missing);

Set<String> existing = new HashSet<>(catalog.listPartitionNames(dbName, tableName));
List<String> toAdd = partitions.stream().filter(p -> !existing.contains(p)).collect(Collectors.toList());
catalog.addPartitions(dbName, tableName, toAdd);

// Safe to ignore: the catalog itself catches AlreadyExistsException and logs a warn.
// No caller-side catch needed; treat the warning as an idempotent no-op.

When it happens

Trigger: Calling addPartitions(dbName, tableName, partitions) with partition strings (e.g. 'year=2024/month=01') that were already appended to the Hive table via a previous addPartitions call, an ALTER TABLE ADD PARTITION, or data written by another job that already registered the partition.

Common situations: Re-running a failed or completed batch job that re-registers the same partitions; two concurrent jobs writing to the same Hive table and both adding the same partition; a checkpoint restart replaying partition registration; manually creating partitions in Hive and then running a SeaTunnel job that also adds them.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

• CREATE_HIVE_TABLE_FAILED
• Database does not exist in catalog hive
• Failed to check if database exists
• Failed to create database
• Failed to create RetryingMetaStoreClient proxy, falling…

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/32230707de65edb1. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/utils/HiveMetaStoreCatalog.java:507

                    HiveConnectorErrorCode.CREATE_HIVE_TABLE_FAILED, errorMsg, e);
        } catch (Exception e) {
            throw new TException(
                    "Unexpected error creating table: "
                            + tbl.getDbName()
                            + "."
                            + tbl.getTableName(),
                    e);
        }
    }

    public void addPartitions(
            @NonNull String dbName, @NonNull String tableName, List<String> partitions)
            throws TException {
        for (String partition : partitions) {
            try {
                getClient().appendPartition(dbName, tableName, partition);
            } catch (AlreadyExistsException ae) {
                log.warn("Partition {} already exists", partition);
            }
        }
    }

    public void dropPartitions(
            @NonNull String dbName, @NonNull String tableName, List<String> partitions)
            throws TException {
        for (String partition : partitions) {
            getClient().dropPartition(dbName, tableName, partition, false);
        }
    }

    public boolean tableExists(@NonNull String dbName, @NonNull String tableName) {
        try {
            return getClient().tableExists(dbName, tableName);
        } catch (TException e) {
            String msg = String.format("Failed to check if table %s.%s exists", dbName, tableName);
            throw new HiveConnectorException(

View on GitHub (pinned to cf67b549a7)