apache/seatunnel · error · UnsupportedOperationException

Cross Partitions Upsert Dynamic Bucket Mode is not supported

Error message

Cross Partitions Upsert Dynamic Bucket Mode is not supported.

What it means

PaimonSinkWriter rejects the CROSS_PARTITION bucket mode with UnsupportedOperationException. Cross-partition upsert dynamic-bucket mode requires maintaining a key-to-partition/bucket index over all existing keys, which the connector does not support due to performance and initialization cost.

Source

Thrown at seatunnel-connectors-v2/connector-paimon/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSinkWriter.java:171

        this.paimonBucketAssignerFactory = paimonBucketAssignerFactory;
        this.parallelism = context.getNumberOfParallelSubtasks();
        this.taskIndex = context.getIndexOfSubtask();
        this.paimonSinkConfig = paimonSinkConfig;
        this.sinkPaimonTableSchema = this.paimonTable.schema();
        this.ioManager =
                (IOManagerImpl)
                        IOManager.create(splitPaths(paimonSinkConfig.getChangelogTmpPath()));
        this.newTableWrite();
        BucketMode bucketMode = this.paimonTable.bucketMode();
        // https://paimon.apache.org/docs/master/primary-key-table/data-distribution/#dynamic-bucket
        // When you need cross partition upsert (primary keys not contain all partition fields),
        // Dynamic Bucket mode directly maintains the mapping of keys to partition and bucket, uses
        // local disks, and initializes indexes by reading all existing keys in the table when
        // starting job. For tables with a large amount of data, there will be a significant loss in
        // performance. Moreover, initialization takes a long time. This mode is not supported at
        // this time.
        if (BucketMode.CROSS_PARTITION == bucketMode) {
            throw new UnsupportedOperationException(
                    "Cross Partitions Upsert Dynamic Bucket Mode is not supported.");
        }
        this.dynamicBucket = BucketMode.HASH_DYNAMIC == bucketMode;
        int bucket = paimonTable.coreOptions().bucket();
        if (bucket == -1 && BucketMode.BUCKET_UNAWARE == bucketMode) {
            log.warn("Append only table currently do not support dynamic bucket");
        }
        if (dynamicBucket) {
            paimonBucketAssignerFactory.init(paimonTablePath, paimonFileStoretable, parallelism);
        }
        PaimonSecurityContext.shouldEnableKerberos(paimonHadoopConfiguration);
    }

    public PaimonSinkWriter(
            Context context,
            ReadonlyConfig readonlyConfig,
            CatalogTable catalogTable,
            Table paimonFileStoretable,

View on GitHub (pinned to cf67b549a7)

Solutions

1. Set an explicit positive `bucket` value in the sink table options so a fixed-bucket hash mode is used
2. Remove primary keys / use append mode if cross-partition dynamic upsert is truly needed (use a non-partitioned layout or include partition keys in the primary key)
3. Define primary keys that contain the partition columns so records map within a single partition
4. Pre-create the table with a supported bucket mode instead of letting the connector auto-infer

Example fix

// before
table_options = { bucket = "-1" } // dynamic bucket on partitioned pk table
// after
table_options = { bucket = "4" }

// Before writing, avoid partitioned + primary-key + bucket=-1 combos:
boolean partitioned = !paimonTable.partitionKeys().isEmpty();
boolean hasPk = !paimonTable.primaryKeys().isEmpty();
int bucket = paimonTable.coreOptions().bucket();
if (partitioned && hasPk && bucket == -1) { throw new IllegalArgumentException("Use an explicit bucket count or include partition keys in the primary key"); }

When it happens

Trigger: Writing with primary keys to a partitioned Paimon table whose inferred/computed BucketMode is CROSS_PARTITION — typically a partitioned table with primary keys and bucket = -1 (dynamic bucket) so keys span partitions.

Common situations: User sets bucket = -1 on a partitioned primary-key table expecting auto-scaling buckets; migrating a Paimon Flink job config that used cross-partition upsert into SeaTunnel.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Unsupported data format type:
• ILLEGAL_ARGUMENT
• DONT_SUPPORT_SINK
• JdbcConnectorErrorCode.DONT_SUPPORT_SINK
• BRANCH_NOT_EXISTS

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/80f1d93d0b93ff38. Report an issue: GitHub.