{"record":{"id":"80f1d93d0b93ff38","repo":"apache/seatunnel","slug":"cross-partitions-upsert-dynamic-bucket-mode-is-not","errorCode":null,"errorMessage":"Cross Partitions Upsert Dynamic Bucket Mode is not supported.","messagePattern":"Cross Partitions Upsert Dynamic Bucket Mode is not supported\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-paimon/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSinkWriter.java","lineNumber":171,"sourceCode":"        this.paimonBucketAssignerFactory = paimonBucketAssignerFactory;\n        this.parallelism = context.getNumberOfParallelSubtasks();\n        this.taskIndex = context.getIndexOfSubtask();\n        this.paimonSinkConfig = paimonSinkConfig;\n        this.sinkPaimonTableSchema = this.paimonTable.schema();\n        this.ioManager =\n                (IOManagerImpl)\n                        IOManager.create(splitPaths(paimonSinkConfig.getChangelogTmpPath()));\n        this.newTableWrite();\n        BucketMode bucketMode = this.paimonTable.bucketMode();\n        // https://paimon.apache.org/docs/master/primary-key-table/data-distribution/#dynamic-bucket\n        // When you need cross partition upsert (primary keys not contain all partition fields),\n        // Dynamic Bucket mode directly maintains the mapping of keys to partition and bucket, uses\n        // local disks, and initializes indexes by reading all existing keys in the table when\n        // starting job. For tables with a large amount of data, there will be a significant loss in\n        // performance. Moreover, initialization takes a long time. This mode is not supported at\n        // this time.\n        if (BucketMode.CROSS_PARTITION == bucketMode) {\n            throw new UnsupportedOperationException(\n                    \"Cross Partitions Upsert Dynamic Bucket Mode is not supported.\");\n        }\n        this.dynamicBucket = BucketMode.HASH_DYNAMIC == bucketMode;\n        int bucket = paimonTable.coreOptions().bucket();\n        if (bucket == -1 && BucketMode.BUCKET_UNAWARE == bucketMode) {\n            log.warn(\"Append only table currently do not support dynamic bucket\");\n        }\n        if (dynamicBucket) {\n            paimonBucketAssignerFactory.init(paimonTablePath, paimonFileStoretable, parallelism);\n        }\n        PaimonSecurityContext.shouldEnableKerberos(paimonHadoopConfiguration);\n    }\n\n    public PaimonSinkWriter(\n            Context context,\n            ReadonlyConfig readonlyConfig,\n            CatalogTable catalogTable,\n            Table paimonFileStoretable,","sourceCodeStart":153,"sourceCodeEnd":189,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-paimon/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSinkWriter.java#L153-L189","documentation":"PaimonSinkWriter rejects the CROSS_PARTITION bucket mode with UnsupportedOperationException. Cross-partition upsert dynamic-bucket mode requires maintaining a key-to-partition/bucket index over all existing keys, which the connector does not support due to performance and initialization cost.","triggerScenarios":"Writing with primary keys to a partitioned Paimon table whose inferred/computed BucketMode is CROSS_PARTITION — typically a partitioned table with primary keys and bucket = -1 (dynamic bucket) so keys span partitions.","commonSituations":"User sets bucket = -1 on a partitioned primary-key table expecting auto-scaling buckets; migrating a Paimon Flink job config that used cross-partition upsert into SeaTunnel.","solutions":["Set an explicit positive `bucket` value in the sink table options so a fixed-bucket hash mode is used","Remove primary keys / use append mode if cross-partition dynamic upsert is truly needed (use a non-partitioned layout or include partition keys in the primary key)","Define primary keys that contain the partition columns so records map within a single partition","Pre-create the table with a supported bucket mode instead of letting the connector auto-infer"],"exampleFix":"// before\ntable_options = { bucket = \"-1\" } // dynamic bucket on partitioned pk table\n// after\ntable_options = { bucket = \"4\" }\n","handlingStrategy":"validation","validationCode":"// Before writing, avoid partitioned + primary-key + bucket=-1 combos:\nboolean partitioned = !paimonTable.partitionKeys().isEmpty();\nboolean hasPk = !paimonTable.primaryKeys().isEmpty();\nint bucket = paimonTable.coreOptions().bucket();\nif (partitioned && hasPk && bucket == -1) { throw new IllegalArgumentException(\"Use an explicit bucket count or include partition keys in the primary key\"); }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Set an explicit `bucket` value for partitioned primary-key tables","Include partition columns in primary keys when upserting across partitions","Pre-create tables with a supported bucket mode"],"tags":["paimon","sink","unsupported"],"backgroundTag":"unsupported-operation","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}