What causes "Shard key not found in table " in apache/seatunnel?

Set sharding_key to an exact existing column name of the table (match case)

apache/seatunnel · error · ClickhouseConnectorException

CLICKHOUSE_SHARD_KEY_NOT_FOUND

CLICKHOUSE_SHARD_KEY_NOT_FOUND

Error message

Shard key  not found in table

What it means

The configured sharding_key does not exist in the target ClickHouse table's column metadata, so the connector cannot determine its type for shard routing. ShardRouter validates that a non-empty shard key resolves to a known shard key type and fails with CLICKHOUSE_SHARD_KEY_NOT_FOUND otherwise.

Source

Thrown at seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/client/ShardRouter.java:69

    private final TreeMap<Integer, Shard> shards;
    private final String shardKey;
    private final String shardKeyType;
    @Getter private final String sortingKey;
    private final boolean splitMode;

    private static final XXHash64 HASH_INSTANCE = XXHashFactory.fastestInstance().hash64();
    private final ThreadLocalRandom threadLocalRandom = ThreadLocalRandom.current();

    public ShardRouter(ClickhouseProxy proxy, ShardMetadata shardMetadata) {
        this.shards = new TreeMap<>();
        this.shardKey = shardMetadata.getShardKey();
        this.shardKeyType = shardMetadata.getShardKeyType();
        this.sortingKey = shardMetadata.getSortingKey();
        this.splitMode = shardMetadata.isSplitMode();
        this.table = shardMetadata.getTable();
        this.tableEngine = shardMetadata.getTableEngine();
        if (StringUtils.isNotEmpty(shardKey) && StringUtils.isEmpty(shardKeyType)) {
            throw new ClickhouseConnectorException(
                    ClickhouseConnectorErrorCode.SHARD_KEY_NOT_FOUND,
                    "Shard key " + shardKey + " not found in table " + table);
        }
        ClickHouseRequest<?> connection = proxy.getClickhouseConnection();
        if (splitMode) {
            DistributedEngine localTable =
                    proxy.getClickhouseDistributedTable(
                            connection, shardMetadata.getDatabase(), table);
            this.shardTable = localTable.getTable();
            this.shardTableEngine = localTable.getTableEngine();
            List<Shard> shardList =
                    proxy.getClusterShardList(
                            connection,
                            localTable.getClusterName(),
                            localTable.getDatabase(),
                            shardMetadata.getDefaultShard().getNode().getPort(),
                            shardMetadata.getUsername(),
                            shardMetadata.getPassword(),

View on GitHub (pinned to cf67b549a7)

Solutions

Set sharding_key to an exact existing column name of the table (match case)
Remove sharding_key to let ClickHouse route with its default sharding expression
If using a shard expression like rand()/hash, verify what the table's actual sharding column is and reference that column

Example fix

// before
sharding_key = "usr_id"
// after
sharding_key = "user_id"

Defensive patterns

Strategy: validation

Validate before calling

Set<String> cols = clickhouse.tableColumns(database, table);
String shardKey = config.getOptional(SHARDING_KEY).orElse(null);
if (shardKey != null && !cols.contains(shardKey))
  throw new IllegalArgumentException("sharding_key not a column: " + shardKey);

Try / catch

try { createSinkWriter(config); } catch (ClickhouseConnectorException e) {
  if (e.getCode().equals(ClickhouseConnectorErrorCode.SHARD_KEY_NOT_FOUND)) {
    /* correct or remove sharding_key */ }
}

Prevention

Match the sharding_key string exactly (including case) to a table column
Re-check sharding_key after any table DDL (rename/drop of columns)
Use the table's real sharding column, not an arbitrary expression

When it happens

Trigger: Configuring sharding_key = "some_col" where some_col is not a column of the ClickHouse table (typo, wrong case, or key defined only on an upstream schema); raised in the ShardRouter constructor.

Common situations: Copy-pasted config from another table, column renamed/dropped in ClickHouse, case-sensitivity mismatch, or using an expression (e.g. rand(), cityHash64(...)) that is not a plain column when the metadata lookup expects one.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/8f579a43e423ca90. Report an issue: GitHub.