apache/seatunnel · error · ClickhouseConnectorException

GET_PRIMARY_KEY_FAILED

GET_PRIMARY_KEY_FAILED

Error message

Cannot get primary key from clickhouse

What it means

ClickhouseProxy.getPrimaryKey queries primary-key metadata (via system tables/JDBC metadata) and wraps ClickHouseException into a ClickhouseConnectorException with GET_PRIMARY_KEY_FAILED. It means the connector could not determine which columns form the primary/replica key, which is required for key-based writes or dedup.

Source

Thrown at seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/util/ClickhouseProxy.java:407

                "SELECT\n"
                        + "    name as column_name\n"
                        + "FROM system.columns\n"
                        + "WHERE table = '"
                        + table
                        + "'\n"
                        + "  AND database = '"
                        + schema
                        + "'\n"
                        + "  AND is_in_primary_key = 1\n"
                        + "ORDER BY position;";
        try (ClickHouseResponse response = clickhouseRequest.query(sql).executeAndWait()) {
            Iterable<ClickHouseRecord> records = response.records();
            pkFields =
                    StreamSupport.stream(records.spliterator(), false)
                            .map(r -> r.getValue(0).asString())
                            .collect(Collectors.toList());
        } catch (ClickHouseException e) {
            throw new ClickhouseConnectorException(
                    SeaTunnelAPIErrorCode.GET_PRIMARY_KEY_FAILED,
                    "Cannot get primary key from clickhouse",
                    e);
        }
        if (!pkFields.isEmpty()) {
            // PK_NAME maybe null according to the javadoc, generate a unique name in that case
            String pkName = "pk_" + String.join("_", pkFields);
            return Optional.of(PrimaryKey.of(pkName, pkFields));
        }
        return Optional.empty();
    }

    public boolean isExistsData(String tableName) throws ExecutionException, InterruptedException {
        String queryDataSql = "SELECT count(*) FROM " + tableName;
        try (ClickHouseResponse response = clickhouseRequest.query(queryDataSql).executeAndWait()) {
            return response.firstRecord().getValue(0).asInteger() > 0;
        } catch (ClickHouseException e) {
            throw new ClickhouseConnectorException(

View on GitHub (pinned to cf67b549a7)

Solutions

1. Verify the table engine actually defines a primary key: SHOW CREATE TABLE <table>;
2. Check the wrapped ClickHouseException cause for auth/connection issues
3. If the table has no PK, configure the connector's key fields explicitly instead of relying on auto-detection
4. Grant the user access to the metadata tables used for key lookup

Example fix

// before
List<String> pk = proxy.getPrimaryKey("default", "log_events"); // Log engine, no PK
// after
List<String> pk = proxy.getPrimaryKey("default", "events"); -- MergeTree ORDER BY (id, ts)

// only auto-detect PK on engines that support keys (MergeTree family)
ClickhouseTable t = proxy.getClickhouseTable(request, database, table);
boolean supportsPk = t.getEngine().contains("MergeTree");

boolean engineSupportsPk = engine != null && (engine.startsWith("MergeTree") || engine.startsWith("ReplicatedMergeTree") || engine.startsWith("Replacing"));

try {
    List<String> pk = proxy.getPrimaryKey(database, table);
    if (pk.isEmpty()) { /* configure key fields explicitly */ }
} catch (ClickhouseConnectorException e) {
    if (SeaTunnelAPIErrorCode.GET_PRIMARY_KEY_FAILED.equals(e.getSeaTunnelAPIErrorCode())) {
        LOG.warn("PK lookup failed for {}.{}; falling back to configured key fields", database, table);
    }
    throw e;
}

When it happens

Trigger: Calling getPrimaryKey when the metadata query throws ClickHouseException (connection/auth/permission failure) or the underlying metadata lookup fails for the target table.

Common situations: Table uses an engine without a primary key (e.g. plain Log/StripeLog) and the metadata query errors; user lacks privileges for key metadata lookups; connection dropped during the query.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• Failed to get primary key for table {}.{}, will create table
• Failed getting table %s
• TABLE_NOT_EXISTED
• TABLE_SCHEMA_GET_FAILED
• CLUSTER_LIST_GET_FAILED

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/6155cfbbdb0f4692. Report an issue: GitHub.