apache/seatunnel · error · ClickhouseConnectorException

Failed to read data from part %s, shard: %s, splitId: %s, me

Error message

Failed to read data from part %s, shard: %s, splitId: %s, message: %s

What it means

Thrown by ClickhouseValueReader.partBatchStrategyRead() when any exception occurs while reading rows from a specific ClickHouse part. The connector wraps the underlying failure (JDBC/driver/network/parsing) in a ClickhouseConnectorException with QUERY_DATA_ERROR, attaching the part name, shard node, split id, and original message. It means data could not be read from that part during the part-based batch read strategy.

Source

Thrown at seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/source/ClickhouseValueReader.java:161

            if (rowBatch.isEmpty()) {
                currentPart.setEndOfPart(true);
                currentPartIndex++;
                return currentPartIndex < partSize && partBatchStrategyRead();
            }

            // update Keyset cursor (last ordering key values)
            String sortingKey = clickhouseSourceTable.getClickhouseTable().getSortingKey();

            SeaTunnelRow lastRow = rowBatch.get(rowBatch.size() - 1);
            List<Object> keyValues = extractOrderingKeyValuesFromRow(lastRow, sortingKey);
            log.debug("lastRow: {}, extract ordering key values from row: {}", lastRow, keyValues);

            currentPart.setLastOrderingKeyValues(keyValues);

            return true;
        } catch (Exception e) {
            throw new ClickhouseConnectorException(
                    ClickhouseConnectorErrorCode.QUERY_DATA_ERROR,
                    String.format(
                            "Failed to read data from part %s, shard: %s, splitId: %s, message: %s",
                            currentPart.getName(),
                            currentPart.getShard().getNode(),
                            clickhouseSourceSplit.getSplitId(),
                            e.getMessage()),
                    e);
        }
    }

    private boolean sqlBatchStrategyRead() {
        String query = buildBatchSqlQuery();

        try {
            rowBatch =
                    proxy.batchFetchRecords(
                            query, clickhouseSourceTable.getTablePath(), rowTypeInfo);

View on GitHub (pinned to cf67b549a7)

Solutions

Verify the shard node in the error message is reachable and healthy (clickhouse-client against that node)
Check the ClickHouse user has SELECT grants on the target table and the part's database
Re-run the job — transient network/shard failures often resolve on retry; enable checkpointing so the split is re-read
Inspect the wrapped 'message' field for the root cause (driver/SQL error) and fix that underlying issue

Example fix

// before
String sql = "SELECT * FROM " + tablePath;
// after
String sql = "SELECT * FROM " + tablePath + " SETTINGS max_execution_time = 300";
// plus verify connectivity: clickhouse-client --host <shardNode> -q 'SELECT 1'

Defensive patterns

Strategy: retry

Validate before calling

// Pre-check shard connectivity and grants before running the job
try (Connection c = DriverManager.getConnection(jdbcUrlOnShard, user, pass)) {
    ResultSet rs = c.createStatement().executeQuery("SELECT 1");
    if (!rs.next()) throw new IllegalStateException("Shard not answering");
}

Try / catch

try {
    rows = reader.next();
} catch (ClickhouseConnectorException e) {
    if (e.getErrorCode() == ClickhouseConnectorErrorCode.QUERY_DATA_ERROR
        && e.getMessage().startsWith("Failed to read data from part")) {
        // inspect wrapped cause; retry with backoff or skip to next split via checkpoint recovery
    }
    throw e;
}

Prevention

Monitor shard health and run the job only when all shards respond
Grant the ClickHouse user SELECT on all source tables before submitting the job
Enable checkpointing so interrupted part reads resume from the last committed position
Set sensible JDBC/socket timeouts matching part sizes

When it happens

Trigger: partBatchStrategyRead() executes the per-part SELECT query against the shard node and any Exception is raised: connection failure to the shard, SQL syntax/schema mismatch, driver errors, or failure while extracting ordering key values from the last row.

Common situations: Shard node temporarily unreachable or restarted mid-read; ClickHouse user lacks SELECT permission on the part's table; schema changed so ordering key extraction fails; network timeouts on large parts.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/f86b3481576f2650. Report an issue: GitHub.