apache/seatunnel · critical · RuntimeException

Failed to discover remaining tables to capture

Error message

Failed to discover remaining tables to capture

What it means

Thrown by SnapshotSplitAssigner.open() when dialect.discoverDataCollections(sourceConfig) fails while enumerating the tables the CDC job should capture. It wraps the underlying exception (usually a JDBC/metadata-connection failure), so the root cause is the chained cause. This happens during split-assigner initialization, before any snapshot work starts.

Source

Thrown at seatunnel-connectors-v2/connector-cdc/connector-cdc-base/src/main/java/org/apache/seatunnel/connectors/cdc/base/source/enumerator/SnapshotSplitAssigner.java:162

        LOG.info(
                "SnapshotSplitAssigner created with assigned splits: {}",
                this.assignedSplits.keySet());
    }

    @Override
    public void open() {
        chunkSplitter = dialect.createChunkSplitter(sourceConfig);

        // the legacy state didn't snapshot remaining tables, discovery remaining table here
        if (!isRemainingTablesCheckpointed && !assignerCompleted) {
            try {
                final List<TableId> discoverTables = dialect.discoverDataCollections(sourceConfig);
                context.getCapturedTables().addAll(discoverTables);
                discoverTables.removeAll(alreadyProcessedTables);
                this.remainingTables.addAll(discoverTables);
                this.isTableIdCaseSensitive = dialect.isDataCollectionIdCaseSensitive(sourceConfig);
            } catch (Exception e) {
                throw new RuntimeException("Failed to discover remaining tables to capture", e);
            }
        }
    }

    @Override
    public Optional<SourceSplitBase> getNext() {
        if (chunkSplitter == null) {
            return Optional.empty();
        }
        if (!remainingSplits.isEmpty()) {
            // return remaining splits firstly
            Iterator<SnapshotSplit> iterator = remainingSplits.iterator();
            SnapshotSplit split = iterator.next();
            iterator.remove();
            assignedSplits.put(split.splitId(), split);
            context.getAssignedSnapshotSplit().put(split.splitId(), split);
            return Optional.of(split);
        } else {

View on GitHub (pinned to cf67b549a7)

Solutions

1. Inspect the wrapped cause exception in the stack trace to identify the real failure (connection vs query vs auth)
2. Verify database connectivity (host, port, credentials) with a plain JDBC client or mysql/psql
3. Check that the table include/exclude patterns and database names in the connector config are valid and match at least one table
4. Grant the CDC user permission to read metadata (e.g. SELECT on information_schema / SHOW DATABASES)
5. Retry once the database is reachable if this was a transient outage

Example fix

// before (job fails with wrapped discovery error)
// source:
//   tables = "orders_*"   (pattern typo: no tables matched / invalid query)
// after:
//   tables = "order.*"    (correct regex, table discovery succeeds)

// Before submitting the CDC job:
// mysql -h $HOST -P $PORT -u $USER -p -e "SELECT 1 FROM information_schema.tables LIMIT 1;"
// Fail fast if DB unreachable or user lacks metadata privileges.
try (Connection c = DriverManager.getConnection(url, user, pass)) {
    c.createStatement().execute("SELECT 1 FROM information_schema.tables LIMIT 1");
}

try { assigner.open(); } catch (RuntimeException e) {
    LOG.error("Table discovery failed; root cause: {}", e.getCause(), e);
    throw e;
}

When it happens

Trigger: open() calls dialect.discoverDataCollections(sourceConfig) and any exception (JDBC connect failure, bad credentials, table-listing query failure, malformed table-name config) is wrapped and rethrown as RuntimeException.

Common situations: Wrong hostname/port in the database config; credentials lacking metadata-read privileges; database unreachable or firewall-blocked; include/exclude table patterns matching nothing or invalid syntax; database server temporarily down at job start.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• Failed to discover captured tables for enumerator
• Error to discover tables:
• Error to check tables:
• Failed to disable auto commit for Db2 CDC connection
• Couldn't get timestamp utils from underlying connection

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/752be28e7b753b03. Report an issue: GitHub.