apache/seatunnel · error · IllegalStateException

Document routing requires whole-file splits, but got split $

Error message

Document routing requires whole-file splits, but got split ${split.splitId()}

What it means

Same whole-file invariant as the base enumerator: with document routing enabled, splits must be whole-file (start=0, length<0) because routing buckets documents, not chunks. The multiple-table split enumerator's getDocumentRouteOwner throws IllegalStateException if a partial split arrives.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/MultipleTableFileSourceSplitEnumerator.java:242

                + ",...("
                + (splits.size() - LOG_SPLIT_ID_LIMIT)
                + " more)";
    }

    private int getSplitOwner(FileSourceSplit split, int assignCount, int numReaders) {
        if (documentRoutingTableIds.contains(split.getTableId())) {
            return getDocumentRouteOwner(split, numReaders);
        }
        return getRoundRobinSplitOwner(assignCount, numReaders);
    }

    private static int getRoundRobinSplitOwner(int assignCount, int numReaders) {
        return assignCount % numReaders;
    }

    private static int getDocumentRouteOwner(FileSourceSplit split, int numReaders) {
        if (split.getStart() != 0L || split.getLength() >= 0L) {
            throw new IllegalStateException(
                    "Document routing requires whole-file splits, but got split "
                            + split.splitId());
        }
        String documentId = FileSourceDocumentRouting.buildDocumentId(split.getFilePath());
        return FileSourceDocumentRouting.routeBucket(documentId, numReaders);
    }

    @Override
    public void run() throws Exception {
        for (int i = 0; i < context.currentParallelism(); i++) {
            log.info("Assigned splits to reader [{}]", i);
            synchronized (lock) {
                assignSplit(i);
            }
        }
    }

    @Override

View on GitHub (pinned to cf67b549a7)

Solutions

1. Disable file splitting in all tables when document routing is enabled.
2. Disable document routing if chunk-level parallel assignment is the goal.
3. Make the enumerator produce whole-file splits (start=0, length=-1) for routing mode.
4. Audit each table block for enable_file_split/file_split_size settings.

Example fix

// before
table1 { enable_file_split = true }
document_routing = true
// after
table1 { enable_file_split = false }
document_routing = true

if (documentRoutingEnabled) require(split.getStart() == 0L && split.getLength() < 0L, "document routing needs whole-file splits");

boolean isWholeFileSplit(FileSourceSplit s) { return s.getStart() == 0L && s.getLength() < 0L; }

try { owner = assigner.getSplitOwner(split, readers); } catch (IllegalStateException e) { log.error("Whole-file split required for document routing, got: {}", e.getMessage()); throw e; }

When it happens

Trigger: Multi-table file source with document routing enabled, but splits were chunked (enable_file_split=true or a splitting strategy active); getSplitOwner delegates to getDocumentRouteOwner and the start/length check fails for the given split id.

Common situations: Mixed config where one table enabled file split; upgrade changed default splitting behavior; copying single-table split config into a routing-enabled multi-table job.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Document routing requires whole-file splits, but got split $
• CATALOG_TABLE_SIZE_IS_ERROR
• MultiTableWriterRunnable can't find writer for tableId:
• Cannot fetch from another split - no split remaining.
• Elasticsearch multi-table writer requires ElasticsearchMulti

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/4d9a5a5767eb9a4a. Report an issue: GitHub.