apache/seatunnel · error · IllegalStateException

Document routing requires whole-file splits, but got split $

Error message

Document routing requires whole-file splits, but got split ${split.splitId()}

What it means

When document routing is enabled, the split assigner routes whole files as atomic units so all splits of a document go to one reader. It requires splits with start=0 and length<0 (the whole-file marker). Any chunked/partial split triggers an IllegalStateException naming the offending split id.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/FileSourceSplitEnumerator.java:163

                        .map(FileSourceSplit::splitId)
                        .collect(Collectors.joining(",")));
        context.signalNoMoreSplits(taskId);
    }

    private int getSplitOwner(FileSourceSplit split, int assignCount, int numReaders) {
        if (documentRoutingEnabled) {
            return getDocumentRouteOwner(split, numReaders);
        }
        return getRoundRobinSplitOwner(assignCount, numReaders);
    }

    private static int getRoundRobinSplitOwner(int assignCount, int numReaders) {
        return assignCount % numReaders;
    }

    private static int getDocumentRouteOwner(FileSourceSplit split, int numReaders) {
        if (split.getStart() != 0L || split.getLength() >= 0L) {
            throw new IllegalStateException(
                    "Document routing requires whole-file splits, but got split "
                            + split.splitId());
        }
        String documentId = FileSourceDocumentRouting.buildDocumentId(split.getFilePath());
        return FileSourceDocumentRouting.routeBucket(documentId, numReaders);
    }

    @Override
    public int currentUnassignedSplitSize() {
        return allSplit.size() - assignedSplit.size();
    }

    @Override
    public void registerReader(int subtaskId) {
        // do nothing
    }

    @Override

View on GitHub (pinned to cf67b549a7)

Solutions

1. Disable file splitting (enable_file_split=false / remove file_split_size) so whole-file splits are produced.
2. Turn off document routing if per-chunk parallelism matters more than per-document ordering.
3. Ensure the split enumerator used generates whole-file FileSourceSplit instances (start=0, length=-1) when routing is on.
4. Check connector version/config mismatch that reintroduced chunked splits.

Example fix

// before
enable_file_split = true
file_split_size = 1048576
document_routing = true
// after
enable_file_split = false
document_routing = true

if (documentRoutingEnabled) require(split.getStart() == 0L && split.getLength() < 0L, "document routing needs whole-file splits");

boolean isWholeFileSplit(FileSourceSplit s) { return s.getStart() == 0L && s.getLength() < 0L; }

try { owner = assigner.getSplitOwner(split, readers); } catch (IllegalStateException e) { log.error("Non whole-file split with document routing: {}", e.getMessage()); throw e; }

When it happens

Trigger: Document routing is active but the split generator produced offset-based chunk splits (start != 0 or length >= 0, e.g. because enable_file_split=true or a ParquetFileSplitStrategy split the file); getSplitOwner calls getDocumentRouteOwner and throws.

Common situations: Combining document-routing configuration with enable_file_split=true; a connector/format that always splits files (parquet row groups); version change where splits became chunked while routing options remained on.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Document routing requires whole-file splits, but got split $
• Cannot fetch from another split - no split remaining.
• The update before event at for table was not followed by a
• there should always be a valid xlog position
• SHOULD_NEVER_HAPPEN

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/09c872b2cad3cbe1. Report an issue: GitHub.