{"record":{"id":"09c872b2cad3cbe1","repo":"apache/seatunnel","slug":"document-routing-requires-whole-file-splits-but-g","errorCode":null,"errorMessage":"Document routing requires whole-file splits, but got split ${split.splitId()}","messagePattern":"Document routing requires whole-file splits, but got split (.+?)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/FileSourceSplitEnumerator.java","lineNumber":163,"sourceCode":"                        .map(FileSourceSplit::splitId)\n                        .collect(Collectors.joining(\",\")));\n        context.signalNoMoreSplits(taskId);\n    }\n\n    private int getSplitOwner(FileSourceSplit split, int assignCount, int numReaders) {\n        if (documentRoutingEnabled) {\n            return getDocumentRouteOwner(split, numReaders);\n        }\n        return getRoundRobinSplitOwner(assignCount, numReaders);\n    }\n\n    private static int getRoundRobinSplitOwner(int assignCount, int numReaders) {\n        return assignCount % numReaders;\n    }\n\n    private static int getDocumentRouteOwner(FileSourceSplit split, int numReaders) {\n        if (split.getStart() != 0L || split.getLength() >= 0L) {\n            throw new IllegalStateException(\n                    \"Document routing requires whole-file splits, but got split \"\n                            + split.splitId());\n        }\n        String documentId = FileSourceDocumentRouting.buildDocumentId(split.getFilePath());\n        return FileSourceDocumentRouting.routeBucket(documentId, numReaders);\n    }\n\n    @Override\n    public int currentUnassignedSplitSize() {\n        return allSplit.size() - assignedSplit.size();\n    }\n\n    @Override\n    public void registerReader(int subtaskId) {\n        // do nothing\n    }\n\n    @Override","sourceCodeStart":145,"sourceCodeEnd":181,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/FileSourceSplitEnumerator.java#L145-L181","documentation":"When document routing is enabled, the split assigner routes whole files as atomic units so all splits of a document go to one reader. It requires splits with start=0 and length<0 (the whole-file marker). Any chunked/partial split triggers an IllegalStateException naming the offending split id.","triggerScenarios":"Document routing is active but the split generator produced offset-based chunk splits (start != 0 or length >= 0, e.g. because enable_file_split=true or a ParquetFileSplitStrategy split the file); getSplitOwner calls getDocumentRouteOwner and throws.","commonSituations":"Combining document-routing configuration with enable_file_split=true; a connector/format that always splits files (parquet row groups); version change where splits became chunked while routing options remained on.","solutions":["Disable file splitting (enable_file_split=false / remove file_split_size) so whole-file splits are produced.","Turn off document routing if per-chunk parallelism matters more than per-document ordering.","Ensure the split enumerator used generates whole-file FileSourceSplit instances (start=0, length=-1) when routing is on.","Check connector version/config mismatch that reintroduced chunked splits."],"exampleFix":"// before\nenable_file_split = true\nfile_split_size = 1048576\ndocument_routing = true\n// after\nenable_file_split = false\ndocument_routing = true","handlingStrategy":"try-catch","validationCode":"if (documentRoutingEnabled) require(split.getStart() == 0L && split.getLength() < 0L, \"document routing needs whole-file splits\");","typeGuard":"boolean isWholeFileSplit(FileSourceSplit s) { return s.getStart() == 0L && s.getLength() < 0L; }","tryCatchPattern":"try { owner = assigner.getSplitOwner(split, readers); } catch (IllegalStateException e) { log.error(\"Non whole-file split with document routing: {}\", e.getMessage()); throw e; }","preventionTips":["Do not combine document routing with enable_file_split=true.","Ensure the enumerator emits whole-file splits when routing is on.","Pin connector versions whose split behavior matches your routing config.","Add a startup assert that all splits are whole-file when routing is enabled."],"tags":["file-connector","split-assignment","document-routing","invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}