{"record":{"id":"f7df65fc822c5329","repo":"apache/flink","slug":"currently-only-block-sizes-up-to-integer-max-value","errorCode":null,"errorMessage":"Currently only block sizes up to Integer.MAX_VALUE are supported","messagePattern":"Currently only block sizes up to Integer\\.MAX_VALUE are supported","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/api/common/io/BinaryInputFormat.java","lineNumber":93,"sourceCode":"\n    /**\n     * The number of records already read from the block. This is used to decide if the end of the\n     * block has been reached.\n     */\n    private long readRecords = 0;\n\n    @Override\n    public void configure(Configuration parameters) {\n        super.configure(parameters);\n    }\n\n    public void setBlockSize(long blockSize) {\n        if (blockSize < 1 && blockSize != NATIVE_BLOCK_SIZE) {\n            throw new IllegalArgumentException(\n                    \"The block size parameter must be set and larger than 0.\");\n        }\n        if (blockSize > Integer.MAX_VALUE) {\n            throw new UnsupportedOperationException(\n                    \"Currently only block sizes up to Integer.MAX_VALUE are supported\");\n        }\n        this.blockSize = blockSize;\n    }\n\n    public long getBlockSize() {\n        return this.blockSize;\n    }\n\n    @Override\n    public FileInputSplit[] createInputSplits(int minNumSplits) throws IOException {\n        final List<FileStatus> files = this.getFiles();\n\n        final List<FileInputSplit> inputSplits = new ArrayList<FileInputSplit>(minNumSplits);\n        for (FileStatus file : files) {\n            final FileSystem fs = file.getPath().getFileSystem();\n            final long blockSize =\n                    this.blockSize == NATIVE_BLOCK_SIZE ? fs.getDefaultBlockSize() : this.blockSize;","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/common/io/BinaryInputFormat.java#L75-L111","documentation":"BinaryInputFormat stores and indexes block boundaries using int arithmetic internally, so it caps the block size at Integer.MAX_VALUE (~2 GB). setBlockSize throws UnsupportedOperationException for any value above Integer.MAX_VALUE even though the field is a long. This is a hard implementation limit of the legacy binary block format.","triggerScenarios":"Calling format.setBlockSize with a value > Integer.MAX_VALUE (e.g. 3L * 1024 * 1024 * 1024 for 3 GB, or any long exceeding 2147483647).","commonSituations":"Migrating to very large file blocks (multi-GB) typical of modern object stores; computing block size from a total file size and exceeding 2 GB; treating the long-typed parameter as supporting full 64-bit ranges.","solutions":["Use a block size <= Integer.MAX_VALUE (<= 2147483647 bytes, ~2 GB).","Use BinaryInputFormat.NATIVE_BLOCK_SIZE if you want the format to derive block size from the filesystem.","If you genuinely need > 2 GB blocks, switch to a modern source (FLIP-27 FileSource / flink-connector-files) that does not have this limit."],"exampleFix":"// before\nformat.setBlockSize(3L * 1024 * 1024 * 1024); // 3 GB -> throws\n\n// after\nformat.setBlockSize(Integer.MAX_VALUE); // ~2 GB ceiling\n// or\nformat.setBlockSize(BinaryInputFormat.NATIVE_BLOCK_SIZE);","handlingStrategy":"validation","validationCode":"long bs = configuredBlockSize;\nif (bs > Integer.MAX_VALUE) {\n    throw new IllegalArgumentException(\"block size must be <= Integer.MAX_VALUE (~2GB), got \" + bs);\n}\nformat.setBlockSize(bs);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Cap block size at Integer.MAX_VALUE before calling setBlockSize.","For files needing > 2 GB blocks, prefer the FLIP-27 FileSource.","Remember the long parameter does not imply full 64-bit support."],"tags":["input-format","binary","block-size","limit","unsupported-operation"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}