{"record":{"id":"f099c3d5f5f7bd63","repo":"prestodb/presto","slug":"invalid-file-size-s","errorCode":null,"errorMessage":"Invalid file size %s","messagePattern":"Invalid file size (.+?)","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/cache/StorageOrcFileTailSource.java","lineNumber":72,"sourceCode":"    public StorageOrcFileTailSource()\n    {\n        this(EXPECTED_FOOTER_SIZE_IN_BYTES, false);\n    }\n\n    public StorageOrcFileTailSource(int expectedFooterSizeInBytes, boolean dwrfStripeCacheEnabled)\n    {\n        checkArgument(expectedFooterSizeInBytes >= MINIMUM_TAIL_SIZE_IN_BYTES, \"expectedFooterSize %s is less than minimum supported\", expectedFooterSizeInBytes);\n        this.expectedFooterSizeInBytes = expectedFooterSizeInBytes;\n        this.dwrfStripeCacheEnabled = dwrfStripeCacheEnabled;\n    }\n\n    @Override\n    public OrcFileTail getOrcFileTail(OrcDataSource orcDataSource, MetadataReader metadataReader, Optional<OrcWriteValidation> writeValidation, boolean cacheable, long fileModificationTime)\n            throws IOException\n    {\n        long size = orcDataSource.getSize();\n        if (size <= MAGIC.length()) {\n            throw new OrcCorruptionException(orcDataSource.getId(), \"Invalid file size %s\", size);\n        }\n\n        // Read the tail of the file\n        byte[] buffer = new byte[toIntExact(min(size, expectedFooterSizeInBytes))];\n        orcDataSource.readFully(size - buffer.length, buffer);\n\n        // get length of PostScript - last byte of the file\n        int postScriptSize = buffer[buffer.length - SIZE_OF_BYTE] & 0xff;\n        if (postScriptSize >= buffer.length) {\n            throw new OrcCorruptionException(orcDataSource.getId(), \"Invalid postscript length %s\", postScriptSize);\n        }\n\n        // decode the post script\n        PostScript postScript;\n        try {\n            postScript = metadataReader.readPostScript(buffer, buffer.length - SIZE_OF_BYTE - postScriptSize, postScriptSize);\n        }\n        catch (OrcCorruptionException e) {","sourceCodeStart":54,"sourceCodeEnd":90,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/cache/StorageOrcFileTailSource.java#L54-L90","documentation":"This library throws OrcCorruptionException when an ORC data source is too small to be a valid ORC file. A file must at least contain the 3-byte magic header plus a postscript byte, so a file of size <= MAGIC.length() (3) cannot hold the ORC file tail. StorageOrcFileTailSource.getOrcFileTail checks this before reading the tail bytes.","triggerScenarios":"Calling getOrcFileTail (directly or via orcFileTail) on an OrcDataSource whose getSize() is 0, 1, 2, or 3 bytes, e.g. a truncated, empty, or placeholder file passed to the ORC tail-source.","commonSituations":"Reading a file still being written (footer not flushed), a truncated download/copy, an empty file created by a failed job, or pointing at a non-ORC placeholder file (e.g. _SUCCESS, .crc).","solutions":["Verify the file exists and has a non-trivial size (ls -l / hdfs dfs -stat) before reading; re-copy or re-download if truncated.","Confirm the file is a real ORC file (starts with 'ORC' and has a postscript); re-run the writer job if the file is incomplete.","Wait for the writer to finish/commit before querying files from the output directory.","If the file is intentionally empty, skip it rather than reading it as ORC."],"exampleFix":"// before\nOrcFileTail tail = fileTailSource.getOrcFileTail(orcDataSource, metadataReader, Optional.empty(), false, 0);\n// after\nif (orcDataSource.getSize() <= 3) {\n    log.warn(\"Skipping %s: not a valid ORC file (size=%s)\", orcDataSource.getId(), orcDataSource.getSize());\n    return;\n}\nOrcFileTail tail = fileTailSource.getOrcFileTail(orcDataSource, metadataReader, Optional.empty(), false, 0);","handlingStrategy":"validation","validationCode":"if (orcDataSource.getSize() <= OrcFile.Magic.length()) {\n    throw new SkipFileException(\"File too small to be ORC: \" + orcDataSource.getId());\n}","typeGuard":null,"tryCatchPattern":"try { tail = source.getOrcFileTail(ds, reader, Optional.empty(), false, time); }\ncatch (OrcCorruptionException e) { log.warn(\"Skipping unreadable ORC file %s\", ds.getId(), e); }","preventionTips":["Check file size > 3 bytes before opening as ORC","Only read files from committed job output directories (after _SUCCESS)","Skip zero-byte files produced by failed tasks"],"tags":["orc","corrupt-file","file-size"],"backgroundTag":"orc-file-corruption","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}