{"record":{"id":"97e27f8f989927fa","repo":"prestodb/presto","slug":"failed-to-read-d-bytes-from-file-s-d-bytes-re","errorCode":null,"errorMessage":"Failed to read %d bytes from file %s : %d bytes read.","messagePattern":"Failed to read (.+?) bytes from file (.+?) : (.+?) bytes read\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"presto-iceberg/src/main/java/com/facebook/presto/iceberg/HdfsCachedInputFile.java","lineNumber":108,"sourceCode":"    }\n\n    private static ManifestFileCachedContent readFully(InputFile input, long fileLength, long chunkSize)\n            throws IOException\n    {\n        try (SeekableInputStream stream = input.newStream()) {\n            long totalBytesToRead = fileLength;\n            List<ByteBuffer> buffers = new ArrayList<>(\n                    ((int) (fileLength / chunkSize)) +\n                            (fileLength % chunkSize == 0 ? 0 : 1));\n\n            while (totalBytesToRead > 0) {\n                int bytesToRead = (int) Math.min(chunkSize, totalBytesToRead);\n                byte[] buf = new byte[bytesToRead];\n                int bytesRead = readRemaining(stream, buf, 0, bytesToRead);\n                totalBytesToRead -= bytesRead;\n\n                if (bytesRead < bytesToRead) {\n                    throw new IOException(\n                            format(\"Failed to read %d bytes from file %s : %d bytes read.\",\n                                    fileLength, input.location(), fileLength - totalBytesToRead));\n                }\n                else {\n                    buffers.add(ByteBuffer.wrap(buf));\n                }\n            }\n            return new ManifestFileCachedContent(buffers, fileLength);\n        }\n    }\n}\n","sourceCodeStart":90,"sourceCodeEnd":120,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-iceberg/src/main/java/com/facebook/presto/iceberg/HdfsCachedInputFile.java#L90-L120","documentation":"HdfsCachedInputFile.readFully reads a file in chunkSize chunks into buffers. If a read returns fewer bytes than requested before the expected total is consumed, it throws an IOException stating how many of the requested bytes were actually read. This surfaces as an under-read from the underlying HDFS/object stream — the file is shorter or the stream failed mid-read.","triggerScenarios":"Calling readFully (e.g. while reading Iceberg metadata/manifest/footer bytes through the cache layer) when the underlying stream ends early: file truncated, concurrent overwrite/delete of the file, or HDFS/network read failure returning short reads that readRemaining cannot fill.","commonSituations":"Files truncated by failed writes, objects deleted or overwritten concurrently by compaction/cleanup jobs, network faults to HDFS or S3, misreported file length (cached length larger than actual object size).","solutions":["Verify the file exists and its actual size matches the length the InputFile reports (ls/stat the path).","Retry the read — transient HDFS/network failures often resolve; clear any stale caches.","Re-generate the file: rewrite manifests/metadata (rewrite_manifests) or restore from snapshot backup if truncated.","Check for concurrent writers deleting/overwriting the file; freeze table maintenance during reads."],"exampleFix":"// before: cached length stale after truncation\nInputFile f = new HdfsCachedInputFile(...); // length from old metadata\n// after: validate length before reading\nlong actual = environment.getFileSystem(context, path).getFileStatus(path).getLen();\nif (actual < expectedLength) { throw new PrestoException(ICEBERG_FILESYSTEM_ERROR, \"File truncated: \" + path); }","handlingStrategy":"retry","validationCode":"FileStatus st = fs.getFileStatus(path);\nif (st.getLen() < expectedLength) throw new PrestoException(ICEBERG_FILESYSTEM_ERROR, \"File truncated: \" + path);","typeGuard":"boolean isReadable(InputFile f) { try { return f.length() > 0 && f.newStream().read() >= 0; } catch (IOException e) { return false; } }","tryCatchPattern":"try { stream.readFully(buffer); } catch (IOException e) { if (isTransient(e)) retryWithBackoff(); else throw new PrestoException(ICEBERG_FILESYSTEM_ERROR, \"under-read: \" + path, e); }","preventionTips":["Avoid deleting/overwriting files while readers are active","Use read-committed snapshot pinning so readers see immutable files","Retain files longer than longest query duration (expiration policies)","Retry short reads with backoff — many are transient network faults"],"tags":["io","hdfs","short-read","cache"],"backgroundTag":"short-read-io","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}