{"record":{"id":"e56873cc75205e7b","repo":"prestodb/presto","slug":"druid-deep-storage-error-e56873","errorCode":"DRUID_DEEP_STORAGE_ERROR","errorMessage":"Error reading from %s at position %s","messagePattern":"Error reading from (.+?) at position (.+?)","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-druid/src/main/java/com/facebook/presto/druid/segment/HdfsDataInputSource.java","lineNumber":84,"sourceCode":"        readFully(position, buffer, 0, buffer.length);\n    }\n\n    @Override\n    public void readFully(long position, byte[] buffer, int bufferOffset, int bufferLength)\n    {\n        long start = System.nanoTime();\n        readInternal(position, buffer, bufferOffset, bufferLength);\n\n        readTimeNanos += System.nanoTime() - start;\n    }\n\n    private void readInternal(long position, byte[] buffer, int bufferOffset, int bufferLength)\n    {\n        try {\n            inputStream.readFully(position, buffer, bufferOffset, bufferLength);\n        }\n        catch (IOException e) {\n            throw new PrestoException(DRUID_DEEP_STORAGE_ERROR, format(\"Error reading from %s at position %s\", id, position), e);\n        }\n    }\n\n    @Override\n    public void close()\n            throws IOException\n    {\n        inputStream.close();\n    }\n}\n","sourceCodeStart":66,"sourceCodeEnd":95,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-druid/src/main/java/com/facebook/presto/druid/segment/HdfsDataInputSource.java#L66-L95","documentation":"Thrown by Druid's HdfsDataInputSource when an IOException occurs while reading bytes of a Druid segment from HDFS deep storage at a given position. It wraps the underlying I/O failure in a DRUID_DEEP_STORAGE_ERROR so callers see a uniform error for deep-storage read problems.","triggerScenarios":"readInternal(position, buffer, ...) calls inputStream.readFully(...) (via readFully) and the underlying HDFS/local input stream throws IOException — e.g. block missing, datanode failure, truncated segment file, permission error.","commonSituations":"HDFS NameNode/DataNode outages, segment files deleted or moved by retention jobs, replication undercount during node failure, corrupted or truncated segments on deep storage, misconfigured HDFS federation paths in Druid deep storage config.","solutions":["Check underlying HDFS cluster health (hdfs fsck on the segment path) and datanode logs","Verify the segment file exists and is not truncated: hdfs dfs -ls -h <segment path>","Retry the query; transient datanode/replication failures often clear","Validate druid.storage.storage-directory and deep-storage connectivity config","Re-ingest or re-publish the affected Druid segment if corrupted"],"exampleFix":"// before\ninputStream.readFully(position, buffer, bufferOffset, bufferLength);\n// after\ntry (FileSystem fs = path.getFileSystem(conf)) {\n    if (!fs.exists(path)) { throw new PrestoException(DRUID_DEEP_STORAGE_ERROR, \"Segment missing: \" + path); }\n    inputStream.readFully(position, buffer, bufferOffset, bufferLength);\n}","handlingStrategy":"try-catch","validationCode":"// caller-side pre-check\nlong len = fileSystem.getFileStatus(segmentPath).getLen();\nif (position + bufferLength > len) {\n    throw new PrestoException(DRUID_DEEP_STORAGE_ERROR, \"Read past end of segment: \" + segmentPath);\n}","typeGuard":"function isDeepStorageReadError(e) { return e instanceof PrestoException && e.getErrorCode().getName().equals('DRUID_DEEP_STORAGE_ERROR'); }","tryCatchPattern":"try {\n    segmentReader.readFully(position, buffer, 0, buffer.length);\n} catch (PrestoException e) {\n    if (e.getErrorCode().getName().equals(\"DRUID_DEEP_STORAGE_ERROR\")) {\n        // log position + segment id, retry or failover to replica segment\n    } else { throw e; }\n}","preventionTips":["Run hdfs fsck periodically on Druid deep-storage segment paths","Keep replication factor >= 3 for segment files","Verify segment existence before reads (fs.exists)","Monitor datanode and NameNode health alerts","Pin segment files from retention/deletion jobs during query windows"],"tags":["hdfs","druid","io","deep-storage"],"backgroundTag":"hdfs-read-failure","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}