{"record":{"id":"98836ba8d9754109","repo":"prestodb/presto","slug":"druid-segment-load-error","errorCode":"DRUID_SEGMENT_LOAD_ERROR","errorMessage":"failed to load druid segment","messagePattern":"failed to load druid segment","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"critical","filePath":"presto-druid/src/main/java/com/facebook/presto/druid/segment/DruidSegmentReader.java","lineNumber":66,"sourceCode":"\n    public DruidSegmentReader(SegmentIndexSource segmentIndexSource, List<ColumnHandle> columns)\n    {\n        try {\n            queryableIndex = segmentIndexSource.loadIndex(columns);\n            totalRowCount = queryableIndex.getNumRows();\n            ImmutableMap.Builder<String, ColumnReader> selectorsBuilder = ImmutableMap.builder();\n            for (ColumnHandle column : columns) {\n                DruidColumnHandle druidColumn = (DruidColumnHandle) column;\n                String columnName = druidColumn.getColumnName();\n                Type type = druidColumn.getColumnType();\n                BaseColumn baseColumn = queryableIndex.getColumnHolder(columnName).getColumn();\n                ColumnValueSelector<?> valueSelector = baseColumn.makeColumnValueSelector(new SimpleReadableOffset());\n                selectorsBuilder.put(columnName, createColumnReader(type, valueSelector));\n            }\n            columnValueSelectors = selectorsBuilder.build();\n        }\n        catch (IOException e) {\n            throw new PrestoException(DRUID_SEGMENT_LOAD_ERROR, \"failed to load druid segment\");\n        }\n    }\n\n    @Override\n    public int nextBatch()\n    {\n        // TODO: dynamic batch sizing\n        currentBatchSize = toIntExact(min(BATCH_SIZE, totalRowCount - currentPosition));\n        currentPosition += currentBatchSize;\n        return currentBatchSize;\n    }\n\n    @Override\n    public Block readBlock(Type type, String columnName)\n    {\n        return columnValueSelectors.get(columnName).readBlock(type, currentBatchSize);\n    }\n}","sourceCodeStart":48,"sourceCodeEnd":84,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-druid/src/main/java/com/facebook/presto/druid/segment/DruidSegmentReader.java#L48-L84","documentation":"DruidSegmentReader's constructor opens a Druid segment file, loads its columns, and creates ColumnValueSelectors wrapped in column readers. Any IOException during segment loading is rethrown as DRUID_SEGMENT_LOAD_ERROR with the generic message 'failed to load druid segment', so the connector cannot read the segment's data.","triggerScenarios":"Constructing DruidSegmentReader over a segment file that cannot be read: corrupted/truncated segment file, missing file at the resolved deep-storage path, network/filesystem failure while streaming from S3/HDFS/GS, or an IO error while building column value selectors.","commonSituations":"Incomplete or corrupted segments in deep storage (e.g. failed Druid ingestion left partial files); deep-storage path misconfiguration so the local cache points at a nonexistent file; HDFS/S3 outage or credentials issue while fetching; segment deleted between planning and read.","solutions":["Inspect the underlying IOException cause (enable debug logging) and verify the segment file exists and is intact at the deep-storage path returned by DruidSegmentInfo.getDeepStoragePath().","Verify the segment in Druid: query the Druid metadata for the segment and re-fetch/re-download it from deep storage; kill and re-ingest the segment if corrupted.","Check filesystem/network access from the Presto coordinator/worker to deep storage (S3/HDFS/GS credentials, connectivity).","Clear the segment cache if present and retry, so the segment is freshly pulled from deep storage."],"exampleFix":"// before (read without checking segment availability)\nDruidSegmentReader reader = new DruidSegmentReader(segmentFile, columns, types);\n\n// after (validate the segment is loadable first)\nif (!segmentFile.exists() || segmentFile.length() == 0) {\n    throw new PrestoException(DRUID_SEGMENT_LOAD_ERROR, \"Segment file missing or empty: \" + segmentFile);\n}\nDruidSegmentReader reader = new DruidSegmentReader(segmentFile, columns, types);","handlingStrategy":"try-catch","validationCode":"Path segmentPath = Paths.get(segmentFileLocation);\nif (!Files.isReadable(segmentPath) || Files.size(segmentPath) == 0) {\n    throw new PrestoException(DRUID_SEGMENT_LOAD_ERROR, \"Segment file missing, unreadable, or empty: \" + segmentFileLocation);\n}\n// optionally: validate magic/header bytes of the segment file before loading","typeGuard":null,"tryCatchPattern":"try {\n    DruidSegmentReader reader = new DruidSegmentReader(file, columns, types);\n} catch (PrestoException e) {\n    if (DRUID_SEGMENT_LOAD_ERROR.getCode().equals(e.getErrorCode())) {\n        log.error(e.getCause(), \"Segment load failed; invalidating cache for %s\", file);\n        // evict cached copy, re-fetch from deep storage, retry once\n    } else {\n        throw e;\n    }\n}","preventionTips":["Verify Druid ingestion completed cleanly so deep storage never holds truncated/partial segment files.","Check connectivity and credentials (S3/HDFS/GS) from Presto workers to deep storage before querying.","Monitor for segment files deleted or moved between query planning and execution; use Druid segment availability checks.","Keep segment cache directories healthy; clear corrupt cache entries after any segment-load failure."],"tags":["druid","segment","io","segment-loading"],"backgroundTag":"segment-load-failure","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}