prestodb/presto · error · PrestoException

DRUID_DEEP_STORAGE_ERROR

DRUID_DEEP_STORAGE_ERROR

Error message

Failed to create page source on ${segmentInfo.getDeepStoragePath()}

What it means

DruidPageSourceProvider.createPageSource wraps IOException from opening/reading a Druid segment's deep-storage file and rethrows it as DRUID_DEEP_STORAGE_ERROR with the failing deepStoragePath. It means the connector could not create a page source over the segment data fetched from deep storage (S3/HDFS/local), so the scan of that segment fails.

Source

Thrown at presto-druid/src/main/java/com/facebook/presto/druid/DruidPageSourceProvider.java:93

        DruidSegmentInfo segmentInfo = druidSplit.getSegmentInfo().get();
        try {
            Path segmentPath = new Path(segmentInfo.getDeepStoragePath());
            FileSystem fileSystem = segmentPath.getFileSystem(hadoopConfiguration);
            long fileSize = fileSystem.getFileStatus(segmentPath).getLen();
            FSDataInputStream inputStream = fileSystem.open(segmentPath);
            DataInputSourceId dataInputSourceId = new DataInputSourceId(segmentPath.toString());
            HdfsDataInputSource dataInputSource = new HdfsDataInputSource(dataInputSourceId, inputStream, fileSize);
            IndexFileSource indexFileSource = new ZipIndexFileSource(dataInputSource);
            SegmentColumnSource segmentColumnSource = new SmooshedColumnSource(indexFileSource);
            SegmentIndexSource segmentIndexSource = new V9SegmentIndexSource(segmentColumnSource);

            return new DruidSegmentPageSource(
                    dataInputSource,
                    columns,
                    new DruidSegmentReader(segmentIndexSource, columns));
        }
        catch (IOException e) {
            throw new PrestoException(DRUID_DEEP_STORAGE_ERROR, "Failed to create page source on " + segmentInfo.getDeepStoragePath(), e);
        }
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

Verify the segment file exists at the printed deepStoragePath in deep storage and that retention/kill policies have not deleted it.
Check deep-storage credentials and connectivity from the Presto worker (S3 access keys/IAM role, HDFS config), then retry the query.
Confirm Druid's deep storage configuration and that Presto's druid.client / data source settings match the same storage location.
Re-ingest or re-publish the affected segment if the file is corrupt or missing, then re-run the query.

Example fix

// before (typical misconfig: wrong bucket/region so open() throws IOException)
druid.deep-storage.type=s3
druid.s3.bucket=old-bucket
// after
druid.deep-storage.type=s3
druid.s3.bucket=current-bucket
druid.s3.region=us-east-1  # must match segment location; also ensure IAM read access

Defensive patterns

Strategy: retry

Validate before calling

// Pre-check segment availability before scanning (conceptual):
// 1. Confirm the object exists at segmentInfo.getDeepStoragePath() via the storage client
// 2. Verify worker credentials can read it
boolean readable = storageClient.exists(deepStoragePath) && storageClient.canRead(deepStoragePath);
if (!readable) {
    throw new IllegalStateException("Segment unreadable at " + deepStoragePath + ": fix deep-storage config/credentials before querying");
}

Try / catch

try {
    return query(sql);
}
catch (PrestoException e) {
    if ("DRUID_DEEP_STORAGE_ERROR".equals(e.getErrorCode().getName())) {
        // transient network blips: bounded retry; do NOT retry missing/corrupt segments
        return retryWithBackoff(() -> query(sql), 3);
    }
    throw e;
}

Prevention

Keep Druid retention/kill rules aligned with how long Presto may query old segments.
Verify Presto worker IAM/credentials have read access to the Druid deep-storage bucket/path.
Monitor deep-storage availability (S3 errors, HDFS NameNode health) before blaming the connector.
After Druid compaction/migration, confirm segment deep-storage paths are updated in metadata.
Log and alert on this error code; the message includes the exact path to inspect.

When it happens

Trigger: A query scans a Druid segment whose deep storage path cannot be read: the object is missing/deleted, credentials are wrong, network to S3/HDFS fails, the file is corrupt/truncated, or the dataInputSource factory cannot open the URI.

Common situations: Segments retained in Druid metadata but already cleaned up from deep storage (retention/kill rules); expired or misconfigured S3 credentials/IAM role for the Presto worker; HDFS NameNode unreachable; segment files moved by compaction or migration.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/7b489a28bf88ded6. Report an issue: GitHub.