apache/druid · error · IllegalStateException (ISE)

Failed to get s3 object for bucket[%s], key[%s], range[bytes

Error message

Failed to get s3 object for bucket[%s], key[%s], range[bytes=%d-%d]

What it means

S3SegmentRangeReader.open issues a GetObject with a byte range; the SDK returned a null InputStream despite not throwing, which is an unexpected/failed fetch of the ranged object. The reader throws ISE naming bucket, key, and byte range.

Source

Thrown at extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3SegmentRangeReader.java:136

    @Override
    public InputStream open(RangeRequest request, long start) throws IOException
    {
      final long absoluteStart = request.offset + start;
      final long absoluteEnd = request.offset + request.length - 1;
      if (absoluteStart > absoluteEnd) {
        // Logically nothing left to read, only reachable if a retry fires after the consumer drained the entire
        // range successfully, which shouldn't happen in practice. Returning empty keeps us robust either way.
        return new ByteArrayInputStream(new byte[0]);
      }
      final GetObjectRequest.Builder requestBuilder = GetObjectRequest.builder()
          .bucket(bucket)
          .key(request.objectKey)
          .range(AwsBytesRange.of(absoluteStart, absoluteEnd).getBytesRange());
      try {
        final InputStream s3Object = s3Client.getObject(requestBuilder);
        if (s3Object == null) {
          throw new ISE(
              "Failed to get s3 object for bucket[%s], key[%s], range[bytes=%d-%d]",
              bucket,
              request.objectKey,
              absoluteStart,
              absoluteEnd
          );
        }
        return s3Object;
      }
      catch (S3Exception e) {
        throw new IOException(e);
      }
    }
  }
}

View on GitHub (pinned to 9b90983fd2)

Solutions

Verify the object still exists at bucket/key before reading.
Check S3 lifecycle/retention policies that might delete segments mid-read.
If using S3-compatible storage, test range GET behavior there.
Retry the read; the underlying RetryUtils may mask transient nulls upstream.

Example fix

// before
InputStream in = rangeReader.open(request);
// after
if (!S3Utils.isObjectInBucket(s3Client, bucket, key)) {
  throw new IllegalStateException("object gone: " + bucket + "/" + key);
}
InputStream in = rangeReader.open(request);

Defensive patterns

Strategy: try-catch

Validate before calling

if (!S3Utils.isObjectInBucket(s3Client, bucket, key)) throw new IllegalStateException("object missing: " + bucket + "/" + key);

Type guard

InputStream in = s3Client.getObject(req);
if (in == null) throw new IOException("null body for " + req.bucket() + "/" + req.key());

Try / catch

try { in = rangeReader.open(req); } catch (IllegalStateException e) { /* object vanished: re-check existence, refresh segment locator */ }

Prevention

Don't delete segments while queries may read them
Beware lifecycle policies racing reads
Test range GET support when using S3-compatible stores

When it happens

Trigger: Calling open() on a segment range when s3Client.getObject returns null (object vanished between check and read, or a proxy/client quirk yields a null body).

Common situations: Segment deleted by retention/lifecycle rules while being read; S3-compatible storage (MinIO) returning null body on range GET; race with segment replacement during re-indexing.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/b943ea24b9878c12. Report an issue: GitHub.