{"record":{"id":"ce1ce54d218c3589","repo":"apache/druid","slug":"failed-to-get-an-s3-object-for-bucket-s-key-s","errorCode":null,"errorMessage":"Failed to get an s3 object for bucket[%s], key[%s], and start[%d]","messagePattern":"Failed to get an s3 object for bucket\\[(.+?)\\], key\\[(.+?)\\], and start\\[(.+?)\\]","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"extensions-core/s3-extensions/src/main/java/org/apache/druid/data/input/s3/S3Entity.java","lineNumber":76,"sourceCode":"  }\n\n  @Override\n  public URI getUri()\n  {\n    return object.toUri(S3StorageDruidModule.SCHEME);\n  }\n\n  @Override\n  protected InputStream readFrom(long offset) throws IOException\n  {\n    GetObjectRequest.Builder requestBuilder = GetObjectRequest.builder()\n        .bucket(object.getBucket())\n        .key(object.getPath())\n        .range(AwsBytesRange.from(offset).getBytesRange());\n    try {\n      final ResponseInputStream<GetObjectResponse> s3Object = s3Client.getObject(requestBuilder);\n      if (s3Object == null) {\n        throw new ISE(\n            \"Failed to get an s3 object for bucket[%s], key[%s], and start[%d]\",\n            object.getBucket(),\n            object.getPath(),\n            offset\n        );\n      }\n      return s3Object;\n    }\n    catch (S3Exception e) {\n      throw new IOException(e);\n    }\n  }\n\n  @Override\n  protected String getPath()\n  {\n    return object.getPath();\n  }","sourceCodeStart":58,"sourceCodeEnd":94,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/s3-extensions/src/main/java/org/apache/druid/data/input/s3/S3Entity.java#L58-L94","documentation":"This IllegalStateException is raised in S3Entity.readFrom when the AWS v2 SDK client's getObject call returns null despite being asked for a specific bucket, key, and byte range. The S3 v2 SDK contract normally returns a non-null ResponseInputStream or throws; a null here indicates an unexpected/unrecoverable client state (e.g. a closed or misbuilt client), so Druid fails fast with an ISE rather than returning a broken stream.","triggerScenarios":"Calling readFrom (input source split/reader open) where s3Client.getObject(requestBuilder) yields null for the given bucket/key/offset — typically with a null or improperly initialized S3Client instance returned by a mocked or custom S3ClientSupplier.","commonSituations":"Unit tests with Mockito mocks that return null for getObject instead of a stubbed stream; a custom S3 client supplier/factory returning null on client creation or after close; SDK version mismatches where an error path returns null instead of throwing.","solutions":["Fix the S3Client supplier so it never returns or yields a null client / null response; stub getObject to return a real ResponseInputStream in tests.","If running tests, add a Mockito stub: when(s3Client.getObject(any(GetObjectRequest.class))).thenReturn(mockStream).","Check that the S3 client is not closed/disposed before readFrom is called (task lifecycle issue).","Upgrade/align the software.amazon.awssdk:s3 version to a release where getObject throws S3Exception instead of returning null."],"exampleFix":"// before\nwhen(s3Client.getObject(any(GetObjectRequest.class))).thenReturn(null);\n// after\nwhen(s3Client.getObject(any(GetObjectRequest.class)))\n    .thenReturn(new ResponseInputStream<>(GetObjectResponse.builder().build(),\n        AbortableInputStream.create(new ByteArrayInputStream(data))));","handlingStrategy":"type-guard","validationCode":"S3Client client = supplier.get();\nif (client == null) {\n  throw new IllegalStateException(\"S3Client supplier returned null client\");\n}","typeGuard":"static boolean isUsableResponse(ResponseInputStream<GetObjectResponse> resp) {\n  return resp != null && resp.getResponse() != null;\n}","tryCatchPattern":"try (ResponseInputStream<GetObjectResponse> in = s3Client.getObject(req)) {\n  if (in == null) {\n    throw new ISE(\"S3 client returned null for bucket[%s] key[%s]\", bucket, key);\n  }\n  // consume stream\n} catch (S3Exception | NoSuchElementException e) {\n  // handle SDK failure; null should be impossible with a healthy client\n}","preventionTips":["Never let custom S3ClientSupplier implementations return null; throw on client construction failure","In tests, always stub getObject to return a real ResponseInputStream (use Mockito RETURNS_DEEP_STUBS or explicit stubs)","Keep AWS SDK v2 versions consistent between Druid and any custom extensions","Ensure clients are not closed before entity read"],"tags":["s3","aws-sdk","null-response","input-source"],"backgroundTag":"unexpected-api-response-shape","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}