apache/druid · error · IllegalStateException

can't reschedule fetch records runnable, recordsResult is nu

Error message

can't reschedule fetch records runnable, recordsResult is null??

What it means

The background fetch runnable in KinesisRecordSupplier reschedules itself after each getRecords call. If recordsResult comes back null the runnable cannot obtain the next shard iterator and cannot reschedule, so it throws this ISE. This indicates an internal invariant break — the Kinesis client returned null where a GetRecordsResult was expected.

Source

Thrown at extensions-core/kinesis-indexing-service/src/main/java/org/apache/druid/indexing/kinesis/KinesisRecordSupplier.java:352

          // may happen if interrupted while BlockingQueue.offer() is waiting
          log.warn(
              e,
              "Interrupted while waiting to add record to buffer, retrying in [%,dms]",
              EXCEPTION_RETRY_DELAY_MS
          );
          scheduleBackgroundFetch(EXCEPTION_RETRY_DELAY_MS);
        }
        catch (ExpiredIteratorException e) {
          log.warn(
              e,
              "ShardIterator expired while trying to fetch records, retrying in [%,dms]",
              fetchDelayMillis
          );
          if (recordsResult != null) {
            shardIterator = recordsResult.nextShardIterator(); // will be null if the shard has been closed
            scheduleBackgroundFetch(fetchDelayMillis);
          } else {
            throw new ISE("can't reschedule fetch records runnable, recordsResult is null??");
          }
        }
        catch (ResourceNotFoundException | InvalidArgumentException e) {
          // aws errors
          log.error(e, "encounted AWS error while attempting to fetch records, will not retry");
          throw e;
        }
        catch (SdkException e) {
          if (AWSClientUtil.isClientExceptionRecoverable(e)) {
            log.warn(e, "encounted unknown recoverable AWS exception, retrying in [%,dms]", EXCEPTION_RETRY_DELAY_MS);
            scheduleBackgroundFetch(EXCEPTION_RETRY_DELAY_MS);
          } else {
            log.warn(e, "encounted unknown unrecoverable AWS exception, will not retry");
            throw new RuntimeException(e);
          }
        }
        catch (Throwable e) {
          // non transient errors

View on GitHub (pinned to 9b90983fd2)

Solutions

Check for supplier lifecycle races — ensure close() is not called while background fetch is scheduled
Inspect shard state in AWS console (merged/closed shards) around the failure time
Capture the full logs just before this ISE; an earlier swallowed AWS error often explains the null
Report to Druid if reproducible — this path is expected to be unreachable

Defensive patterns

Strategy: try-catch

Validate before calling

if (supplier.isClosed()) { throw new IllegalStateException("supplier closed; not scheduling fetch"); }

Type guard

static boolean isInvariantFailure(Throwable t) {
  return t instanceof ISE && t.getMessage() != null && t.getMessage().contains("recordsResult is null");
}

Try / catch

try {
  runFetchLoop();
} catch (ISE e) {
  if (e.getMessage().contains("recordsResult is null")) {
    LOG.error(e, "Kinesis background fetch invariant broken; restarting task is required");
  }
  throw e;
}

Prevention

Avoid closing the supplier while background fetch is still scheduled
Check AWS SDK version compatibility with the Druid kinesis extension
Review shard merge/close events around failure time — keep shards' parent tracking consistent
Treat this as a bug: capture full logs and file an issue if reproducible

When it happens

Trigger: fetchRecords, run from scheduleBackgroundFetch, receives a null recordsResult from the underlying getRecords future/call and reaches the else branch that reschedules the next fetch.

Common situations: Race between supplier close() and an in-flight fetch; AWS SDK returning unexpected null after a shard merge/closure; bugs in custom AWS SDK or proxy layers intercepting the response.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/f77ca82826035451. Report an issue: GitHub.