apache/druid · info

The 'maxRecordsPerPoll' config property of the kinesis…

Error message

The 'maxRecordsPerPoll' config property of the kinesis tuning config has been deprecated. Please use 'maxBytesPerPoll'.

What it means

KinesisIndexTask.runTask() logs a deprecation warning when the tuning config's maxRecordsPerPoll field was explicitly set (getMaxRecordsPerPollConfigured() != null). The byte-based maxBytesPerPoll property replaces it. Execution continues; it is advisory only.

Solutions

  1. Replace 'maxRecordsPerPoll' with 'maxBytesPerPoll' in the tuning config.
  2. Estimate bytes per poll from average record size times desired records per poll.
  3. Update task-generation tooling and stored supervisor specs.
  4. Redeploy the supervisor with the corrected tuning config.

Example fix

// before
"tuningConfig": { "type": "kinesis", "maxRecordsPerPoll": 10000 }
// after
"tuningConfig": { "type": "kinesis", "maxBytesPerPoll": 4194304 }
Defensive patterns

Strategy: validation

Validate before calling

Object node = tuningConfigNode.get("maxRecordsPerPoll");
if (node != null) throw new IllegalArgumentException("Use maxBytesPerPoll instead");

Prevention

When it happens

Trigger: Running a Kinesis indexing task whose tuning config contains the deprecated 'maxRecordsPerPoll' property.

Common situations: Legacy specs retained across a Druid upgrade; templates or IaC still emitting maxRecordsPerPoll; mixed configs where some tasks use old and some new fields.

Understand the failure class

Background: "is deprecated and will be removed" — deprecation warnings for old API names, keywords, and options, and how to migrate before the removal release — this error's family across 29 libraries.

Related errors


AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/b6d3c4f7c52b0090. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/kinesis-indexing-service/src/main/java/org/apache/druid/indexing/kinesis/KinesisIndexTask.java:103

        ioConfig,
        context,
        getFormattedGroupId(Configs.valueOrDefault(supervisorId, dataSchema.getDataSource()), TYPE),
        serverPriority
    );
    this.useListShards = useListShards;
    this.awsCredentialsConfig = awsCredentialsConfig;
  }

  @Override
  public TaskStatus runTask(TaskToolbox toolbox)
  {
    this.runtimeInfo = toolbox.getAdjustedRuntimeInfo();
    if (getTuningConfig().getRecordBufferSizeConfigured() != null) {
      log.warn("The 'recordBufferSize' config property of the kinesis tuning config has been deprecated. "
               + "Please use 'recordBufferSizeBytes'.");
    }
    if (getTuningConfig().getMaxRecordsPerPollConfigured() != null) {
      log.warn("The 'maxRecordsPerPoll' config property of the kinesis tuning config has been deprecated. "
               + "Please use 'maxBytesPerPoll'.");
    }
    return super.runTask(toolbox);
  }

  @Override
  protected SeekableStreamIndexTaskRunner<String, String, KinesisRecordEntity> createTaskRunner()
  {
    //noinspection unchecked
    return new KinesisIndexTaskRunner(this, lockGranularityToUse);
  }

  @Override
  protected KinesisRecordSupplier newTaskRecordSupplier(final TaskToolbox toolbox)
      throws RuntimeException
  {
    KinesisIndexTaskIOConfig ioConfig = ((KinesisIndexTaskIOConfig) super.ioConfig);
    KinesisIndexTaskTuningConfig tuningConfig = ((KinesisIndexTaskTuningConfig) super.tuningConfig);

View on GitHub (pinned to 9b90983fd2)