apache/druid · info

The 'recordBufferSize' config property of the kinesis…

Error message

The 'recordBufferSize' config property of the kinesis tuning config has been deprecated. Please use 'recordBufferSizeBytes'.

What it means

KinesisIndexTask.runTask() logs a deprecation warning when the tuning config's recordBufferSize field was explicitly set (getRecordBufferSizeConfigured() != null). The byte-based recordBufferSizeBytes property replaces the record-count-based property. The task still runs; this is advisory only.

Solutions

1. Replace 'recordBufferSize' with 'recordBufferSizeBytes' in the tuning config (value in bytes instead of records).
2. Convert the old record count to an approximate byte budget given your average record size.
3. Update stored spec templates and any automation that generates supervisor specs.
4. Re-submit/re-create the supervisor so the new tuning config takes effect.

Example fix

// before
"tuningConfig": { "type": "kinesis", "recordBufferSize": 1000 }
// after
"tuningConfig": { "type": "kinesis", "recordBufferSizeBytes": 1048576 }

if (tuningConfig instanceof KinesisIndexTaskClient) { /* check spec JSON */ }
Object node = tuningConfigNode.get("recordBufferSize");
if (node != null) throw new IllegalArgumentException("Use recordBufferSizeBytes instead");

When it happens

Trigger: Submitting or running a Kinesis indexing task whose tuning config contains the deprecated 'recordBufferSize' property.

Common situations: Older JSON specs copied from pre-upgrade templates; docs or tooling still emitting recordBufferSize after upgrade to the bytes-based tuning config.

Understand the failure class

Background: "is deprecated and will be removed" — deprecation warnings for old API names, keywords, and options, and how to migrate before the removal release — this error's family across 29 libraries.

Related errors

• The 'maxRecordsPerPoll' config property of the kinesis…
• fetchThreads [ ] being lowered to [ ]
• can't reschedule fetch records runnable, recordsResult is…
• Can't use both maxNumSubTasks and maxNumConcurrentSubTasks…
• Conflicting segment granularities found

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/de1c70d3fc07e64e. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/kinesis-indexing-service/src/main/java/org/apache/druid/indexing/kinesis/KinesisIndexTask.java:99

        supervisorId,
        taskResource,
        dataSchema,
        tuningConfig,
        ioConfig,
        context,
        getFormattedGroupId(Configs.valueOrDefault(supervisorId, dataSchema.getDataSource()), TYPE),
        serverPriority
    );
    this.useListShards = useListShards;
    this.awsCredentialsConfig = awsCredentialsConfig;
  }

  @Override
  public TaskStatus runTask(TaskToolbox toolbox)
  {
    this.runtimeInfo = toolbox.getAdjustedRuntimeInfo();
    if (getTuningConfig().getRecordBufferSizeConfigured() != null) {
      log.warn("The 'recordBufferSize' config property of the kinesis tuning config has been deprecated. "
               + "Please use 'recordBufferSizeBytes'.");
    }
    if (getTuningConfig().getMaxRecordsPerPollConfigured() != null) {
      log.warn("The 'maxRecordsPerPoll' config property of the kinesis tuning config has been deprecated. "
               + "Please use 'maxBytesPerPoll'.");
    }
    return super.runTask(toolbox);
  }

  @Override
  protected SeekableStreamIndexTaskRunner<String, String, KinesisRecordEntity> createTaskRunner()
  {
    //noinspection unchecked
    return new KinesisIndexTaskRunner(this, lockGranularityToUse);
  }

  @Override
  protected KinesisRecordSupplier newTaskRecordSupplier(final TaskToolbox toolbox)

View on GitHub (pinned to 9b90983fd2)