What causes "Option 'batch_size' must be between 1 and 32" in apache/seatunnel?

Set batch_size to an integer between 1 and 32 (e.g. 16)

apache/seatunnel · error · IllegalArgumentException

Option 'batch_size' must be between 1 and 32

Error message

Option 'batch_size' must be between 1 and 32

What it means

AzureQueueSourceConfig.validate throws this when batch_size is outside the 1..MAX_BATCH_SIZE (32) range, matching the Azure Queue receiveMessages batch limit. It runs during from() so invalid batches fail at config time.

Source

Thrown at seatunnel-connectors-v2/connector-azure-queue-storage/src/main/java/org/apache/seatunnel/connectors/seatunnel/azure/queue/config/AzureQueueSourceConfig.java:88

                                        AzureQueueStorageSourceOptions.VISIBILITY_TIMEOUT_SECONDS))
                        .pollIntervalMillis(
                                config.get(AzureQueueStorageSourceOptions.POLL_INTERVAL_MS))
                        .maxInFlightMessages(
                                config.get(AzureQueueStorageSourceOptions.MAX_IN_FLIGHT_MESSAGES))
                        .operationTimeoutMillis(
                                config.get(AzureQueueStorageSourceOptions.OPERATION_TIMEOUT_MS))
                        .build();
        sourceConfig.validate();
        return sourceConfig;
    }

    private void validate() {
        AzureQueueConfigValidator.validateClient(this);
        if (format == MessageFormat.TEXT && fieldDelimiter.isEmpty()) {
            throw new IllegalArgumentException("Option 'field_delimiter' cannot be empty");
        }
        if (batchSize < 1 || batchSize > MAX_BATCH_SIZE) {
            throw new IllegalArgumentException("Option 'batch_size' must be between 1 and 32");
        }
        if (visibilityTimeoutSeconds < 1
                || visibilityTimeoutSeconds > MAX_VISIBILITY_TIMEOUT_SECONDS) {
            throw new IllegalArgumentException(
                    "Option 'visibility_timeout_seconds' must be between 1 and 604800");
        }
        if (pollIntervalMillis <= 0) {
            throw new IllegalArgumentException(
                    "Option 'poll_interval_ms' must be greater than zero");
        }
        if (maxInFlightMessages < batchSize) {
            throw new IllegalArgumentException(
                    "Option 'max_in_flight_messages' must be greater than or equal to batch_size");
        }
        if (operationTimeoutMillis <= 0) {
            throw new IllegalArgumentException(
                    "Option 'operation_timeout_ms' must be greater than zero");
        }

View on GitHub (pinned to cf67b549a7)

Solutions

Set batch_size to an integer between 1 and 32 (e.g. 16)
Increase parallelism (source parallelism) instead of raising batch_size above 32
Remove the option to use the plugin default batch size

Example fix

// before
batch_size = 50
// after
batch_size = 16

Defensive patterns

Strategy: validation

Validate before calling

java
if (batchSize < 1 || batchSize > 32) {
    throw new IllegalArgumentException("batch_size must be between 1 and 32");
}

Try / catch

java
try {
    AzureQueueSourceConfig.from(config);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("batch_size")) {
        log.error("Clamp batch_size to 1..32 (Azure service limit)");
    } else { throw e; }
}

Prevention

Remember 32 is the Azure Storage Queue receive limit, not a tunable default
Scale with parallelism, not batch size
Clamp values in config generators: Math.min(32, Math.max(1, n))

When it happens

Trigger: Building AzureQueueSourceConfig via from() with batch_size < 1 or batch_size > 32, e.g. batch_size = 50 to 'speed up' reads.

Common situations: User mistook 32 for a configurable default and set 100; value supplied in seconds/percentage by mistake; auto-generated config produced 0.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/863c6cdabd5570dd. Report an issue: GitHub.