apache/seatunnel · error · IllegalArgumentException

Option 'max_in_flight_messages' must be greater than or equa

Error message

Option 'max_in_flight_messages' must be greater than or equal to batch_size

What it means

AzureQueueSourceConfig.validate throws this when max_in_flight_messages < batch_size. Each batch received needs at least that many in-flight slots, so a smaller value would make the pipeline unable to track a full batch of messages concurrently.

Source

Thrown at seatunnel-connectors-v2/connector-azure-queue-storage/src/main/java/org/apache/seatunnel/connectors/seatunnel/azure/queue/config/AzureQueueSourceConfig.java:100

    private void validate() {
        AzureQueueConfigValidator.validateClient(this);
        if (format == MessageFormat.TEXT && fieldDelimiter.isEmpty()) {
            throw new IllegalArgumentException("Option 'field_delimiter' cannot be empty");
        }
        if (batchSize < 1 || batchSize > MAX_BATCH_SIZE) {
            throw new IllegalArgumentException("Option 'batch_size' must be between 1 and 32");
        }
        if (visibilityTimeoutSeconds < 1
                || visibilityTimeoutSeconds > MAX_VISIBILITY_TIMEOUT_SECONDS) {
            throw new IllegalArgumentException(
                    "Option 'visibility_timeout_seconds' must be between 1 and 604800");
        }
        if (pollIntervalMillis <= 0) {
            throw new IllegalArgumentException(
                    "Option 'poll_interval_ms' must be greater than zero");
        }
        if (maxInFlightMessages < batchSize) {
            throw new IllegalArgumentException(
                    "Option 'max_in_flight_messages' must be greater than or equal to batch_size");
        }
        if (operationTimeoutMillis <= 0) {
            throw new IllegalArgumentException(
                    "Option 'operation_timeout_ms' must be greater than zero");
        }
        long visibilityTimeoutMillis = visibilityTimeoutSeconds * 1_000L;
        if (operationTimeoutMillis >= visibilityTimeoutMillis / 2) {
            throw new IllegalArgumentException(
                    "Option 'operation_timeout_ms' must be less than half of visibility_timeout_seconds");
        }
    }
}

View on GitHub (pinned to cf67b549a7)

Solutions

  1. Raise max_in_flight_messages to be >= batch_size (e.g. equal to or a multiple of batch_size)
  2. Or lower batch_size until it does not exceed max_in_flight_messages
  3. Remove one of the options to use defaults that are already consistent

Example fix

// before
batch_size = 16
max_in_flight_messages = 8
// after
batch_size = 16
max_in_flight_messages = 16
Defensive patterns

Strategy: validation

Validate before calling

java
if (maxInFlightMessages < batchSize) {
    maxInFlightMessages = batchSize;
}

Try / catch

java
try {
    AzureQueueSourceConfig.from(config);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("max_in_flight_messages")) {
        log.error("max_in_flight_messages must be >= batch_size");
    } else { throw e; }
}

Prevention

When it happens

Trigger: Building AzureQueueSourceConfig via from() where max_in_flight_messages is set lower than batch_size, e.g. batch_size = 16 and max_in_flight_messages = 8.

Common situations: Tuning max_in_flight down for backpressure without checking batch_size; defaults tuned independently after changing batch_size; misunderstanding that the two options are unrelated.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/0d18c498349c33df. Report an issue: GitHub.