apache/pulsar · error · IllegalArgumentException

Sources cannot be update between regular sources and batchso

Error message

Sources cannot be update between regular sources and batchsource

What it means

A source cannot be converted between a regular Source and a BatchSource via update; the two runtimes differ fundamentally. validateUpdate compares isBatchSource(existingConfig) with isBatchSource(newConfig) and rejects any mismatch.

Source

Thrown at pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/SourceConfigUtils.java:433

        }
        if (newConfig.getParallelism() != null) {
            mergedConfig.setParallelism(newConfig.getParallelism());
        }
        if (newConfig.getResources() != null) {
            mergedConfig
                    .setResources(ResourceConfigUtils.merge(existingConfig.getResources(), newConfig.getResources()));
        }
        if (!StringUtils.isEmpty(newConfig.getArchive())) {
            mergedConfig.setArchive(newConfig.getArchive());
        }
        if (!StringUtils.isEmpty(newConfig.getRuntimeFlags())) {
            mergedConfig.setRuntimeFlags(newConfig.getRuntimeFlags());
        }
        if (!StringUtils.isEmpty(newConfig.getCustomRuntimeOptions())) {
            mergedConfig.setCustomRuntimeOptions(newConfig.getCustomRuntimeOptions());
        }
        if (isBatchSource(existingConfig) != isBatchSource(newConfig)) {
            throw new IllegalArgumentException("Sources cannot be update between regular sources and batchsource");
        }
        if (newConfig.getBatchSourceConfig() != null) {
            validateBatchSourceConfigUpdate(existingConfig.getBatchSourceConfig(), newConfig.getBatchSourceConfig());
            mergedConfig.setBatchSourceConfig(newConfig.getBatchSourceConfig());
        }
        if (newConfig.getProducerConfig() != null) {
            mergedConfig.setProducerConfig(newConfig.getProducerConfig());
        }
        return mergedConfig;
    }

    public static void validateBatchSourceConfig(BatchSourceConfig batchSourceConfig) throws IllegalArgumentException {
        if (isEmpty(batchSourceConfig.getDiscoveryTriggererClassName())) {
            log.error("BatchSourceConfig does not specify Discovery Trigger ClassName");
            throw new IllegalArgumentException("BatchSourceConfig does not specify Discovery Trigger ClassName");
        }
    }

View on GitHub (pinned to 820761864e)

Solutions

1. Submit the update keeping the same source kind: include batchSourceConfig for batch sources, omit it for regular sources
2. To switch kinds, delete the source and create a new one of the desired type
3. Check serialization/deserialization of the update payload so batchSourceConfig is not silently dropped

Example fix

// before
{"className":"com.example.MyBatchSource", ...} // batchSourceConfig omitted in update
// after
{"className":"com.example.MyBatchSource","batchSourceConfig":{"discoveryTriggererClassName":"com.example.MyTriggerer", ...}, ...}

if ((existing.getBatchSourceConfig() == null) != (update.getBatchSourceConfig() == null)) {
    throw new IllegalArgumentException("cannot switch between regular and batch source on update");
}

boolean isBatchSource(SourceConfig c) { return c.getBatchSourceConfig() != null; }

try { SourceConfigUtils.validateUpdate(existing, update); } catch (IllegalArgumentException e) { if (e.getMessage().contains("regular sources and batchsource")) { if (SourceConfigUtils.isBatchSource(existing)) update.setBatchSourceConfig(existing.getBatchSourceConfig()); else update.setBatchSourceConfig(null); } else { throw e; } }

When it happens

Trigger: validateUpdate where exactly one of the existing/new configs has batchSourceConfig set (presence of batchSourceConfig defines batch source), e.g. adding batchSourceConfig to an existing regular source, or removing/nulling it on a batch source update.

Common situations: Upgrading a regular source to batch mode by patching the config; a deserializer drops batchSourceConfig from the update payload, making a batch source look regular; template reuse across batch and non-batch sources.

Related errors

• Batch Configs cannot be found
• BatchSource does not implement the correct interface
• BatchSourceTriggerer does not implement the correct interfac
• error writing discovered task to intermediate topic
• Tenants differ

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/1132b59182e1cb2d. Report an issue: GitHub.