apache/pulsar · error · IllegalArgumentException

Sink package doesn't contain the META-INF/services/pulsar-io

Error message

Sink package doesn't contain the META-INF/services/pulsar-io.yaml file.

What it means

When the sink config has no className, the framework treats the submitted package as a built-in NAR connector and reads META-INF/services/pulsar-io.yaml from it to obtain the ConnectorDefinition. If that descriptor is missing, validateAndExtractDetails throws this IllegalArgumentException — the archive is neither a plain-Java sink (no className) nor a valid NAR connector (no pulsar-io.yaml).

Source

Thrown at pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/SinkConfigUtils.java:464

        if (sinkConfig.getParallelism() != null && sinkConfig.getParallelism() <= 0) {
            throw new IllegalArgumentException("Sink parallelism must be a positive number");
        }

        if (sinkConfig.getResources() != null) {
            ResourceConfigUtils.validate(sinkConfig.getResources());
        }

        if (sinkConfig.getTimeoutMs() != null && sinkConfig.getTimeoutMs() < 0) {
            throw new IllegalArgumentException("Sink timeout must be a positive number");
        }

        String sinkClassName = sinkConfig.getClassName();
        // if class name in sink config is not set, this should be a built-in sink
        // thus we should try to find it class name in the NAR service definition
        if (sinkClassName == null) {
            ConnectorDefinition connectorDefinition = sinkFunction.getFunctionMetaData(ConnectorDefinition.class);
            if (connectorDefinition == null) {
                throw new IllegalArgumentException(
                        "Sink package doesn't contain the META-INF/services/pulsar-io.yaml file.");
            }
            sinkClassName = connectorDefinition.getSinkClass();
            if (sinkClassName == null) {
                throw new IllegalArgumentException("Failed to extract sink class from archive");
            }
        }

        // check if sink implements the correct interfaces
        TypeDefinition sinkClass;
        try {
            sinkClass = sinkFunction.resolveType(sinkClassName);
        } catch (TypePool.Resolution.NoSuchTypeException e) {
            throw new IllegalArgumentException(
                    String.format("Sink class %s not found", sinkClassName), e);
        }

        String functionClassName = sinkConfig.getTransformFunctionClassName();

View on GitHub (pinned to 820761864e)

Solutions

Add META-INF/services/pulsar-io.yaml to the NAR archive declaring the sinkClass.
Or set sinkConfig.setClassName("com.example.MySink") so the archive is treated as a regular jar.
Verify with 'unzip -l my-sink.nar | grep pulsar-io.yaml' that the descriptor is present at the right path.
Check the NAR build config (pulsar-io NAR plugin / maven-nar) isn't excluding resources.

Example fix

// before
SinkConfig cfg = new SinkConfig();
cfg.setTenant("public");
cfg.setNamespace("default");
cfg.setName("my-sink");
// archive: plain jar without pulsar-io.yaml and no className
// after — either declare className:
cfg.setClassName("org.example.MySink");
// or ship a proper NAR containing:
// META-INF/services/pulsar-io.yaml with:
// name: my-sink
// description: ...
// sinkClass: org.example.MySink

Defensive patterns

Strategy: validation

Validate before calling

if (sinkConfig.getClassName() == null) {
    try (JarFile jar = new JarFile(archivePath.toFile())) {
        if (jar.getJarEntry("META-INF/services/pulsar-io.yaml") == null) {
            throw new IllegalArgumentException("archive lacks META-INF/services/pulsar-io.yaml; set className or use a valid NAR");
        }
    }
}

Type guard

static boolean isBuiltInConnectorArchive(Path archive, String className) throws IOException {
    if (className != null) return true;
    try (JarFile jar = new JarFile(archive.toFile())) {
        return jar.getJarEntry("META-INF/services/pulsar-io.yaml") != null;
    }
}

Try / catch

try {
    SinkConfigUtils.validateAndExtractDetails(cfg, sinkPkg, transformPkg, true);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("pulsar-io.yaml")) {
        log.error("Archive is not a valid connector NAR: provide className or fix packaging", e);
    }
    throw e;
}

Prevention

For plain jar sinks, always set className explicitly.
Verify NAR packaging includes META-INF/services/pulsar-io.yaml (unzip -l).
Use the official pulsar-io NAR build plugin so the descriptor is generated/kept.
Check the archive path passed to --archive points at the NAR, not a transitive jar.

When it happens

Trigger: Submitting a sink with className unset and an archive (jar/nar) that lacks META-INF/services/pulsar-io.yaml; uploading a plain fat-jar without the connector descriptor while omitting className.

Common situations: Building a custom connector NAR and forgetting the pulsar-io.yaml service file; using the wrong archive path in --archive; packaging a plain sink jar but not passing --className; NAR built with the wrong packaging plugin that strips resources.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/756f70a8214e9fc7. Report an issue: GitHub.