apache/pulsar · error · IllegalArgumentException

Max message size need smaller than jvm directMemory

Error message

Max message size need smaller than jvm directMemory

What it means

At broker startup, maxFrameSize = brokerConfig.getMaxMessageSize() + Commands.MESSAGE_SIZE_FRAME_PADDING is checked against the JVM's max direct memory (DirectMemoryUtils.jvmMaxDirectMemory()). If the frame size is >= direct memory, the broker could not safely buffer a message, so startup aborts with this IllegalArgumentException.

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/PulsarBrokerStarter.java:179

                CmdGenerateDocs cmd = new CmdGenerateDocs("pulsar");
                cmd.addCommand("broker", commander);
                cmd.run(null);
                return 0;
            }

            // init broker config
            if (isBlank(starterArguments.brokerConfigFile)) {
                commander.usage(commander.getOut());
                throw new IllegalArgumentException("Need to specify a configuration file for broker");
            } else {
                final String filepath = Path.of(starterArguments.brokerConfigFile)
                        .toAbsolutePath().normalize().toString();
                brokerConfig = loadConfig(filepath);
            }

            int maxFrameSize = brokerConfig.getMaxMessageSize() + Commands.MESSAGE_SIZE_FRAME_PADDING;
            if (maxFrameSize >= DirectMemoryUtils.jvmMaxDirectMemory()) {
                throw new IllegalArgumentException("Max message size need smaller than jvm directMemory");
            }

            if (!NamespaceBundleSplitAlgorithm.AVAILABLE_ALGORITHMS.containsAll(
                    brokerConfig.getSupportedNamespaceBundleSplitAlgorithms())) {
                throw new IllegalArgumentException(
                        "The given supported namespace bundle split algorithm has unavailable algorithm. "
                                + "Available algorithms are " + NamespaceBundleSplitAlgorithm.AVAILABLE_ALGORITHMS);
            }

            if (!brokerConfig.getSupportedNamespaceBundleSplitAlgorithms().contains(
                    brokerConfig.getDefaultNamespaceBundleSplitAlgorithm())) {
                throw new IllegalArgumentException("Supported namespace bundle split algorithms "
                        + "must contains the default namespace bundle split algorithm");
            }

            // init functions worker
            if (starterArguments.runFunctionsWorker || brokerConfig.isFunctionsWorkerEnabled()) {
                final String filepath = Path.of(starterArguments.fnWorkerConfigFile)

View on GitHub (pinned to 820761864e)

Solutions

Increase JVM -XX:MaxDirectMemorySize so it exceeds maxMessageSize + MESSAGE_SIZE_FRAME_PADDING
Or reduce maxMessageSize in broker.conf to fit under the current direct memory limit
Note Java heap settings (-Xmx) do not affect direct memory; set MaxDirectMemorySize explicitly

Example fix

// before
PULSAR_JVM_OPTS="-XX:MaxDirectMemorySize=1g"  # broker.conf: maxMessageSize=1048576000
// after
PULSAR_JVM_OPTS="-XX:MaxDirectMemorySize=4g"  # keep maxMessageSize=1048576000

Defensive patterns

Strategy: validation

Validate before calling

long maxDirect = sun.misc.VM.maxDirectMemory();
int maxFrame = brokerConf.getMaxMessageSize() + Commands.MESSAGE_SIZE_FRAME_PADDING;
if (maxFrame >= maxDirect) {
    throw new IllegalStateException("Increase -XX:MaxDirectMemorySize above " + maxFrame);
}

Try / catch

try {
    int rc = starter.call();
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("jvm directMemory")) {
        throw new IllegalStateException("Lower maxMessageSize or raise -XX:MaxDirectMemorySize", e);
    }
    throw e;
}

Prevention

Set -XX:MaxDirectMemorySize explicitly (e.g. 2x heap) for brokers with large maxMessageSize
Keep maxMessageSize well below direct memory to leave room for other buffers
Re-check limits when moving to containers with lower memory caps

When it happens

Trigger: broker.conf sets maxMessageSize (or defaultMaxMemory / related sizing) so large that maxMessageSize + padding >= -XX:MaxDirectMemorySize.

Common situations: Raising maxMessageSize to e.g. 1GB while leaving the default 2G (or smaller) MaxDirectMemorySize; container memory limits shrinking the JVM direct memory while a large maxMessageSize config is copied in.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/d1799e2cf94e5d52. Report an issue: GitHub.