{"record":{"id":"d1799e2cf94e5d52","repo":"apache/pulsar","slug":"max-message-size-need-smaller-than-jvm-directmemor","errorCode":null,"errorMessage":"Max message size need smaller than jvm directMemory","messagePattern":"Max message size need smaller than jvm directMemory","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"pulsar-broker/src/main/java/org/apache/pulsar/PulsarBrokerStarter.java","lineNumber":179,"sourceCode":"                CmdGenerateDocs cmd = new CmdGenerateDocs(\"pulsar\");\n                cmd.addCommand(\"broker\", commander);\n                cmd.run(null);\n                return 0;\n            }\n\n            // init broker config\n            if (isBlank(starterArguments.brokerConfigFile)) {\n                commander.usage(commander.getOut());\n                throw new IllegalArgumentException(\"Need to specify a configuration file for broker\");\n            } else {\n                final String filepath = Path.of(starterArguments.brokerConfigFile)\n                        .toAbsolutePath().normalize().toString();\n                brokerConfig = loadConfig(filepath);\n            }\n\n            int maxFrameSize = brokerConfig.getMaxMessageSize() + Commands.MESSAGE_SIZE_FRAME_PADDING;\n            if (maxFrameSize >= DirectMemoryUtils.jvmMaxDirectMemory()) {\n                throw new IllegalArgumentException(\"Max message size need smaller than jvm directMemory\");\n            }\n\n            if (!NamespaceBundleSplitAlgorithm.AVAILABLE_ALGORITHMS.containsAll(\n                    brokerConfig.getSupportedNamespaceBundleSplitAlgorithms())) {\n                throw new IllegalArgumentException(\n                        \"The given supported namespace bundle split algorithm has unavailable algorithm. \"\n                                + \"Available algorithms are \" + NamespaceBundleSplitAlgorithm.AVAILABLE_ALGORITHMS);\n            }\n\n            if (!brokerConfig.getSupportedNamespaceBundleSplitAlgorithms().contains(\n                    brokerConfig.getDefaultNamespaceBundleSplitAlgorithm())) {\n                throw new IllegalArgumentException(\"Supported namespace bundle split algorithms \"\n                        + \"must contains the default namespace bundle split algorithm\");\n            }\n\n            // init functions worker\n            if (starterArguments.runFunctionsWorker || brokerConfig.isFunctionsWorkerEnabled()) {\n                final String filepath = Path.of(starterArguments.fnWorkerConfigFile)","sourceCodeStart":161,"sourceCodeEnd":197,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-broker/src/main/java/org/apache/pulsar/PulsarBrokerStarter.java#L161-L197","documentation":"At broker startup, maxFrameSize = brokerConfig.getMaxMessageSize() + Commands.MESSAGE_SIZE_FRAME_PADDING is checked against the JVM's max direct memory (DirectMemoryUtils.jvmMaxDirectMemory()). If the frame size is >= direct memory, the broker could not safely buffer a message, so startup aborts with this IllegalArgumentException.","triggerScenarios":"broker.conf sets maxMessageSize (or defaultMaxMemory / related sizing) so large that maxMessageSize + padding >= -XX:MaxDirectMemorySize.","commonSituations":"Raising maxMessageSize to e.g. 1GB while leaving the default 2G (or smaller) MaxDirectMemorySize; container memory limits shrinking the JVM direct memory while a large maxMessageSize config is copied in.","solutions":["Increase JVM -XX:MaxDirectMemorySize so it exceeds maxMessageSize + MESSAGE_SIZE_FRAME_PADDING","Or reduce maxMessageSize in broker.conf to fit under the current direct memory limit","Note Java heap settings (-Xmx) do not affect direct memory; set MaxDirectMemorySize explicitly"],"exampleFix":"// before\nPULSAR_JVM_OPTS=\"-XX:MaxDirectMemorySize=1g\"  # broker.conf: maxMessageSize=1048576000\n// after\nPULSAR_JVM_OPTS=\"-XX:MaxDirectMemorySize=4g\"  # keep maxMessageSize=1048576000","handlingStrategy":"validation","validationCode":"long maxDirect = sun.misc.VM.maxDirectMemory();\nint maxFrame = brokerConf.getMaxMessageSize() + Commands.MESSAGE_SIZE_FRAME_PADDING;\nif (maxFrame >= maxDirect) {\n    throw new IllegalStateException(\"Increase -XX:MaxDirectMemorySize above \" + maxFrame);\n}","typeGuard":null,"tryCatchPattern":"try {\n    int rc = starter.call();\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().contains(\"jvm directMemory\")) {\n        throw new IllegalStateException(\"Lower maxMessageSize or raise -XX:MaxDirectMemorySize\", e);\n    }\n    throw e;\n}","preventionTips":["Set -XX:MaxDirectMemorySize explicitly (e.g. 2x heap) for brokers with large maxMessageSize","Keep maxMessageSize well below direct memory to leave room for other buffers","Re-check limits when moving to containers with lower memory caps"],"tags":["configuration","memory","startup"],"backgroundTag":"direct-memory-exhausted","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}