openjdk/jdk · error

OOM error in native tmp buffer allocation

Error message

OOM error in native tmp buffer allocation

What it means

This message is emitted by the native (JNI) side of the JVM's instrumentation agent library (libinstrument) when a malloc for a small temporary path buffer fails. basePath() allocates a copy of the directory portion of a path (everything before the last '/') while resolving the agent JAR's location during -javaagent startup. A failure here means the process is out of native heap memory at agent-load time, and the function returns NULL, aborting agent path resolution and typically failing Agent_OnLoad.

Source

Thrown at src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c:46

#include <string.h>

#include "FileSystemSupport_md.h"

/*
 * Solaris/Linux implementation of the file system support functions.
 */

#define slash           '/'

char* basePath(const char* path) {
    const char* last = strrchr(path, slash);
    if (last == NULL) {
        return (char*)path;
    } else {
        int len = last - path;
        char* str = (char*)malloc(len+1);
        if (str == NULL) {
            fprintf(stderr, "OOM error in native tmp buffer allocation");
            return NULL;
        }
        if (len > 0) {
            memcpy(str, path, len);
        }
        str[len] = '\0';
        return str;
    }
}

int isAbsolute(const char* path) {
    return (path[0] == slash) ? 1 : 0;
}

/* Ported from src/solaris/classes/java/io/UnixFileSystem.java */

/* A normal Unix pathname contains no duplicate slashes and does not end
   with a slash.  It may be the empty string. */

View on GitHub (pinned to 88dfb74bbe)

Solutions

  1. Raise the container/cgroup memory limit or ulimit -v so the JVM plus native allocations fit
  2. Check vm.overcommit_memory / overcommit settings on the host if malloc fails despite free RAM
  3. Reduce native memory pressure at startup (fewer concurrently loaded agents/libs, smaller -XX:MaxMetaspaceSize if metaspace competes for the cap)
  4. Reproduce with a minimal command line (java -javaagent:app.jar -version) to confirm memory, not the agent itself, is the cause

Example fix

# before
docker run -m 128m myapp java -javaagent:/opt/agent.jar -jar app.jar

# after
docker run -m 1g myapp java -javaagent:/opt/agent.jar -jar app.jar
Defensive patterns

Strategy: validation

Validate before calling

// Before launching, verify memory headroom exists for native allocations
// POSIX: check address-space limit is not smothering small mallocs
long addressSpaceLimit = com.sun.management.OperatingSystemMXBean.class.isInstance(osMxBean) ? -1 : -1; // native limits not visible via Java; use shell checks instead:
# ulimit -v  -> should be 'unlimited' or comfortably above JVM+native needs
# free -m    -> verify available memory before 'java -javaagent:...' starts

Try / catch

// Native fprintf+NULL is not catchable from Java; guard at launch time instead.
// If agent load fails, the JVM exits with 'Error opening zip file or JAR manifest missing' style diagnostics —
// treat any -javaagent startup failure as environmental and validate memory before retrying.

Prevention

When it happens

Trigger: Starting the JVM with -javaagent:somepath.jar (or attach-based agent load) on a machine where native memory is exhausted (ulimit -v, cgroup limits, overcommit settings, container memory cap). basePath() is called on the agent JAR path string; only the malloc failure branch prints this.

Common situations: Containers (Docker/Kubernetes) with tight memory limits; ulimit -v set too low; hosts with strict overcommit (vm.overcommit_memory=2); very large native allocations elsewhere before JVM startup; CI runners with constrained memory.

Related errors

AI-assisted analysis of openjdk/jdk@88dfb74bbe (2026-08-14). Data as JSON: /api/errors/0c6b761c86d9e2f2. Report an issue: GitHub.