What causes "OOM error in native tmp buffer allocation" in openjdk/jdk?

Increase the container/process memory limit or reduce -Xmx to leave native headroom

OOM error in native tmp buffer allocation

Error message

OOM error in native tmp buffer allocation

What it means

getModuleObject failed to malloc a small buffer for the package name before calling JVMTI GetNamedModule (used when the agent asks the JVM to load a class into a module via ToInstrumentableModuleName/transform). It prints the message and returns NULL, so that instrumentation lookup fails. Pure out-of-native-memory condition.

Source

Thrown at src/java.instrument/share/native/libinstrument/JPLISAgent.c:788

            agent->mRedefineAdded = JNI_TRUE;
        }
    }
}

static jobject
getModuleObject(jvmtiEnv*               jvmti,
                jobject                 loaderObject,
                const char*             cname) {
    jvmtiError err = JVMTI_ERROR_NONE;
    jobject moduleObject = NULL;

    /* find last slash in the class name */
    const char* last_slash = (cname == NULL) ? NULL : strrchr(cname, '/');
    int len = (last_slash == NULL) ? 0 : (int)(last_slash - cname);
    char* pkg_name_buf = (char*)malloc(len + 1);

    if (pkg_name_buf == NULL) {
        fprintf(stderr, "OOM error in native tmp buffer allocation");
        return NULL;
    }
    if (last_slash != NULL) {
        strncpy(pkg_name_buf, cname, len);
    }
    pkg_name_buf[len] = '\0';

    err = (*jvmti)->GetNamedModule(jvmti, loaderObject, pkg_name_buf, &moduleObject);
    free((void*)pkg_name_buf);
    check_phase_ret_blob(err, NULL);
    jplis_assert_msg(err == JVMTI_ERROR_NONE, "error in the JVMTI GetNamedModule");

    return moduleObject;
}

/*
 *  Support for the JVMTI callbacks
 */

View on GitHub (pinned to 88dfb74bbe)

Solutions

Increase the container/process memory limit or reduce -Xmx to leave native headroom
Reduce retransformation batch sizes so transient native allocations are smaller
Use NativeMemoryTracking (jcmd VM.native_memory) to find native consumers
Check for leaks in other native agents / off-heap caches with pmap or jemalloc profiling

Example fix

# before
java -Xmx8g --container-memory=8g -javaagent:agent.jar -jar app.jar
# after: leave native headroom
java -Xmx6g --container-memory=8g -javaagent:agent.jar -jar app.jar

Defensive patterns

Strategy: validation

Validate before calling

// before heavy retransformation, assert native headroom
((com.sun.management.OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean());
long free = Runtime.getRuntime().freeMemory(); // heap proxy only; pair with container RSS metric externally

Prevention

Keep container limit - max heap >= a few hundred MB for native allocations
Batch class retransformations instead of transforming everything at once

When it happens

Trigger: Native heap exhausted when the agent performs class-file transformation that resolves a class's module; container memory cap reached; RSS limits.

Common situations: Heavy retransformation workloads under tight container limits; JVMs with large heaps on small physical memory; leaky co-loaded native libraries consuming the process address space.

Related errors

AI-assisted analysis of openjdk/jdk@88dfb74bbe (2026-08-14). Data as JSON: /api/errors/c150f3c8b4c6cfe5. Report an issue: GitHub.