elastic/elasticsearch · critical · IllegalStateException

Third party audit was killed buy SIGKILL, could be a victim

Error message

Third party audit was killed buy SIGKILL, could be a victim of the Linux OOM killer

What it means

Thrown by ThirdPartyAuditTask.runForbiddenAPIsCli when running on Linux and the forbiddenapis CLI process exited with the SIGKILL exit value (137). Because forbiddenapis is launched with -Xmx1g, an exit code 137 on Linux almost always means the process exceeded available memory and was killed by the kernel OOM killer (or an external SIGKILL).

Source

Thrown at build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/precommit/ThirdPartyAuditTask.java:417

                || isJavaVersion(VERSION_22)
                || isJavaVersion(VERSION_23)
                || isJavaVersion(VERSION_24)
                || isJavaVersion(VERSION_25)
                || isJavaVersion(VERSION_26)
                || isJavaVersion(VERSION_27)) {
                spec.jvmArgs("--add-modules", "jdk.incubator.vector");
            }
            spec.jvmArgs("-Xmx1g");
            spec.getMainClass().set("de.thetaphi.forbiddenapis.cli.CliMain");
            spec.args("-f", getSignatureFile().getAbsolutePath(), "-d", getJarExpandDir(), "--debug", "--allowmissingclasses");
            spec.setErrorOutput(errorOut);
            if (getLogger().isInfoEnabled() == false) {
                spec.setStandardOutput(new NullOutputStream());
            }
            spec.setIgnoreExitValue(true);
        });
        if (OS.current().equals(OS.LINUX) && result.getExitValue() == SIG_KILL_EXIT_VALUE) {
            throw new IllegalStateException("Third party audit was killed buy SIGKILL, could be a victim of the Linux OOM killer");
        }
        final String forbiddenApisOutput;
        try (ByteArrayOutputStream outputStream = errorOut) {
            forbiddenApisOutput = outputStream.toString(StandardCharsets.UTF_8);
        }
        if (EXPECTED_EXIT_CODES.contains(result.getExitValue()) == false) {
            throw new IllegalStateException("Forbidden APIs cli failed: " + forbiddenApisOutput);
        }
        return forbiddenApisOutput;
    }

    /** Returns true iff the build Java version is the same as the given version. */
    private boolean isJavaVersion(JavaVersion version) {
        if (getRuntimeJavaVersion().isPresent()) {
            return getRuntimeJavaVersion().get().equals(version);
        }
        return version.getMajorVersion().equals(VersionProperties.getBundledJdkMajorVersion());
    }

View on GitHub (pinned to db6a809a66)

Solutions

Free memory on the host or raise the container/cgroup memory limit above 1g plus JVM overhead (~1.5g safe).
Reduce parallelism (e.g. --max-workers) so the forbiddenapis JVM has headroom, or run the audit task in isolation.
If the heap is genuinely too small for the audited jar set, investigate why the jar set is abnormally large (e.g. a fat/uber jar accidentally on the audit classpath).
Check dmesg / kernel logs for 'Out of memory: Killed process' to confirm OOM-kill; if absent, look for an external process sending SIGKILL.

Defensive patterns

Strategy: retry

Validate before calling

// Provision enough memory before running the audit
long free = Runtime.getRuntime().freeMemory();
long limit = parseCgroupMemoryLimit(); // from /sys/fs/cgroup/memory.max
if (limit > 0 && limit < 1_500_000_000L) {
    throw new IllegalStateException("cgroup memory limit too low for forbiddenapis (-Xmx1g): " + limit);
}

Try / catch

// Retry once with reduced parallelism if the audit is OOM-killed
try { runForbiddenApis(); }
catch (IllegalStateException e) {
    if (e.getMessage().contains("SIGKILL")) { runForbiddenApis(serial=true); }
    else throw e;
}

Prevention

Size CI containers to at least 1.5g for tasks that run forbiddenapis.
Limit --max-workers when many projects run the audit concurrently.
Monitor dmesg for OOM-kill events during release builds.

When it happens

Trigger: The javaexec invocation of de.thetaphi.forbiddenapis.cli.CliMain is killed by the OS: setIgnoreExitValue(true) lets the task read the exit code, and on Linux a value of 137 (SIG_KILL_EXIT_VALUE) trips this branch. This occurs when the host is memory-constrained or the cgroup memory limit is too low for a 1g heap plus classloading overhead.

Common situations: CI containers with tight memory limits; running a large multi-project build in parallel starving the forbiddenapis JVM; a host under heavy load where the OOM killer targets the audit process; Docker memory limits lower than -Xmx1g + JVM overhead.

Related errors

AI-assisted analysis of elastic/elasticsearch@db6a809a66 (2026-08-12). Data as JSON: /api/errors/e44d85cce5704102. Report an issue: GitHub.