What causes "Reaper process died unexpectedly! Check the log at {}" in elastic/elasticsearch?

Read reaper.log at the path in the exception (typically .gradle/reaper/build- /reaper.log) for the crash cause.

elastic/elasticsearch · error · IllegalStateException

Reaper process died unexpectedly! Check the log at

Error message

Reaper process died unexpectedly! Check the log at {}

What it means

ReaperService.ensureReaperAlive() throws IllegalStateException when reaperProcess.isAlive() is false. This is checked before issuing commands to the reaper (registerCommand, registerPid) and during shutdown. It means the reaper JVM launched in ensureReaperStarted() has exited while the Gradle build is still expecting it to be available — i.e., the reaper crashed. The log at <inputDir>/reaper.log has the reaper's output.

Solutions

Read reaper.log at the path in the exception (typically .gradle/reaper/build-<pid>/reaper.log) for the crash cause.
If the reaper is OOMing, investigate the .cmd workload or the reaper's heap budget.
Check dmesg/system logs for OOM-killer or external signal delivery to the reaper PID.
Re-run the build; transient reaper crashes (e.g., external kill) are often non-reproducible.

Defensive patterns

Strategy: try-catch

Try / catch

try { reaperService.registerCommand(id, cmd); } catch (IllegalStateException e) { /* reaper died; read reaper.log */ Path log = Path.of(e.getMessage().replaceAll(".*log at ", "")); throw e; }

Prevention

Inspect reaper.log at the path in the exception for the crash cause.
Investigate OOM-killer or external signals if the reaper JVM is being killed.
Re-run the build — transient external kills are often non-reproducible.

When it happens

Trigger: After the reaper process is started, any subsequent call to ensureReaperStarted() (which calls ensureReaperAlive() on the existing process) or shutdown() (via ensureReaperAlive()) finds the process dead. Also fires if registerPid/registerCommand is called after a crash.

Common situations: The reaper JVM crashed with an uncaught exception (OOM due to -Xmx16m being too small for a pathological input, a bug in the reaper); the reaper was killed externally (OOM killer, manual kill, container reap); the reaper exited normally but a stale reference is being used.

Related errors

AI-assisted analysis of elastic/elasticsearch@db6a809a66 (2026-08-12). Data as JSON: /api/errors/978b1ded768e45c4. Report an issue: GitHub.

Appendix: source

Thrown at build-tools/src/main/java/org/elasticsearch/gradle/ReaperService.java:165

            }

            try (
                OutputStream out = Files.newOutputStream(jarPath);
                InputStream jarInput = this.getClass().getResourceAsStream("/META-INF/reaper.jar");
            ) {
                logger.info("Copying reaper.jar...");
                jarInput.transferTo(out);
            } catch (IOException e) {
                throw new UncheckedIOException(e);
            }

            return jarPath;
        }
    }

    private void ensureReaperAlive() {
        if (reaperProcess.isAlive() == false) {
            throw new IllegalStateException("Reaper process died unexpectedly! Check the log at " + logFilePath().toString());
        }
    }

    @Override
    public void close() throws Exception {
        shutdown();
    }

    public interface Params extends BuildServiceParameters {
        Boolean getInternal();

        void setInternal(Boolean internal);

        DirectoryProperty getBuildDir();

        DirectoryProperty getInputDir();
    }
}

View on GitHub (pinned to db6a809a66)