aeron-io/aeron · critical · AgentTerminationException

failed to start service=<ctx.serviceId()> leadershipTermId=

Error message

failed to start service=<ctx.serviceId()> leadershipTermId=<leadershipTermId> logPosition=<logPosition> clusterTime=<clusterTime> snapshotRecordingId=<snapshotRecordingId>

What it means

Thrown as AgentTerminationException when a recovered service fails to start (onStart or onLoadSnapshot threw) during recovery, after loading a snapshot with the given leadershipTermId/logPosition/snapshotRecordingId. The library wraps the underlying exception because a service that cannot start cannot participate in the cluster; the container cannot make progress.

Solutions

1. Fix the root exception reported as the cause of AgentTerminationException in the service's onStart/onLoadSnapshot
2. Make the service's onLoadSnapshot/onStart defensive against missing or old-format state
3. Recover from an earlier good snapshot or take a fresh one by restarting from an empty log if data loss is acceptable
4. Verify the service class name/instance configuration (aeron.cluster.service.class.name) matches the code that wrote the snapshot

Example fix

// before
public void onLoadSnapshot(InputStream in) {
    state = deserialize(in); // throws on old format
}
// after
public void onLoadSnapshot(InputStream in) {
    try {
        state = deserialize(in);
    } catch (Exception ex) {
        throw new RuntimeException("snapshot load failed", ex);
    }
}

try { service.onLoadSnapshot(in); service.onStart(session); }
catch (AgentTerminationException ex) {
    log("service failed to start from snapshot", ex.getCause());
    // fall back to earlier snapshot or halt node for operator intervention
}

When it happens

Trigger: ClusteredService.onStart() or onLoadSnapshot(SnapshotTaker/InputStream) throws during service container recovery after a snapshot restore; service constructor fails; snapshot bytes are corrupt or written by an incompatible service version.

Common situations: Deploying a new service jar whose onStart deserializes state incompatible with an old snapshot; corrupted or truncated snapshot recording in the archive; service throwing NPE on restored state; rolling upgrade where snapshot app version differs.

Related errors

• snapshot ended unexpectedly
• snapshot ended unexpectedly
• expected schemaId= , actual=
• unexpected snapshot type
• already in snapshot

AI-assisted analysis of aeron-io/aeron@6d60124e15 (2026-09-12). Data as JSON: /api/errors/1169620b5deeea4e. Report an issue: GitHub.

Appendix: source

Thrown at aeron-cluster/src/main/java/io/aeron/cluster/service/ClusteredServiceAgent.java:812

        {
            activeLifecycleCallback = LIFECYCLE_CALLBACK_NONE;
        }

        final long id = ackId++;
        final long relevantId = (null == exception) ? aeron.clientId() : NULL_VALUE;
        while (!consensusModuleProxy.ack(logPosition, clusterTime, id, relevantId, serviceId))
        {
            idle();
        }

        if (null != exception)
        {
            final String message = "failed to start service=" + ctx.serviceId() +
                " leadershipTermId=" + leadershipTermId +
                " logPosition=" + logPosition +
                " clusterTime=" + clusterTime +
                " snapshotRecordingId=" + snapshotRecordingId;
            throw new AgentTerminationException(message, exception);
        }
    }

    private int awaitRecoveryCounter(final CountersReader counters)
    {
        idleStrategy.reset();
        int counterId = RecoveryState.findCounterId(counters, ctx.clusterId());
        while (NULL_COUNTER_ID == counterId)
        {
            idle();
            counterId = RecoveryState.findCounterId(counters, ctx.clusterId());
        }

        return counterId;
    }

    private void closeLog()
    {

View on GitHub (pinned to 6d60124e15)