aeron-io/aeron · critical · ClusterException

recording stopped unexpectedly

Error message

recording stopped unexpectedly: ${recordingId}

What it means

Thrown as ClusterException while a leader records a snapshot: the RecordingPos counter for the archive recording stopped being active before the recording reached the expected position, i.e. the archive recording ended unexpectedly instead of completing the snapshot.

Solutions

  1. Check archive and driver error logs for why the recording aborted
  2. Ensure the archive has sufficient disk space and correct recording configuration
  3. Retry the snapshot (take a new one) after fixing the archive state
  4. Verify catalog/recording log integrity if the recording was marked complete prematurely
Defensive patterns

Strategy: retry

Validate before calling

if (!RecordingPos.isActive(counters, counterId, recordingId)) {
    throw new IllegalStateException("recording " + recordingId + " not active before reaching target position");
}

Try / catch

try { awaitRecordingPosition(recordingId, targetPos); }
catch (ClusterException ex) { /* re-trigger snapshot after verifying archive health */ }

Prevention

When it happens

Trigger: While awaiting snapshot recording completion, RecordingPos.isActive(counters, counterId, recordingId) returns false before the recording position reaches the target; archive stopped the recording; recording failed mid-stream; archive reached its max capacity or was restarted.

Common situations: Media driver or archive crash during snapshot; snapshot publication failure causing the recorder to abort; archive configuration (recording events) removed the counter; disk full on the archive host.

Related errors

AI-assisted analysis of aeron-io/aeron@6d60124e15 (2026-09-12). Data as JSON: /api/errors/2f10ce8a40324453. Report an issue: GitHub.

Appendix: source

Thrown at aeron-cluster/src/main/java/io/aeron/cluster/service/ClusteredServiceAgent.java:1036

        }
    }

    private void awaitRecordingComplete(
        final long recordingId,
        final long position,
        final CountersReader counters,
        final int counterId,
        final AeronArchive archive)
    {
        idleStrategy.reset();
        while (counters.getCounterValue(counterId) < position)
        {
            idle();
            archive.checkForErrorResponse();

            if (!RecordingPos.isActive(counters, counterId, recordingId))
            {
                throw new ClusterException("recording stopped unexpectedly: " + recordingId);
            }
        }
    }

    private void snapshotState(
        final ExclusivePublication publication, final long logPosition, final long leadershipTermId)
    {
        final ServiceSnapshotTaker snapshotTaker = new ServiceSnapshotTaker(
            publication, idleStrategy, aeronAgentInvoker);

        snapshotTaker.markBegin(SNAPSHOT_TYPE_ID, logPosition, leadershipTermId, 0, timeUnit, ctx.appVersion());

        for (int i = 0, size = sessions.size(); i < size; i++)
        {
            snapshotTaker.snapshotSession(sessions.get(i));
        }

        snapshotTaker.markEnd(SNAPSHOT_TYPE_ID, logPosition, leadershipTermId, 0, timeUnit, ctx.appVersion());

View on GitHub (pinned to 6d60124e15)