{"record":{"id":"ec65a537eabae2a2","repo":"aeron-io/aeron","slug":"agentterminationexception-storage-space-error-on-local","errorCode":null,"errorMessage":"AgentTerminationException (storage space error on local archive)","messagePattern":"AgentTerminationException \\(storage space error on local archive\\)","errorType":"exception","errorClass":"AgentTerminationException","httpStatus":null,"severity":"critical","filePath":"aeron-cluster/src/main/java/io/aeron/cluster/ClusterBackupAgent.java","lineNumber":1073,"sourceCode":"\n        if (null != backupArchive)\n        {\n            final RecordingSignalPoller poller = this.recordingSignalPoller;\n            workCount += poller.poll();\n\n            if (poller.isPollComplete())\n            {\n                final int templateId = poller.templateId();\n\n                if (ControlResponseDecoder.TEMPLATE_ID == templateId && poller.code() == ControlResponseCode.ERROR)\n                {\n                    final ArchiveException ex = new ArchiveException(\n                        poller.errorMessage(), (int)poller.relevantId(), poller.correlationId());\n\n                    if (ex.errorCode() == ArchiveException.STORAGE_SPACE)\n                    {\n                        ctx.countedErrorHandler().onError(ex);\n                        throw new AgentTerminationException();\n                    }\n                    else\n                    {\n                        throw ex;\n                    }\n                }\n                else if (RecordingSignalEventDecoder.TEMPLATE_ID == templateId && null != snapshotReplication)\n                {\n                    snapshotReplication.onSignal(\n                        poller.correlationId(),\n                        poller.recordingId(),\n                        poller.recordingPosition(),\n                        poller.recordingSignal());\n                }\n            }\n            else if (0 == workCount && !poller.subscription().isConnected())\n            {\n                ctx.countedErrorHandler().onError(new ClusterEvent(\"local archive not connected\"));","sourceCodeStart":1055,"sourceCodeEnd":1091,"githubUrl":"https://github.com/aeron-io/aeron/blob/6d60124e15e35c11b49ba2e3c2c2858a09a18803/aeron-cluster/src/main/java/io/aeron/cluster/ClusterBackupAgent.java#L1055-L1091","documentation":"The backup agent polls the local archive for snapshot recording progress; if the archive returns an error with errorCode STORAGE_SPACE, the agent reports it and throws AgentTerminationException because backup cannot proceed without storage. Other archive errors are rethrown as-is.","triggerScenarios":"While retrieving a snapshot via the archive control poller, the poller surfaces an ArchiveException whose errorCode() == ArchiveException.STORAGE_SPACE.","commonSituations":"Backup node disk full (snapshot/replay writes exhausted space), undersized volume, leftover recordings filling the archive directory.","solutions":["Free disk space on the backup node's archive storage","Increase the volume/quota backing the aeron archive data directory","Purge old/failed backup recordings from the archive","Add monitoring/alerting on disk usage before it fills"],"exampleFix":"// before\ndf -h # 100% on /var/lib/aeron/archive\n// after\nfind /var/lib/aeron/archive -name '*.rec' -mtime +7 -delete\ndf -h # space reclaimed, restart backup","handlingStrategy":"validation","validationCode":"File archiveDir = new File(ctx.archiveDirName());\nif (archiveDir.getUsableSpace() < minRequiredBytes) {\n    throw new IllegalStateException(\"insufficient archive storage: \" + archiveDir.getUsableSpace());\n}","typeGuard":null,"tryCatchPattern":"try {\n    backup.pollSnapshotProgress();\n} catch (AgentTerminationException e) {\n    // STORAGE_SPACE: free space or expand volume before restarting\n}","preventionTips":["Alert at 70-80% disk usage on archive volumes","Schedule cleanup of obsolete backup recordings","Size volumes for the largest snapshot plus replay overhead"],"tags":["aeron","storage","disk-full","archive"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"6d60124e15e35c11b49ba2e3c2c2858a09a18803","analyzedAt":"2026-09-12T11:17:07.683Z","contentChangedAt":"2026-09-12T11:17:07.683Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}