aeron-io/aeron · error · ClusterException

failed to write recording

Error message

failed to write recording

What it means

When rewriting the recording log, ClusterToolOperator writes each entry into a ByteBuffer and calls fileChannel.write at the target position, verifying the return value equals the entry length. A short write throws ClusterException("failed to write recording"). This guards against partial writes corrupting the recording log.

Solutions

  1. Free disk space on the filesystem holding the recording log and retry the operation.
  2. Check filesystem health (dmesg / mount status) for I/O errors on the storage device.
  3. Ensure no other process is writing to the recording log concurrently; run tools on a quiescent node.
  4. Retry the tool command after resolving the storage issue; verify the recording log with ClusterTool.describeRecordingLog.

Example fix

// before
tool.recordingLogAction(clusterDir, ...); // fails on full disk
// after
df -h /var/aeron/cluster  # ensure free space first
tool.recordingLogAction(clusterDir, ...);
Defensive patterns

Strategy: retry

Validate before calling

// before tool operation
File clusterDir = new File(dirArg);
if (clusterDir.getFreeSpace() < MIN_REQUIRED_BYTES)
{
    throw new IllegalStateException("insufficient disk space for recording log rewrite");
}

Try / catch

try
{
    tool.recordingLogAction(clusterDir, ...);
}
catch (ClusterException ex)
{
    if (!"failed to write recording".equals(ex.getMessage())) throw ex;
    // free space / fix storage, then retry once
}

Prevention

When it happens

Trigger: Running a ClusterTool operation that rewrites the recording log file where FileChannel.write returns fewer bytes than the entry length (disk full, I/O error, underlying storage issue).

Common situations: Disk full or quota exceeded on the volume holding the cluster directory; filesystem errors (ENOSPC, EIO); writing to a network mount that drops writes; concurrent modification of the recording log.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of aeron-io/aeron@6d60124e15 (2026-09-12). Data as JSON: /api/errors/48a707ad7910e905. Report an issue: GitHub.

Appendix: source

Thrown at aeron-cluster/src/main/java/io/aeron/cluster/ClusterToolOperator.java:1407

            }
            else
            {
                final Path newRecordingLog = clusterDir.toPath().resolve(RecordingLog.RECORDING_LOG_FILE_NAME + ".tmp");
                Files.deleteIfExists(newRecordingLog);
                final ByteBuffer byteBuffer =
                    ByteBuffer.allocateDirect(RecordingLog.MAX_ENTRY_LENGTH).order(LITTLE_ENDIAN);
                final UnsafeBuffer buffer = new UnsafeBuffer(byteBuffer);
                try (FileChannel fileChannel = FileChannel.open(newRecordingLog, CREATE_NEW, WRITE))
                {
                    long position = 0;
                    for (final RecordingLog.Entry e : entries)
                    {
                        RecordingLog.writeEntryToBuffer(e, buffer);
                        byteBuffer.limit(e.length()).position(0);

                        if (e.length() != fileChannel.write(byteBuffer, position))
                        {
                            throw new ClusterException("failed to write recording");
                        }
                        position += e.length();
                    }
                }
                finally
                {
                    BufferUtil.free(byteBuffer);
                }

                Files.delete(recordingLog);
                Files.move(newRecordingLog, recordingLog);
            }
        }
        catch (final IOException ex)
        {
            throw new UncheckedIOException(ex);
        }
    }

View on GitHub (pinned to 6d60124e15)