apache/cassandra · error · FSReadError

FSReadError

Error message

FSReadError

What it means

FSReadError is Cassandra's wrapper for unexpected filesystem read failures, thrown in HintsReader.computeNext when reading a hint file raises an IOException that is not a benign EOF. It indicates the hints file on disk cannot be read (I/O error rather than a truncated/zero-filled tail).

Source

Thrown at src/java/org/apache/cassandra/hints/HintsReader.java:309

                if (input.isEOF())
                    return endOfData(); // reached EOF

                if (position.subtract(offset) >= PAGE_SIZE)
                    return endOfData(); // read page size or more bytes

                try
                {
                    buffer = computeNextInternal();
                }
                catch (EOFException e)
                {
                    logger.warn("Unexpected EOF replaying hints ({}), likely due to unflushed hint file on shutdown; continuing", descriptor.fileName(), e);
                    return endOfData();
                }
                catch (IOException e)
                {
                    throw new FSReadError(e, file);
                }
            }
            while (buffer == null);

            return buffer;
        }

        private ByteBuffer computeNextInternal() throws IOException
        {
            input.resetCrc();
            input.resetLimit();

            int size = input.readInt();
            if (size == 0)
            {
                // Avoid throwing IOException when a hint file ends with a run of zeros - this
                // can happen when hard-rebooting unresponsive machines.
                if (!verifyAllZeros(input))

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Check the node's system.log for the wrapped IOException cause and the file name, and inspect disk health (dmesg, smartctl)
2. Verify permissions and existence of files in the hints directory (hinted_handoff directory, default /var/lib/cassandra/hints)
3. If the file is corrupt, move the offending hints file out of the hints directory and restart replay
4. Restore from backup or let other replicas' hints/divergence-repair (e.g. repair) cover the data

Example fix

// before: failing hard on unreadable hint file
// move the unreadable file aside instead
mv /var/lib/cassandra/hints/<uuid>-1-1.hints /tmp/bad_hints/
// then restart the node or re-trigger dispatch

File dir = new File(hintsDir);
if (!dir.canRead() || !dir.isDirectory()) throw new IllegalStateException("hints dir unreadable");

boolean isReadableHintsFile(File f) { return f != null && f.isFile() && f.canRead(); }

try { reader.readNext(); } catch (FSReadError e) { logger.error("Hints read failed", e); file.renameTo(new File(file + ".corrupt")); }

When it happens

Trigger: Reading hints files during hint replay when the underlying file read fails: disk errors, permission problems, file deleted/renamed underneath the reader, or checksum-region reads throwing IOException.

Common situations: Failing disk or bad sector in the hints directory; hints file removed by an external cleanup script while replay is in progress; permissions changed on the hints directory; NFS/storage faults.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• FSWriteError
• FSWriteError
• Failed importing SSTables
• Unable to resolve canonical path for
• Digest mismatch exception

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/8a389c33d7cc1c75. Report an issue: GitHub.