apache/cassandra · error · IOException

Corrupt hint file found

Error message

Corrupt hint file found

What it means

HintsReader's iterator (computeNextInternal) throws IOException("Corrupt hint file found") when it reads a zero entry size but verifyAllZeros shows the rest of the file is NOT all zeros. A size of 0 is tolerated only as trailing zero padding produced by hard-rebooting machines; zeros mixed with non-zero garbage mean real corruption, and reading is aborted.

Source

Thrown at src/java/org/apache/cassandra/hints/HintsReader.java:222

                }
            }
            while (hint == null);

            return hint;
        }

        private Hint computeNextInternal() throws IOException
        {
            input.resetCrc();
            input.resetLimit();

            int size = input.readInt();
            if (size == 0)
            {
                // Avoid throwing IOException when a hint file ends with a run of zeros - this
                // can happen when hard-rebooting unresponsive machines.
                if (!verifyAllZeros(input))
                    throw new IOException("Corrupt hint file found");
                throw new EOFException("Unexpected end of file (size == 0)");
            }

            // if we cannot corroborate the size via crc, then we cannot safely skip this hint
            if (!input.checkCrc())
                throw new IOException("Digest mismatch exception");

            return readHint(size);
        }

        private Hint readHint(int size) throws IOException
        {
            applyThrottleRateLimit(size);
            input.limit(size);

            Hint hint;
            try
            {

View on GitHub (pinned to 88fd0f6a0e)

Solutions

  1. Remove/quarantine the corrupt hints file from the hints directory and restart hint delivery.
  2. Check disk/filesystem health (fsck, SMART) — recurring occurrences indicate hardware issues.
  3. Run repair on affected ranges so missed hint deliveries are replaced by anti-entropy repair.
  4. Preserve the file for support/debugging before deleting if the corruption is unexpected.

Example fix

// before
Hint hint = hintsReader.iterator().next(); // IOException: Corrupt hint file found
// after
try (HintsReader reader = HintsReader.open(file))
{
    for (Hint hint : reader)
        deliver(hint);
}
catch (IOException e)
{
    logger.warn("Corrupt hints file {}, skipping", file, e);
}
Defensive patterns

Strategy: try-catch

Validate before calling

// detect a suspicious file before opening: all-zero tail vs mixed garbage requires stream check, so at minimum verify size is a multiple of plausible entry granularity
if (!Files.isRegularFile(file) || Files.size(file) == 0) skipFile(file);

Try / catch

try { for (Hint h : HintsReader.open(file)) deliver(h); } catch (IOException e) { logger.warn("Corrupt hints file {}", file, e); archiveAndSkip(file); }

Prevention

When it happens

Trigger: Reading a hints file after a hard reboot/power loss where the file was extended with zero pages but later regions contain stale non-zero data; disk corruption in the tail of a hints file; a partially overwritten hints file.

Common situations: Datacenter power loss or VM hard-kill; hints on filesystems without proper fsync behavior; faulty disks; manually manipulating hints files.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors


AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/4a6d24f3f193d986. Report an issue: GitHub.