apache/cassandra · error · StartupException

ERR_WRONG_DISK_STATE

ERR_WRONG_DISK_STATE

Error message

FS ownership check failed; error when checking for fs ownership file

What it means

Part of Cassandra's filesystem ownership verification: while walking up the data directory tree looking for the .cassandra/ownership (fs ownership) properties file, an exception occurred reading a file that was found. The check cannot complete, so startup/attach fails with ERR_WRONG_DISK_STATE.

Solutions

1. Fix permissions/ownership of the .cassandra/ownership file so the Cassandra process can read it (`chown cassandra:cassandra …; chmod 644 …)
2. Regenerate the ownership file (re-run the ownership initialization tooling / remove the corrupt file and recreate it) and restart
3. Verify the data directories and any network mounts are mounted and readable before starting Cassandra

Example fix

// before
cassandra@cassandra /mnt/data/.cassandra/ownership  (owned by root, mode 600)
// after
sudo chown cassandra:cassandra /mnt/data/.cassandra/ownership && sudo chmod 644 /mnt/data/.cassandra/ownership

File f = new File(dataDir, ".cassandra/ownership");
if (f.exists()) {
    try (FileReader r = new FileReader(f)) { new Properties().load(r); }
    catch (IOException e) { throw new IllegalStateException("Ownership file unreadable: " + f, e); }
}

static boolean ownershipFileReadable(Path p) {
    try (var in = Files.newInputStream(p)) { in.read(); return true; }
    catch (IOException e) { return false; }
}

try {
    FileSystemOwnershipCheck.executeAndFail(...);
} catch (FileSystemOwnershipcheck.FileSystemOwnershipException e) {
    if (e.getCode() == FileSystemOwnershipcheck.ErrorCode.READ_EXCEPTION) {
        // fix permissions / regenerate ownership file, then restart
    } else throw e;
}

When it happens

Trigger: Calling executeAndFail/execute during startup with fs ownership checking enabled, when an ownership marker file exists under a data directory but opening/loading its Properties throws an I/O error (permissions, corrupt file, unreadable path).

Common situations: Ownership file has wrong file permissions (not readable by the cassandra user); the file is corrupt or truncated after a crash or bad sync; the file is on a mount that became unavailable; file copied with incorrect ownership.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Could not perform startup sequence and join cluster
• ERR_WRONG_MACHINE_STATE
• 3
• Cannot delete the directory
• Cannot move the file

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/3d702f2f22953d83. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/service/FileSystemOwnershipCheck.java:171

            Path dir = File.getPath(dataDir).normalize();
            do
            {
                File tokenFile = resolve(dir, tokenFilename);
                if (tokenFile.exists())
                {
                    foundFiles++;
                    if (!foundProperties.containsKey(tokenFile.toPath().toAbsolutePath()))
                    {
                        try (BufferedReader reader = Files.newBufferedReader(tokenFile.toPath()))
                        {
                            Properties props = new Properties();
                            props.load(reader);
                            foundProperties.put(tokenFile.toPath().toAbsolutePath(), props);
                        }
                        catch (Exception e)
                        {
                            logger.error("Error reading fs ownership file from disk", e);
                            throw exception(READ_EXCEPTION);
                        }
                    }
                }
                dir = dir.getParent();
            } while (dir != null);

            foundPerTargetDir.put(dataDir, foundFiles);
        }

        // If a marker file couldn't be found for every target directory, error.
        if (foundPerTargetDir.containsValue(0))
        {
            throw exception(String.format(NO_OWNERSHIP_FILE, foundPerTargetDir.entrySet()
                                                                              .stream()
                                                                              .filter(e -> e.getValue() == 0)
                                                                              .map(Map.Entry::getKey)
                                                                              .collect(Collectors.joining("', '", "'", "'"))));
        }

View on GitHub (pinned to 88fd0f6a0e)