{"record":{"id":"ab569894d8a94f13","repo":"apache/cassandra","slug":"failed-to-deserialize-heartbeat-file-falling-b","errorCode":null,"errorMessage":"Failed to deserialize heartbeat file {}. Falling back to file last modified time.","messagePattern":"Failed to deserialize heartbeat file (.+?)\\. Falling back to file last modified time\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/service/DataResurrectionCheck.java","lineNumber":236,"sourceCode":"\n        Map<String, Object> config = configuration.getConfig(name());\n        File heartbeatFile = getHeartbeatFile(config);\n\n        if (!heartbeatFile.exists())\n        {\n            LOGGER.debug(\"Heartbeat file {} not found! Skipping heartbeat startup check.\", heartbeatFile.absolutePath());\n            return;\n        }\n\n        Heartbeat heartbeat;\n\n        try\n        {\n            heartbeat = Heartbeat.deserializeFromJsonFile(heartbeatFile);\n        }\n        catch (IOException ex)\n        {\n            LOGGER.warn(\"Failed to deserialize heartbeat file {}. Falling back to file last modified time.\",\n                        heartbeatFile, ex);\n            Instant lastModified = Instant.ofEpochMilli(heartbeatFile.lastModified());\n            heartbeat = new Heartbeat(lastModified);\n        }\n\n        if (heartbeat.lastHeartbeat == null)\n            return;\n\n        long heartbeatMillis = heartbeat.lastHeartbeat.toEpochMilli();\n\n        List<Pair<String, String>> violations = new ArrayList<>();\n\n        Set<String> excludedKeyspaces = getExcludedKeyspaces(config);\n        Set<Pair<String, String>> excludedTables = getExcludedTables(config);\n\n        long minimumThresholdMillis = getMinimumThresholdMillis(config);\n\n        long currentTimeMillis = currentTimeMillis();","sourceCodeStart":218,"sourceCodeEnd":254,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/service/DataResurrectionCheck.java#L218-L254","documentation":"During execute(), the data-resurrection check reads a heartbeat JSON file; if it cannot be deserialized (IOException), the check falls back to using the file's last-modified timestamp as the heartbeat instant. The check still runs, but with a potentially less accurate heartbeat time.","triggerScenarios":"Heartbeat.deserializeFromJsonFile(heartbeatFile) throws IOException: the heartbeat JSON is corrupt/truncated, empty, or unreadable due to permissions or concurrent writes during an unclean shutdown.","commonSituations":"Unclean node shutdown that left a partial heartbeat file; manual edits to the heartbeat file with invalid JSON; disk/permission issues on the heartbeat directory; version mismatch where the file format changed.","solutions":["Inspect the heartbeat JSON file named in the log; delete it so a fresh one is written on next clean start, accepting the last-modified fallback","Fix JSON syntax errors if the file was hand-edited","Check file permissions/ownership and disk health for the heartbeat file's directory","Ensure clean shutdowns (nodetool drain/stop) so the heartbeat file is written atomically"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"// validate heartbeat file before the check reads it\nFile f = heartbeatFile;\nif (!f.exists() || f.length() == 0 || !f.canRead())\n    log.warn(\"Heartbeat file {} missing/empty/unreadable; expect last-modified fallback\", f);","typeGuard":null,"tryCatchPattern":"try {\n    heartbeat = Heartbeat.deserializeFromJsonFile(heartbeatFile);\n} catch (IOException ex) {\n    LOGGER.warn(\"Failed to deserialize heartbeat file {}. Falling back to file last modified time.\", heartbeatFile, ex);\n    heartbeat = new Heartbeat(Instant.ofEpochMilli(heartbeatFile.lastModified()));\n}","preventionTips":["Ensure clean shutdowns so the heartbeat JSON is fully written","Never hand-edit the heartbeat file; regenerate by clean restart","Verify write permissions and disk health of the heartbeat directory","Keep the heartbeat file on a local, non-network filesystem to avoid partial writes"],"tags":["json","heartbeat","fallback"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}