{"record":{"id":"ef4b0645ee501314","repo":"nsqio/nsq","slug":"failed-to-parse-metadata-in-s-s","errorCode":null,"errorMessage":"failed to parse metadata in %s - %s","messagePattern":"failed to parse metadata in (.+?) - (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nsqd/nsqd.go","lineNumber":362,"sourceCode":"\nfunc (n *NSQD) LoadMetadata() error {\n\tatomic.StoreInt32(&n.isLoading, 1)\n\tdefer atomic.StoreInt32(&n.isLoading, 0)\n\n\tfn := newMetadataFile(n.getOpts())\n\n\tdata, err := readOrEmpty(fn)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif data == nil {\n\t\treturn nil // fresh start\n\t}\n\n\tvar m Metadata\n\terr = json.Unmarshal(data, &m)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to parse metadata in %s - %s\", fn, err)\n\t}\n\n\tfor _, t := range m.Topics {\n\t\tif !protocol.IsValidTopicName(t.Name) {\n\t\t\tn.logf(LOG_WARN, \"skipping creation of invalid topic %s\", t.Name)\n\t\t\tcontinue\n\t\t}\n\t\ttopic := n.GetTopic(t.Name)\n\t\tif t.Paused {\n\t\t\tif err := topic.Pause(); err != nil {\n\t\t\t\tn.logf(LOG_ERROR, \"TOPIC(%s): failed to pause while loading metadata - %s\", t.Name, err)\n\t\t\t}\n\t\t}\n\t\tfor _, c := range t.Channels {\n\t\t\tif !protocol.IsValidChannelName(c.Name) {\n\t\t\t\tn.logf(LOG_WARN, \"skipping creation of invalid channel %s\", c.Name)\n\t\t\t\tcontinue\n\t\t\t}","sourceCodeStart":344,"sourceCodeEnd":380,"githubUrl":"https://github.com/nsqio/nsq/blob/85cf10c09c6c3c86160d6f0eb156f62d0efc1648/nsqd/nsqd.go#L344-L380","documentation":"nsqd found --data-path/nsqd.dat but json.Unmarshal of its bytes into the Metadata struct failed, meaning the file exists yet is not valid JSON matching the expected shape ({\"topics\":[{\"name\":..., \"paused\":..., \"channels\":[...]}]}). The error includes the file path and the encoding/json reason ('unexpected token', 'invalid character ...', truncated JSON). loadNsqdMetadata aborts, so topics/channels are not restored.","triggerScenarios":"nsqd.dat truncated by a crash mid-write (writeSyncFile interrupted); manual editing that left trailing commas or comments; a version downgrade reading a newer format; writing the file with an external tool/UTF-16/BOM encoding; an empty-but-present file (0 bytes) after a bad disk.","commonSituations":"Power loss or OOM-kill during persistMetadata; operators hand-editing nsqd.dat to pre-create topics; restoring from backup with a partially copied file; disk-full during the atomic-free direct write used by writeSyncFile.","solutions":["Validate and inspect: `cat /var/lib/nsqd/nsqd.dat | jq .` — jq pinpoints the syntax error location.","Restore a good copy from backup if topic/channel state matters.","If state is expendable, archive and restart fresh: `mv nsqd.dat nsqd.dat.bak && systemctl start nsqd` (topics reappear as clients re-create them).","If hands-on repair is preferred, fix the JSON (remove comments/trailing commas), keep the schema {\"topics\":[...]}, and ensure the process can re-read it.","Check disk health/fullness (`df -h`, dmesg for I/O errors) so the corruption does not recur."],"exampleFix":"# before (nsqd.dat truncated by crash)\n{\"topics\":[{\"name\":\"orders\",\"paused\":false,\"chan\n\n# after (repaired minimal valid file)\n{\"topics\":[{\"name\":\"orders\",\"paused\":false,\"channels\":[]}]}\n\n# or discard state\nmv /var/lib/nsqd/nsqd.dat /var/lib/nsqd/nsqd.dat.bak","handlingStrategy":"validation","validationCode":"// preflight: JSON-parse the metadata file yourself\ndata, _ := os.ReadFile(filepath.Join(dataPath, \"nsqd.dat\"))\nif len(data) > 0 {\n    var probe struct{ Topics []struct{ Name string } }\n    if err := json.Unmarshal(data, &probe); err != nil {\n        return fmt.Errorf(\"nsqd.dat corrupt: %w — restore backup or archive it\", err)\n    }\n}","typeGuard":"func looksLikeMetadata(b []byte) bool {\n    var m map[string]any\n    return json.Unmarshal(b, &m) == nil && m[\"topics\"] != nil\n}","tryCatchPattern":null,"preventionTips":["Back up nsqd.dat on a schedule (it is small) so corruption is recoverable.","Never hand-edit nsqd.dat; manage topics via nsqd HTTP API or clients.","Monitor disk fullness — writes under ENOSPC truncate files."],"tags":["nsqd","json","corruption","metadata","persistence","startup"],"backgroundTag":null,"analyzedSha":"85cf10c09c6c3c86160d6f0eb156f62d0efc1648","analyzedAt":"2026-08-16T00:53:05.009Z","schemaVersion":2},"datasetVersion":"2026-08-16T03:17:38.424Z"}