nsqio/nsq · error

failed to parse metadata in %s - %s

Error message

failed to parse metadata in %s - %s

What it means

nsqd found --data-path/nsqd.dat but json.Unmarshal of its bytes into the Metadata struct failed, meaning the file exists yet is not valid JSON matching the expected shape ({"topics":[{"name":..., "paused":..., "channels":[...]}]}). The error includes the file path and the encoding/json reason ('unexpected token', 'invalid character ...', truncated JSON). loadNsqdMetadata aborts, so topics/channels are not restored.

Source

Thrown at nsqd/nsqd.go:362

func (n *NSQD) LoadMetadata() error {
	atomic.StoreInt32(&n.isLoading, 1)
	defer atomic.StoreInt32(&n.isLoading, 0)

	fn := newMetadataFile(n.getOpts())

	data, err := readOrEmpty(fn)
	if err != nil {
		return err
	}
	if data == nil {
		return nil // fresh start
	}

	var m Metadata
	err = json.Unmarshal(data, &m)
	if err != nil {
		return fmt.Errorf("failed to parse metadata in %s - %s", fn, err)
	}

	for _, t := range m.Topics {
		if !protocol.IsValidTopicName(t.Name) {
			n.logf(LOG_WARN, "skipping creation of invalid topic %s", t.Name)
			continue
		}
		topic := n.GetTopic(t.Name)
		if t.Paused {
			if err := topic.Pause(); err != nil {
				n.logf(LOG_ERROR, "TOPIC(%s): failed to pause while loading metadata - %s", t.Name, err)
			}
		}
		for _, c := range t.Channels {
			if !protocol.IsValidChannelName(c.Name) {
				n.logf(LOG_WARN, "skipping creation of invalid channel %s", c.Name)
				continue
			}

View on GitHub (pinned to 85cf10c09c)

Solutions

1. Validate and inspect: `cat /var/lib/nsqd/nsqd.dat | jq .` — jq pinpoints the syntax error location.
2. Restore a good copy from backup if topic/channel state matters.
3. If state is expendable, archive and restart fresh: `mv nsqd.dat nsqd.dat.bak && systemctl start nsqd` (topics reappear as clients re-create them).
4. If hands-on repair is preferred, fix the JSON (remove comments/trailing commas), keep the schema {"topics":[...]}, and ensure the process can re-read it.
5. Check disk health/fullness (`df -h`, dmesg for I/O errors) so the corruption does not recur.

Example fix

# before (nsqd.dat truncated by crash)
{"topics":[{"name":"orders","paused":false,"chan

# after (repaired minimal valid file)
{"topics":[{"name":"orders","paused":false,"channels":[]}]}

# or discard state
mv /var/lib/nsqd/nsqd.dat /var/lib/nsqd/nsqd.dat.bak

// preflight: JSON-parse the metadata file yourself
data, _ := os.ReadFile(filepath.Join(dataPath, "nsqd.dat"))
if len(data) > 0 {
    var probe struct{ Topics []struct{ Name string } }
    if err := json.Unmarshal(data, &probe); err != nil {
        return fmt.Errorf("nsqd.dat corrupt: %w — restore backup or archive it", err)
    }
}

func looksLikeMetadata(b []byte) bool {
    var m map[string]any
    return json.Unmarshal(b, &m) == nil && m["topics"] != nil
}

When it happens

Trigger: nsqd.dat truncated by a crash mid-write (writeSyncFile interrupted); manual editing that left trailing commas or comments; a version downgrade reading a newer format; writing the file with an external tool/UTF-16/BOM encoding; an empty-but-present file (0 bytes) after a bad disk.

Common situations: Power loss or OOM-kill during persistMetadata; operators hand-editing nsqd.dat to pre-create topics; restoring from backup with a partially copied file; disk-full during the atomic-free direct write used by writeSyncFile.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• failed to read metadata from %s - %s
• failed to append certificate to pool
• unknown tlsVersionOption %q
• cannot flock directory %s - %s (possibly in use by another i
• failed to lock data-path: %v

AI-assisted analysis of nsqio/nsq@85cf10c09c (2026-08-16). Data as JSON: /api/errors/ef4b0645ee501314. Report an issue: GitHub.