hashicorp/nomad · error

recovery failed to parse peers.json: %v

Error message

recovery failed to parse peers.json: %v

What it means

When recovering Raft configuration from a manually supplied peers.json, Nomad parses it with raft.ReadPeersJSON (protocol < 3) or raft.ReadConfigJSON (>= 3). This error wraps the parse failure, meaning the file is missing, unreadable, or not valid JSON in the expected peer-list/configuration format.

Source

Thrown at nomad/server.go:1557

			// Blow away the peers.json file if present, since the
			// peers.info sentinel wasn't there.
			if _, err := os.Stat(peersFile); err == nil {
				if err := os.Remove(peersFile); err != nil {
					return fmt.Errorf("failed to delete peers.json, please delete manually (see peers.info for details): %v", err)
				}
				s.logger.Info("deleted peers.json file (see peers.info for details)")
			}
		} else if _, err := os.Stat(peersFile); err == nil {
			s.logger.Info("found peers.json file, recovering Raft configuration...")
			var configuration raft.Configuration
			if s.config.RaftConfig.ProtocolVersion < 3 {
				configuration, err = raft.ReadPeersJSON(peersFile)
			} else {
				configuration, err = raft.ReadConfigJSON(peersFile)
			}
			if err != nil {
				return fmt.Errorf("recovery failed to parse peers.json: %v", err)
			}
			tmpFsm, err := NewFSM(fsmConfig)
			if err != nil {
				return fmt.Errorf("recovery failed to make temp FSM: %v", err)
			}
			if err := raft.RecoverCluster(s.config.RaftConfig, tmpFsm,
				log, stable, snap, trans, configuration); err != nil {
				return fmt.Errorf("recovery failed: %v", err)
			}
			if err := os.Remove(peersFile); err != nil {
				return fmt.Errorf("recovery failed to delete peers.json, please delete manually (see peers.info for details): %v", err)
			}
			s.logger.Info("deleted peers.json file after successful recovery")
		}
	}

	// If we are a single server cluster and the state is clean then we can
	// bootstrap now.

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Validate peers.json is correct JSON in the format matching your raft protocol version (raft.Configuration object for protocol >= 3, e.g. {"server_id":"...","server_addr":"..."} entries under "Servers")
2. For Nomad >= 0.8 use the raft Configuration JSON format, not the legacy flat peer list
3. Recreate the file from the documented template with the current cluster server IDs and addresses
4. Ensure the file is readable by the Nomad user and not truncated (cat the file, run it through jq)

Example fix

// before (legacy format)
["10.0.0.1:4647","10.0.0.2:4647"]
// after (raft.Configuration format, protocol >= 3)
{"Servers":[{"ID":"7e4e2d0a-...","Address":"10.0.0.1:4647"},{"ID":"a1b2c3d4-...","Address":"10.0.0.2:4647"}]}

// Validate peers.json parses and has the expected shape before restart
const cfg = JSON.parse(fs.readFileSync(dataDir + '/peers.json', 'utf8'));
if (!Array.isArray(cfg.Servers) || cfg.Servers.length === 0)
  throw new Error('peers.json must be a raft.Configuration with a non-empty Servers array (protocol >= 3)');
for (const s of cfg.Servers) if (!s.ID || !s.Address) throw new Error('each server needs ID and Address');

try {
  startNomadServer();
} catch (e) {
  if (String(e).includes('recovery failed to parse peers.json')) {
    console.error('Fix peers.json format:', e.message);
  }
  throw e;
}

When it happens

Trigger: Server start with peers.json present and no peers.info, and the file cannot be read/parsed — wrong JSON schema (e.g. old flat array vs raft.Configuration object), trailing garbage, wrong protocol-version format, or missing file mid-flight.

Common situations: Operator hand-edited peers.json and broke the syntax; copied a pre-0.8 format peers.json into a modern Nomad (needs raft.Configuration format with ProtocolVersion >= 3); protocol_version config mismatch with the file's format; file truncated during copy.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• unsupported raft log store backend: %q
• failed to delete peers.json, please delete manually (see pee
• recovery failed to make temp FSM: %v
• recovery failed: %v
• recovery failed to delete peers.json, please delete manually

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/4708ce92d291bcfe. Report an issue: GitHub.