hashicorp/nomad · critical

downgrading Raft is not supported, current version is %d, pr

Error message

downgrading Raft is not supported, current version is %d, previous version was %d

What it means

Nomad persists the previously used Raft protocol version in a Raft version file under the data dir. On startup, if the stored previous version is higher than the configured raft.protocol_version, startup refuses to proceed, because downgrading the Raft protocol can corrupt the log store. This protects against rolling back an upgraded cluster.

Source

Thrown at nomad/server.go:1733

		s.logger.Warn(fmt.Sprintf("unable to read Raft version file, %s", baseWarning), "error", err)
		return nil
	}

	v, err := os.ReadFile(path)
	if err != nil {
		s.logger.Warn(fmt.Sprintf("unable to read Raft version file, %s", baseWarning), "error", err)
		return nil
	}

	previousVersion, err := strconv.Atoi(strings.TrimSpace(string(v)))
	if err != nil {
		s.logger.Warn(fmt.Sprintf("invalid Raft protocol version in Raft version file, %s", baseWarning), "error", err)
		return nil
	}

	if raft.ProtocolVersion(previousVersion) > raftVersion {
		return fmt.Errorf("downgrading Raft is not supported, current version is %d, previous version was %d", raftVersion, previousVersion)
	}

	return nil
}

// setupSerf is used to setup and initialize a Serf
func (s *Server) setupSerf(conf *serf.Config, ch chan serf.Event, path string) (*serf.Serf, error) {
	conf.Init()
	conf.NodeName = fmt.Sprintf("%s.%s", s.config.NodeName, s.config.Region)
	conf.Tags["role"] = "nomad"
	conf.Tags["region"] = s.config.Region
	conf.Tags["dc"] = s.config.Datacenter
	conf.Tags["build"] = s.config.Build
	conf.Tags["revision"] = s.config.Revision
	conf.Tags["vsn"] = deprecatedAPIMajorVersionStr // for Nomad <= v1.2 compat
	conf.Tags["raft_vsn"] = fmt.Sprintf("%d", s.config.RaftConfig.ProtocolVersion)
	conf.Tags["id"] = s.config.NodeID
	conf.Tags["rpc_addr"] = s.clientRpcAdvertise.(*net.TCPAddr).IP.String()         // Address that clients will use to RPC to servers

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Restore protocol_version in the server config to the value previously used (typically 3) and restart
2. If you truly must run an older Nomad, do NOT reuse the upgraded data_dir — start with an empty data dir
3. Verify the Raft version file under data_dir/raft/ is intact; if corrupt, correct it or restore from backup
4. Plan upgrades as one-way: snapshot the data dir before upgrading so rollback uses the backup, not the live dir

Example fix

// before
server {
  protocol_version = 2
}
// after
server {
  protocol_version = 3
}

// Compare configured protocol_version against the stored Raft version file before start
const configured = 3; // from server config
const verFile = dataDir + '/raft/versionfile'; // location per Nomad docs
if (fs.existsSync(verFile)) {
  const prev = parseInt(fs.readFileSync(verFile, 'utf8').trim(), 10);
  if (!Number.isNaN(prev) && prev > configured) throw new Error('downgrading raft protocol is not supported');
}

try {
  startNomadServer();
} catch (e) {
  if (String(e).includes('downgrading Raft is not supported')) {
    throw new Error('Restore protocol_version=' + extractPrevVersion(e) + ' in config, or start with an empty data_dir');
  }
  throw e;
}

When it happens

Trigger: Server start where raft.ProtocolVersion(previousVersion) > raftVersion — i.e. the agent previously ran with a higher protocol_version than the current config value, or the Raft version file is corrupt/stale (the code path logs 'invalid Raft protocol version in Raft version file' as a warning just above).

Common situations: Operator rolled back a Nomad upgrade from 0.8+ to an older binary while keeping the same data_dir; config management (Chef/Puppet/Terraform) reset protocol_version from 3 to 2; copying a data dir from a newer cluster to an older node.

Related errors

• unsupported minimum common raft protocol version
• raft_protocol must be 3 in Nomad v1.4 and later, got %d
• failed to decode log entry at index %d: %v
• existing BoltDB raft store found at %s; run 'nomad operator
• unsupported raft log store backend: %q

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/44fd4308d5d45855. Report an issue: GitHub.