What causes "Failed to query for root keys: %v" in hashicorp/nomad?

Inspect the embedded error to identify whether it is a decode or store-open failure and address that root cause.

Failed to query for root keys: %v

Error message

Failed to query for root keys: %v

What it means

After restoring the snapshot into an in-memory store, RedactSnapshot enumerates root keys via store.RootKeys(nil) to rewrite them. This error wraps a failure of that key iteration, meaning the restored keyring store could not be opened or scanned. The underlying store error is embedded with %v (unwrappable).

Source

Thrown at helper/raftutil/snapshot.go:64

	select {
	case err := <-errCh:
		return nil, nil, nil, err
	case meta := <-metaCh:
		return fsm, fsm.State(), meta, nil
	}
}

func RedactSnapshot(srcFile *os.File) error {
	srcFile.Seek(0, 0)
	fsm, store, meta, err := RestoreFromArchive(srcFile, nil)
	if err != nil {
		return fmt.Errorf("Failed to load snapshot from archive: %w", err)
	}

	iter, err := store.RootKeys(nil)
	if err != nil {
		return fmt.Errorf("Failed to query for root keys: %v", err)
	}

	for {
		raw := iter.Next()
		if raw == nil {
			break
		}
		rootKey := raw.(*structs.RootKey)
		if rootKey == nil {
			break
		}
		if len(rootKey.WrappedKeys) > 0 {
			rootKey.KeyID = rootKey.KeyID + " [REDACTED]"
			rootKey.WrappedKeys = nil
		}
		msg, err := structs.Encode(structs.WrappedRootKeysUpsertRequestType,
			&structs.KeyringUpsertWrappedRootKeyRequest{
				WrappedRootKeys: rootKey,

View on GitHub (pinned to 482b49bf1a)

Solutions

Inspect the embedded error to identify whether it is a decode or store-open failure and address that root cause.
Re-take the snapshot from a healthy cluster and retry the redaction.
Verify snapshot was produced by a compatible Consul version; upgrade the redaction tool if the keyring format is newer.
If only keyring data is corrupt, consider extracting state without keyring redaction or restoring the snapshot to a test cluster to regenerate it.

Defensive patterns

Strategy: try-catch

Try / catch

if err := raftutil.RedactSnapshot(f); err != nil {
    if strings.Contains(err.Error(), "Failed to query for root keys") {
        log.Printf("keyring unreadable after restore: %v — re-take snapshot from a healthy cluster", err)
    }
}

Prevention

Validate the snapshot with the vendor's inspect/verify command first.
Match tool version to the cluster version that wrote the keyring.
Test redaction on a copy before processing the only snapshot you have.

When it happens

Trigger: store.RootKeys(nil) returning an error after a successful restore — typically an in-memory store iteration/decode failure on the keyring data restored from the snapshot, or an unexpected store state (nil/misconfigured store passed to the FSM).

Common situations: Redacting snapshots whose keyring entries were written by very old or newer Consul versions with changed keyring encoding, or snapshots where the secure/keyring area is corrupt even though the archive parses.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c9fc80ef1ee5b9c8. Report an issue: GitHub.