{"record":{"id":"710babc6f7b9010a","repo":"hashicorp/nomad","slug":"failed-to-configure-keyring-v","errorCode":null,"errorMessage":"failed to configure keyring: %v","messagePattern":"failed to configure keyring: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"command/agent/agent.go","lineNumber":1195,"sourceCode":"\tif !a.config.Server.Enabled {\n\t\treturn nil\n\t}\n\n\t// Setup the configuration\n\tconf, err := a.serverConfig()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"server config setup failed: %s\", err)\n\t}\n\n\t// Generate a node ID and persist it if it is the first instance, otherwise\n\t// read the persisted node ID.\n\tif err := a.setupNodeID(conf); err != nil {\n\t\treturn fmt.Errorf(\"setting up server node ID failed: %s\", err)\n\t}\n\n\t// Sets up the keyring for gossip encryption\n\tif err := a.setupKeyrings(conf); err != nil {\n\t\treturn fmt.Errorf(\"failed to configure keyring: %v\", err)\n\t}\n\n\t// Create the server\n\tserver, err := nomad.NewServer(conf,\n\t\ta.consulCatalog,           // self service discovery\n\t\ta.consulConfigEntriesFunc, // writing config entries for gateways\n\t)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"server setup failed: %v\", err)\n\t}\n\ta.server = server\n\n\t// Consul check addresses default to bind but can be toggled to use advertise\n\trpcCheckAddr := a.config.normalizedAddrs.RPC\n\tserfCheckAddr := a.config.normalizedAddrs.Serf\n\n\tdefaultConsul := conf.ConsulConfigs[structs.ConsulDefaultCluster]\n","sourceCodeStart":1177,"sourceCodeEnd":1213,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/command/agent/agent.go#L1177-L1213","documentation":"setupServer initializes the Serf gossip keyring via setupKeyrings when server.encrypt is configured. If the keyring cannot be created, loaded, or installed (bad key format, missing keyring file, permission issues), the error is wrapped as this message and the agent fails to start.","triggerScenarios":"server.encrypt set to a key that is not a valid 16/32-byte base64 gossip key, a keyring file in data_dir/server/keystate that cannot be parsed or written, or mismatched keys when joining a cluster — during NewAgent.","commonSituations":"Hand-generating an encrypt key that is not 16 or 32 bytes base64, keyring files left over with different keys after cluster rekey, permission problems on data_dir/server/keystate, or copying data_dir between clusters.","solutions":["Generate the key with `nomad operator keygen` and put exactly that in server.encrypt","Check data_dir/server/keystate for stale/corrupt keyring files; remove or fix them (cluster-wide key consistency required)","Fix filesystem permissions on the keystate directory","Ensure all servers use the same key/number of keys when rekeying"],"exampleFix":"// before\nserver {\n  enabled = true\n  encrypt = \"mysecretkey\"\n}\n// after (shell: nomad operator keygen)\nserver {\n  enabled = true\n  encrypt = \"pUqJrVyVRj5jsiYEkM/tFQYfiIxD9hP8Bc3Uw3AzwU0=\"\n}","handlingStrategy":"validation","validationCode":"// Validate gossip key before configuring server.encrypt\nkey := \"pUqJrVyVRj5jsiYEkM/tFQYfiIxD9hP8Bc3Uw3AzwU0=\"\nraw, err := base64.StdEncoding.DecodeString(key)\nif err != nil || (len(raw) != 16 && len(raw) != 32) {\n    log.Fatalf(\"server.encrypt must be base64 of 16 or 32 bytes: use `nomad operator keygen`\")\n}","typeGuard":null,"tryCatchPattern":"if err := setupKeyrings(conf); err != nil {\n    return fmt.Errorf(\"failed to configure keyring: %w\", err)\n}","preventionTips":["Generate keys only with `nomad operator keygen`","Keep keyring files under data_dir/server/keystate backed up and permission-protected","Coordinate rekeys across all servers to avoid key mismatches","Verify keystate directory writability before restarts"],"tags":["nomad","server","gossip","encryption","keyring"],"backgroundTag":"gossip-keyring-error","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}