{"record":{"id":"e6b43cb5c5328baf","repo":"hashicorp/nomad","slug":"invalid-variable-operation-s","errorCode":null,"errorMessage":"Invalid variable operation '%s'","messagePattern":"Invalid variable operation '(.+?)'","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/fsm.go","lineNumber":2404,"sourceCode":"\t\tpanic(fmt.Errorf(\"failed to decode request: %v\", err))\n\t}\n\tdefer metrics.MeasureSinceWithLabels([]string{\"nomad\", \"fsm\", \"apply_sv_operation\"}, time.Now(),\n\t\t[]metrics.Label{{Name: \"op\", Value: string(req.Op)}})\n\tswitch req.Op {\n\tcase structs.VarOpSet:\n\t\treturn n.state.VarSet(msgType, index, &req)\n\tcase structs.VarOpDelete:\n\t\treturn n.state.VarDelete(msgType, index, &req)\n\tcase structs.VarOpDeleteCAS:\n\t\treturn n.state.VarDeleteCAS(msgType, index, &req)\n\tcase structs.VarOpCAS:\n\t\treturn n.state.VarSetCAS(msgType, index, &req)\n\tcase structs.VarOpLockAcquire:\n\t\treturn n.state.VarLockAcquire(msgType, index, &req)\n\tcase structs.VarOpLockRelease:\n\t\treturn n.state.VarLockRelease(msgType, index, &req)\n\tdefault:\n\t\terr := fmt.Errorf(\"Invalid variable operation '%s'\", req.Op)\n\t\tn.logger.Warn(\"Invalid variable operation\", \"operation\", req.Op)\n\t\treturn err\n\t}\n}\n\nfunc (n *nomadFSM) applyRootKeyMetaUpsert(msgType structs.MessageType, buf []byte, index uint64) any {\n\tdefer metrics.MeasureSince([]string{\"nomad\", \"fsm\", \"apply_root_key_meta_upsert\"}, time.Now())\n\n\tvar req structs.KeyringUpdateRootKeyMetaRequest\n\tif err := structs.Decode(buf, &req); err != nil {\n\t\tpanic(fmt.Errorf(\"failed to decode request: %v\", err))\n\t}\n\n\twrappedRootKeys := structs.NewRootKey(req.RootKeyMeta)\n\n\tif err := n.state.UpsertRootKey(index, wrappedRootKeys, req.Rekey); err != nil {\n\t\tn.logger.Error(\"UpsertWrappedRootKeys failed\", \"error\", err)\n\t\treturn err","sourceCodeStart":2386,"sourceCodeEnd":2422,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/fsm.go#L2386-L2422","documentation":"The FSM's variable apply dispatcher switches on the Variables operation type (Set/Delete/CAS/Lock/Release). If req.Op does not match any known VarOp, it returns \"Invalid variable operation '<op>'\" and logs a warning. Since the FSM applies replicated Raft entries, an unknown op means malformed or version-incompatible data reached the state machine.","triggerScenarios":"applyVariableOperation receives a structs.VarOpSet/VarOpDelete/VarOpCAS/VarOpLockAcquire/VarOpLockRelease-unknown Op value — typically a Raft entry written by a newer Nomad version with a variable op this server does not understand.","commonSituations":"Mixed-version server fleets where a newer server replicated a new variable op to older servers; hand-crafted or corrupted Raft log entries; custom tooling submitting variables RPCs with invalid Op values.","solutions":["Check `nomad version` on all servers and upgrade the reporting server to match the cluster's latest version.","Identify which Op string appears in the warning log line; confirm it exists in your Nomad version's structs.VarOp constants.","Ensure clients/tools submit variables via supported RPCs only, never fabricating Op values.","If one server repeatedly logs this while others do not, drain/replace that server as it may be lagging or corrupted."],"exampleFix":"// upgrade the lagging server so it recognizes the variable op\n// before: nomad 1.6.x in a 1.7.x cluster\nnomad agent -server -consul ...  # v1.6\n// after\nnomad agent -server -consul ...  # v1.7 (supports the new variable op)","handlingStrategy":"validation","validationCode":"// ensure the whole server fleet supports the variables feature before use\nfor _, m := range members.Members {\n    if version.Compare(version.Must(version.Parse(m.Tags[\"build\"]), minVarSupportVersion) < 0) {\n        return fmt.Errorf(\"server %s too old for variables ops\", m.Name)\n    }\n}","typeGuard":null,"tryCatchPattern":"_, _, err := client.Variables().Create(var)\nif err != nil && strings.Contains(err.Error(), \"Invalid variable operation\") {\n    // a server cannot parse the op: halt writes, upgrade fleet\n}","preventionTips":["Finish rolling upgrades before using the newest variable ops","Only write variables via official clients/RPCs","Pin server fleet to one Nomad version"],"tags":["nomad","fsm","variables","version-skew"],"backgroundTag":"version-skew-unsupported-operation","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}