{"record":{"id":"503bd57ae9a2c4e5","repo":"hashicorp/consul","slug":"failed-to-decode-request-v","errorCode":null,"errorMessage":"failed to decode request: %v","messagePattern":"failed to decode request: (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"agent/consul/fsm/commands_ce.go","lineNumber":166,"sourceCode":"\tregisterCommand(structs.PeeringWriteType, (*FSM).applyPeeringWrite)\n\tregisterCommand(structs.PeeringDeleteType, (*FSM).applyPeeringDelete)\n\tregisterCommand(structs.PeeringTerminateByIDType, (*FSM).applyPeeringTerminate)\n\tregisterCommand(structs.PeeringTrustBundleWriteType, (*FSM).applyPeeringTrustBundleWrite)\n\tregisterCommand(structs.PeeringTrustBundleDeleteType, (*FSM).applyPeeringTrustBundleDelete)\n\tregisterCommand(structs.PeeringSecretsWriteType, (*FSM).applyPeeringSecretsWrite)\n\tregisterCommand(structs.ResourceOperationType, (*FSM).applyResourceOperation)\n\tregisterCommand(structs.UpdateVirtualIPRequestType, (*FSM).applyManualVirtualIPs)\n}\n\nfunc (c *FSM) applyRegister(buf []byte, index uint64) interface{} {\n\tdefer metrics.MeasureSince([]string{\"fsm\", \"register\"}, time.Now())\n\tvar req structs.RegisterRequest\n\tif err := decodeRegistrationReq(buf, &req); err != nil {\n\t\tif errors.Is(err, ErrDroppingTenantedReq) {\n\t\t\tc.logger.Warn(\"dropping tenanted register request\")\n\t\t\treturn nil\n\t\t}\n\t\tpanic(fmt.Errorf(\"failed to decode request: %v\", err))\n\t}\n\n\t// Apply all updates in a single transaction\n\tif err := c.state.EnsureRegistration(index, &req); err != nil {\n\t\tc.logger.Warn(\"EnsureRegistration failed\", \"error\", err)\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc (c *FSM) applyDeregister(buf []byte, index uint64) interface{} {\n\tdefer metrics.MeasureSince([]string{\"fsm\", \"deregister\"}, time.Now())\n\tvar req structs.DeregisterRequest\n\tif err := decodeDeregistrationReq(buf, &req); err != nil {\n\t\tif errors.Is(err, ErrDroppingTenantedReq) {\n\t\t\tc.logger.Warn(\"dropping tenanted deregister request\")\n\t\t\treturn nil\n\t\t}","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/hashicorp/consul/blob/2397ff0d763d34f2fe37fe59fde6a7f7fc430a3e/agent/consul/fsm/commands_ce.go#L148-L184","documentation":"FSM apply handler for catalog registration RPCs on Consul servers. Every committed Raft log entry of type RegisterRequest is msgpack-decoded here; if decoding fails — and the error is not the ErrDroppingTenantedReq sentinel that CE downgrade mode maps to a warning and drop — the FSM panics deliberately. An undecodable committed log means the cluster's state machines disagree, and halting the server is considered safer than skipping the entry.","triggerScenarios":"A committed RegisterRequest log that this binary cannot msgpack-decode: mixed server versions writing/reading different schemas, replaying a snapshot or raft logs from an incompatible Consul version, corrupted raft storage, or tenanted enterprise data replayed on a CE build whose downgrade decode path fails outside the recognized sentinel.","commonSituations":"Rolling upgrades where servers temporarily run different versions; restoring snapshots across major versions; running enterprise (namespaced/partitioned) data against a CE binary; disk corruption in the raft/data directories.","solutions":["Verify all servers run the same Consul version and agree on the raft protocol (consul version, consul members, operator raft list-peers)","Check preceding logs for 'dropping tenanted register request' warnings — they indicate enterprise tenanted data hitting a CE build in downgrade mode","If versions are mixed, complete or roll back the upgrade so a single version writes and applies raft logs","If corruption is suspected, restore the node from a known-good snapshot or reseed its raft directory","If versions are consistent and no tenanted data is involved, preserve the raft logs and open an issue with HashiCorp"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# before upgrade/restore, verify snapshot integrity and version compatibility\nconsul snapshot inspect /path/to/snapshot.tgz\nconsul version   # run on every server; all peers must match","typeGuard":null,"tryCatchPattern":"// embedded-Consul usage ONLY: recover in an FSM wrapper to convert the panic\n// to a crash report. Never do this in a real server: skipping a committed\n// register log forks the state machine.\nfunc (w *WrappedFSM) Apply(log *raft.Log) (resp interface{}) {\n    defer func() {\n        if r := recover(); r != nil {\n            w.log.Error(\"fsm panic\", \"index\", log.Index, \"panic\", r)\n            resp = fmt.Errorf(\"fsm apply failed: %v\", r)\n        }\n    }()\n    return w.inner.Apply(log)\n}","preventionTips":["Upgrade servers one version path at a time and never leave a cluster with mixed versions writing raft logs","Validate snapshots with 'consul snapshot inspect' before restoring","Keep raft/data directories on reliable storage; monitor disk health","Before reverting enterprise to CE, ensure no tenanted data remains in recent raft logs"],"tags":["go","consul","raft","fsm","msgpack","panic","data-corruption"],"backgroundTag":null,"analyzedSha":"2397ff0d763d34f2fe37fe59fde6a7f7fc430a3e","analyzedAt":"2026-08-15T19:19:47.700Z","schemaVersion":2},"datasetVersion":"2026-08-16T03:17:38.424Z"}