{"record":{"id":"775b45f8298ed57a","repo":"hashicorp/nomad","slug":"failed-to-read-device-manager-plugin-state-v","errorCode":null,"errorMessage":"failed to read device manager plugin state: %v","messagePattern":"failed to read device manager plugin state: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/state/db_bolt.go","lineNumber":869,"sourceCode":"}\n\n// GetDevicePluginState stores the device manager's plugin state or returns an\n// error.\nfunc (s *BoltStateDB) GetDevicePluginState() (*dmstate.PluginState, error) {\n\tvar ps *dmstate.PluginState\n\n\terr := s.db.View(func(tx *boltdd.Tx) error {\n\t\tdevBkt := tx.Bucket(devManagerBucket)\n\t\tif devBkt == nil {\n\t\t\t// No state, return\n\t\t\treturn nil\n\t\t}\n\n\t\t// Restore Plugin State if it exists\n\t\tps = &dmstate.PluginState{}\n\t\tif err := devBkt.Get(managerPluginStateKey, ps); err != nil {\n\t\t\tif !boltdd.IsErrNotFound(err) {\n\t\t\t\treturn fmt.Errorf(\"failed to read device manager plugin state: %v\", err)\n\t\t\t}\n\n\t\t\t// Key not found, reset ps to nil\n\t\t\tps = nil\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ps, nil\n}\n\n// PutDriverPluginState stores the driver manager's plugin state or returns an\n// error.","sourceCodeStart":851,"sourceCodeEnd":887,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/state/db_bolt.go#L851-L887","documentation":"Returned during restore of the Nomad client state database when the device manager's plugin-state bucket cannot be read from BoltDB. This wraps the underlying deserialization/IO error returned by boltdd.Get and only fires for errors other than a simple key-not-found (which is handled by resetting ps to nil).","triggerScenarios":"boltdd.Tx.Get on the device-manager bucket with managerPluginStateKey fails with an error other than boltdd.ErrNotFound — e.g. corrupted bytes that fail msgpack unmarshal into dmstate.PluginState, or a bolt I/O error reading the page.","commonSituations":"Corrupt or truncated state.db after a hard crash/power loss, disk errors, or a state.db written by an incompatible Nomad version so the stored PluginState payload no longer deserializes.","solutions":["Inspect the wrapped %v error: if it is a decode error, the state.db is corrupt — stop the Nomad agent and back up the state directory.","Restore state.db from state.db.backup (created during upgrades) or from your backups.","If no backup exists, move state.db aside and restart the agent; the client will re-register and rebuild device plugin state.","Check disk health and free space; run a filesystem check if I/O errors are reported."],"exampleFix":"// before (corrupt state.db blocks startup)\nagent start -> failed to read device manager plugin state: decoding failed\n// after\nsystemctl stop nomad\nmv /var/lib/nomad/state.db /var/lib/nomad/state.db.corrupt\nsystemctl start nomad # client re-initializes state","handlingStrategy":"fallback","validationCode":"// before starting agent: sanity-check state db\nimport bolt \"go.etcd.io/bbolt\"\ndb, err := bolt.Open(stateDBPath, 0o600, &bolt.Options{ReadOnly: true})\nif err != nil { /* restore from state.db.backup */ }","typeGuard":"func isNotFound(err error) bool { return errors.Is(err, boltdd.ErrNotFound) }","tryCatchPattern":"if err := restore(...); err != nil {\n    logger.Error(\"state restore failed\", \"error\", err)\n    // failover: archive corrupt db and reinitialize\n    os.Rename(stateDBPath, stateDBPath+\".corrupt\")\n}","preventionTips":["Enable periodic backups of the client state_dir","Use UPS on nodes running Nomad clients to prevent crash corruption","Keep state_dir on reliable, monitored storage","Monitor agent logs for early decode warnings"],"tags":["bolt","state-store","deserialization","nomad"],"backgroundTag":"boltdb-deserialization-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}