{"record":{"id":"1a91ae8fa3ce05dd","repo":"hashicorp/nomad","slug":"failed-to-read-dynamic-plugin-registry-state-v","errorCode":null,"errorMessage":"failed to read dynamic plugin registry state: %v","messagePattern":"failed to read dynamic plugin registry state: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/state/db_bolt.go","lineNumber":962,"sourceCode":"}\n\n// GetDynamicPluginRegistryState stores the dynamic plugin registry's\n// registry state or returns an error.\nfunc (s *BoltStateDB) GetDynamicPluginRegistryState() (*dynamicplugins.RegistryState, error) {\n\tvar ps *dynamicplugins.RegistryState\n\n\terr := s.db.View(func(tx *boltdd.Tx) error {\n\t\tdynamicBkt := tx.Bucket(dynamicPluginBucketName)\n\t\tif dynamicBkt == nil {\n\t\t\t// No state, return\n\t\t\treturn nil\n\t\t}\n\n\t\t// Restore Plugin State if it exists\n\t\tps = &dynamicplugins.RegistryState{}\n\t\tif err := dynamicBkt.Get(registryStateKey, ps); err != nil {\n\t\t\tif !boltdd.IsErrNotFound(err) {\n\t\t\t\treturn fmt.Errorf(\"failed to read dynamic plugin registry state: %v\", err)\n\t\t\t}\n\n\t\t\t// Key not found, reset ps to nil\n\t\t\tps = nil\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ps, nil\n}\n\nfunc keyForCheck(allocID string, checkID structs.CheckID) []byte {\n\treturn fmt.Appendf([]byte{}, \"%s_%s\", allocID, checkID)","sourceCodeStart":944,"sourceCodeEnd":980,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/state/db_bolt.go#L944-L980","documentation":"Returned during state restore when the dynamic plugin registry (CSI/host-volume plugin) state cannot be read from its bucket. Like the other plugin-state errors, a boltdd not-found is tolerated (registry state reset to nil); any other read/decode failure is wrapped and aborts restore.","triggerScenarios":"dynamicBkt.Get(registryStateKey) fails decoding into dynamicplugins.RegistryState or hits a bolt I/O error; typically with a corrupt state.db or payload from an incompatible schema.","commonSituations":"Crash during a CSI plugin state write, Nomad version mismatch in the client data dir, or underlying disk corruption on the node.","solutions":["Inspect the wrapped error to confirm decode vs I/O failure.","Restore state.db from backup; otherwise move it aside and let the client rebuild registry state (CSI plugins re-register and volumes re-detect).","Check storage health on the node before restarting.","Avoid downgrading Nomad on nodes sharing a state directory."],"exampleFix":"// before\nfailed to read dynamic plugin registry state: decoding failed\n// after\nsystemctl stop nomad\nmv /var/lib/nomad/state.db /var/lib/nomad/state.db.corrupt\nsystemctl start nomad","handlingStrategy":"fallback","validationCode":"// preflight: attempt a read of the registry state key\ndb.View(func(tx *boltdd.Tx) error {\n    ps := &dynamicplugins.RegistryState{}\n    return tx.Bucket([]byte(dynamicBktName)).Get(registryStateKey, ps)\n})","typeGuard":"func isNotFound(err error) bool { return errors.Is(err, boltdd.ErrNotFound) }","tryCatchPattern":"if err := restoreDynamicRegistry(); err != nil && !errors.Is(err, boltdd.ErrNotFound) {\n    logger.Warn(\"dropping dynamic registry state; plugins will re-register\", \"error\", err)\n    resetStateDB(stateDir)\n}","preventionTips":["Gracefully stop CSI plugins before agent restarts","Back up state_dir before Nomad version changes","Avoid sharing a state_dir across Nomad versions","Watch for disk-full alerts on client nodes"],"tags":["bolt","state-store","deserialization","csi","nomad"],"backgroundTag":"boltdb-deserialization-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}