{"record":{"id":"4fb5bad6ee80943f","repo":"hashicorp/nomad","slug":"failed-to-read-dynamic-plugin-registry-state-v-4fb5ba","errorCode":null,"errorMessage":"failed to read dynamic plugin registry state: %v","messagePattern":"failed to read dynamic plugin registry state: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/state/upgrade.go","lineNumber":335,"sourceCode":"\t\tif err := putTaskStateImpl(tx, allocID, taskName, taskState); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc UpgradeDynamicPluginRegistry(logger hclog.Logger, tx *boltdd.Tx) error {\n\n\tdynamicBkt := tx.Bucket(dynamicPluginBucketName)\n\tif dynamicBkt == nil {\n\t\treturn nil // no previous plugins upgrade\n\t}\n\n\toldState := &RegistryState12{}\n\tif err := dynamicBkt.Get(registryStateKey, oldState); err != nil {\n\t\tif !boltdd.IsErrNotFound(err) {\n\t\t\treturn fmt.Errorf(\"failed to read dynamic plugin registry state: %v\", err)\n\t\t}\n\t}\n\n\tnewState := &dynamicplugins.RegistryState{\n\t\tPlugins: make(map[string]map[string]*list.List),\n\t}\n\n\tfor ptype, plugins := range oldState.Plugins {\n\t\tnewState.Plugins[ptype] = make(map[string]*list.List)\n\t\tfor pname, pluginInfo := range plugins {\n\t\t\tnewState.Plugins[ptype][pname] = list.New()\n\t\t\tentry := list.Element{Value: pluginInfo}\n\t\t\tnewState.Plugins[ptype][pname].PushFront(entry)\n\t\t}\n\t}\n\treturn dynamicBkt.Put(registryStateKey, newState)\n}\n","sourceCodeStart":317,"sourceCodeEnd":353,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/state/upgrade.go#L317-L353","documentation":"UpgradeDynamicPluginRegistry reads the dynamic plugin RegistryState12 blob from the client state store. If dynamicBkt.Get fails with anything other than boltdd's not-found error, the upgrade cannot read prior plugin registrations and returns this wrapped error to abort the migration.","triggerScenarios":"tx.Bucket(dynamicPluginBucketName) exists but Get(registryStateKey, &RegistryState12{}) fails with a non-NotFound error (msgpack decode failure of the stored blob, or an internal boltdd/bbolt read error).","commonSituations":"Corrupted dynamic plugin registry state after an unclean shutdown on a client using CSI/docker plugin tasks; schema mismatch when jumping across unsupported Nomad versions; disk errors.","solutions":["Read the wrapped %v error: if it's a decode failure, the registry blob is corrupt and should be dropped (tasks re-register plugins at restart)","Back up and clean/restore the client state file, then restart nomad so plugins re-discover","Avoid skipping version ranges; migrate through the documented upgrade path","Report upstream with the wrapped error if it reproduces on healthy state"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Go: pre-check the registry key decodes before migration\nvar probe RegistryState12\nif err := dynamicBkt.Get(registryStateKey, &probe); err != nil && !boltdd.IsErrNotFound(err) {\n    return fmt.Errorf(\"pre-flight: dynamic plugin registry unreadable: %w\", err)\n}","typeGuard":"// Go: distinguish benign absence from real failure\nif err := dynamicBkt.Get(registryStateKey, oldState); err != nil {\n    if boltdd.IsErrNotFound(err) {\n        return nil // fresh state, nothing to upgrade\n    }\n    return fmt.Errorf(\"failed to read dynamic plugin registry state: %v\", err)\n}","tryCatchPattern":"// Go\nif err := UpgradeDynamicPluginRegistry(logger, tx); err != nil {\n    logger.Error(\"dynamic plugin registry migration failed; plugins will re-register after state reset\", \"err\", err)\n    return err\n}","preventionTips":["Back up client state before upgrading","Do not skip Nomad versions when upgrading clients","Watch for disk errors on the node hosting data_dir","If only plugin state is corrupt, clearing it is safe — plugins re-register at startup"],"tags":["nomad","boltdb","dynamic-plugins","state-migration"],"backgroundTag":"state-read-decode-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}