{"record":{"id":"8b00d7ce9cf467e0","repo":"hashicorp/nomad","slug":"failed-to-create-volume-watcher-v","errorCode":null,"errorMessage":"failed to create volume watcher: %v","messagePattern":"failed to create volume watcher: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"nomad/server.go","lineNumber":524,"sourceCode":"\t\treturn nil, fmt.Errorf(\"Failed to start workers: %v\", err)\n\t}\n\n\t// Setup the Consul syncer\n\tif err := s.setupConsulSyncer(); err != nil {\n\t\ts.logger.Error(\"failed to create server consul syncer\", \"error\", err)\n\t\treturn nil, fmt.Errorf(\"failed to create server Consul syncer: %v\", err)\n\t}\n\n\t// Setup the deployment watcher.\n\tif err := s.setupDeploymentWatcher(); err != nil {\n\t\ts.logger.Error(\"failed to create deployment watcher\", \"error\", err)\n\t\treturn nil, fmt.Errorf(\"failed to create deployment watcher: %v\", err)\n\t}\n\n\t// Setup the volume watcher\n\tif err := s.setupVolumeWatcher(); err != nil {\n\t\ts.logger.Error(\"failed to create volume watcher\", \"error\", err)\n\t\treturn nil, fmt.Errorf(\"failed to create volume watcher: %v\", err)\n\t}\n\ts.volumeControllerFutures = map[string]context.Context{}\n\n\t// Start the eval broker notification system so any subscribers can get\n\t// updates when the processes SetEnabled is triggered.\n\tgo s.evalBroker.enabledNotifier.Run()\n\n\t// Setup the node drainer.\n\ts.setupNodeDrainer()\n\n\t// Setup the enterprise state\n\tif err := s.setupEnterprise(config); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Monitor leadership changes\n\tgo s.monitorLeadership()\n","sourceCodeStart":506,"sourceCodeEnd":542,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/server.go#L506-L542","documentation":"NewServer wraps any error from setupVolumeWatcher, which builds the volume controller/watcher used to reconcile CSI volume claims. A failure here aborts server creation; the error text is just the inner cause wrapped.","triggerScenarios":"NewServer invoked when setupVolumeWatcher cannot initialize its internal state — typically the CSI volume watcher's eval/notify queue cannot be created or a config-dependent dependency (broker, blockResult) fails.","commonSituations":"Environments with CSI plugins misconfigured, servers under severe resource pressure during startup, or versions where the CSI controller was partially disabled via feature flags/build tags.","solutions":["Inspect the 'failed to create volume watcher' log entry immediately above for the root cause.","Verify CSI plugin configuration and server config validity.","Ensure the server host has adequate resources and restart the agent.","If CSI is not used and failure persists, report/upgrade — this path should never fail with valid config."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"srv, err := nomad.NewServer(cfg, logger)\nif err != nil && strings.Contains(err.Error(), \"failed to create volume watcher\") {\n    logger.Error(\"volume watcher init failed\", \"err\", err)\n    return err\n}","preventionTips":["Keep server hosts adequately resourced.","Verify CSI-related config on clusters using CSI.","Restart the agent promptly on this fatal startup error."],"tags":["startup","csi","volume-watcher"],"backgroundTag":"server-startup-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T15:18:49.778Z"}