{"record":{"id":"63992ca630b9d134","repo":"hashicorp/nomad","slug":"failed-to-create-deployment-watcher-v","errorCode":null,"errorMessage":"failed to create deployment watcher: %v","messagePattern":"failed to create deployment watcher: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"nomad/server.go","lineNumber":518,"sourceCode":"\t}\n\n\t// Initialize the scheduling workers\n\tif err := s.setupWorkers(s.shutdownCtx); err != nil {\n\t\ts.Shutdown()\n\t\ts.logger.Error(\"failed to start workers\", \"error\", err)\n\t\treturn nil, fmt.Errorf(\"Failed to start workers: %v\", err)\n\t}\n\n\t// Setup the Consul syncer\n\tif err := s.setupConsulSyncer(); err != nil {\n\t\ts.logger.Error(\"failed to create server consul syncer\", \"error\", err)\n\t\treturn nil, fmt.Errorf(\"failed to create server Consul syncer: %v\", err)\n\t}\n\n\t// Setup the deployment watcher.\n\tif err := s.setupDeploymentWatcher(); err != nil {\n\t\ts.logger.Error(\"failed to create deployment watcher\", \"error\", err)\n\t\treturn nil, fmt.Errorf(\"failed to create deployment watcher: %v\", err)\n\t}\n\n\t// Setup the volume watcher\n\tif err := s.setupVolumeWatcher(); err != nil {\n\t\ts.logger.Error(\"failed to create volume watcher\", \"error\", err)\n\t\treturn nil, fmt.Errorf(\"failed to create volume watcher: %v\", err)\n\t}\n\ts.volumeControllerFutures = map[string]context.Context{}\n\n\t// Start the eval broker notification system so any subscribers can get\n\t// updates when the processes SetEnabled is triggered.\n\tgo s.evalBroker.enabledNotifier.Run()\n\n\t// Setup the node drainer.\n\ts.setupNodeDrainer()\n\n\t// Setup the enterprise state\n\tif err := s.setupEnterprise(config); err != nil {","sourceCodeStart":500,"sourceCodeEnd":536,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/server.go#L500-L536","documentation":"NewServer wraps any error from setupDeploymentWatcher, which constructs the deployment watcher that tracks deployments and feeds them to the deployment watcher RPC endpoints. If the watcher cannot be created, the server cannot start and NewServer returns nil.","triggerScenarios":"NewServer with a config that makes setupDeploymentWatcher fail — typically the internal deployment watcher cannot be initialized (e.g. queue/eval broker setup issues or invalid config values used when constructing the watcher).","commonSituations":"Rare in practice; seen when Nomad binaries are miscompiled/partially initialized, when memory limits kill internal components during server bring-up, or when a plugin/config-dependent component the watcher relies on fails early in NewServer.","solutions":["Read the 'failed to create deployment watcher' log line above the error for the inner cause.","Confirm server configuration is valid (validate with `nomad agent -config ... -verify-only` if available).","Check host memory/CPU limits; retry server startup after freeing resources.","Upgrade/reinstall Nomad if the failure persists with known-good config."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"srv, err := nomad.NewServer(cfg, logger)\nif err != nil && strings.Contains(err.Error(), \"failed to create deployment watcher\") {\n    logger.Error(\"deployment watcher init failed; check resources and config\", \"err\", err)\n    return err\n}","preventionTips":["Validate server config before launch.","Ensure adequate memory on server nodes.","Alert on any NewServer failure since it is fatal."],"tags":["startup","deployment-watcher","server"],"backgroundTag":"server-startup-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}