{"record":{"id":"6e37224e35b0216d","repo":"hashicorp/nomad","slug":"namespace-q-has-non-terminal-jobs-in-regions-v","errorCode":null,"errorMessage":"namespace %q has non-terminal jobs in regions: %v","messagePattern":"namespace %q has non-terminal jobs in regions: (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/namespace_endpoint.go","lineNumber":207,"sourceCode":"\n\tfor _, region := range regions {\n\t\tif region == thisRegion {\n\t\t\tcontinue\n\t\t}\n\n\t\tif remoteCheckFunc != nil {\n\t\t\tremoteTerminal, err := remoteCheckFunc(authToken, namespace, region)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif !remoteTerminal {\n\t\t\t\tterminal = append(terminal, region)\n\t\t\t}\n\t\t}\n\t}\n\n\tif len(terminal) != 0 {\n\t\treturn fmt.Errorf(errorMsg, namespace, terminal)\n\t}\n\n\treturn nil\n}\n\n// namespaceTerminalJobsLocally returns true if the namespace contains only\n// terminal jobs in the local region.\nfunc (n *Namespace) namespaceTerminalJobsLocally(namespace string, snap *state.StateSnapshot) (bool, error) {\n\titer, err := snap.JobsByNamespace(nil, namespace, state.SortDefault)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\tfor {\n\t\traw := iter.Next()\n\t\tif raw == nil {\n\t\t\tbreak\n\t\t}\n","sourceCodeStart":189,"sourceCodeEnd":225,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/namespace_endpoint.go#L189-L225","documentation":"Before deleting a namespace, Nomad checks every region for non-terminal (pending/running) jobs and allocations tied to that namespace via nonTerminalObjectsInNS. If any region still reports non-terminal objects, the delete is aborted with this error naming the offending regions. This prevents removing a namespace that still has live workloads.","triggerScenarios":"Calling Namespace.Delete while jobs in the namespace are pending/running in one or more regions; orphaned non-terminal allocations on dead nodes; leftover workloads after a failed cleanup.","commonSituations":"Deleting namespaces during cluster teardown while a job is still running; multi-region deployments where a remote region still runs workloads; stuck allocations from a down node that never reached terminal state.","solutions":["Stop or purge all jobs in the namespace in every listed region (nomad job stop -purge).","Verify allocations are terminal (nomad status <job>); rebalance or clean orphaned allocs on dead nodes.","Retry the namespace delete once all regions report only terminal objects.","Check other regions explicitly if running a multi-region/federated cluster."],"exampleFix":"// before\nclient.Namespaces().Delete(&api.NamespaceDeleteRequest{Namespaces: []string{\"prod\"}}) // fails: jobs still running\n// after\nfor _, job := range jobsInNamespace(\"prod\") {\n    client.Jobs().Deregister(job.ID, true, &api.WriteOptions{Namespace: \"prod\"}) // purge\n}\nclient.Namespaces().Delete(&api.NamespaceDeleteRequest{Namespaces: []string{\"prod\"}})","handlingStrategy":"validation","validationCode":"jobs, _, _ := client.Jobs().List(&api.QueryOptions{Namespace: ns})\nfor _, j := range jobs {\n    if j.Status != \"dead\" { return fmt.Errorf(\"job %s still %s\", j.ID, j.Status) }\n}\n_, err := client.Namespaces().Delete(&api.NamespaceDeleteRequest{Namespaces: []string{ns}})","typeGuard":null,"tryCatchPattern":"_, err := client.Namespaces().Delete(req)\nif err != nil && strings.Contains(err.Error(), \"non-terminal\") {\n    // purge remaining jobs in each listed region, then retry once\n}","preventionTips":["Purge (not just stop) all jobs before deleting a namespace.","Check jobs in every region, not just the local one.","Watch for orphaned allocs on down nodes and clean them up.","Automate: stop -> wait for terminal -> delete namespace."],"tags":["nomad","namespace-delete","precondition-failed"],"backgroundTag":"resource-in-use","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}