{"record":{"id":"179f76903d20c31e","repo":"hashicorp/nomad","slug":"can-t-set-health-of-allocations-for-a-terminal-dep","errorCode":null,"errorMessage":"can't set health of allocations for a terminal deployment","messagePattern":"can't set health of allocations for a terminal deployment","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"nomad/structs/errors.go","lineNumber":85,"sourceCode":"\tErrMalformedChooseParameter   = errors.New(errMalformedChooseParameter)\n\n\t// ErrResultPaginatorCreation is returned by list RPC handlers when the\n\t// result paginator cannot be built, for example when the server cannot\n\t// evaluate a requested filter expression. api.ResultPaginatorErrorContent\n\t// duplicates its message so the CLI can match it without importing structs.\n\t// Keep the two in sync.\n\tErrResultPaginatorCreation = errors.New(errResultPaginatorCreation)\n\n\tErrUnknownNode = errors.New(ErrUnknownNodePrefix)\n\n\tErrDeploymentTerminalNoCancel    = errors.New(errDeploymentTerminalNoCancel)\n\tErrDeploymentTerminalNoFail      = errors.New(errDeploymentTerminalNoFail)\n\tErrDeploymentTerminalNoPause     = errors.New(errDeploymentTerminalNoPause)\n\tErrDeploymentTerminalNoPromote   = errors.New(errDeploymentTerminalNoPromote)\n\tErrDeploymentTerminalNoResume    = errors.New(errDeploymentTerminalNoResume)\n\tErrDeploymentTerminalNoUnblock   = errors.New(errDeploymentTerminalNoUnblock)\n\tErrDeploymentTerminalNoRun       = errors.New(errDeploymentTerminalNoRun)\n\tErrDeploymentTerminalNoSetHealth = errors.New(errDeploymentTerminalNoSetHealth)\n\tErrDeploymentRunningNoUnblock    = errors.New(errDeploymentRunningNoUnblock)\n\n\tErrCSIClientRPCIgnorable  = errors.New(\"CSI client error (ignorable)\")\n\tErrCSIClientRPCRetryable  = errors.New(\"CSI client error (retryable)\")\n\tErrCSIVolumeMaxClaims     = errors.New(\"volume max claims reached\")\n\tErrCSIVolumeUnschedulable = errors.New(\"volume is currently unschedulable\")\n\tErrCSIPluginInUse         = errors.New(\"plugin in use\")\n)\n\n// IsErrNoLeader returns whether the error is due to there being no leader.\nfunc IsErrNoLeader(err error) bool {\n\treturn err != nil && strings.Contains(err.Error(), errNoLeader)\n}\n\n// IsErrNoRegionPath returns whether the error is due to there being no path to\n// the given region.\nfunc IsErrNoRegionPath(err error) bool {\n\treturn err != nil && strings.Contains(err.Error(), errNoRegionPath)","sourceCodeStart":67,"sourceCodeEnd":103,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/structs/errors.go#L67-L103","documentation":"ErrDeploymentTerminalNoSetHealth is returned when a client tries to set allocation health on a deployment that is already terminal (completed, failed, cancelled, or paused). Terminal deployments no longer accept state updates, so the server rejects the SetAllocHealth RPC rather than mutating a finished deployment. It keeps deployment state transitions strictly one-way.","triggerScenarios":"Calling the Deployment.SetAllocHealth RPC with a deployment ID whose deployment is no longer Active(), e.g. reporting alloc health after the deployment was cancelled, promoted to completion, failed, or paused.","commonSituations":"A client (nomad agent alloc status / CSI or health hooks) races with an operator cancelling the deployment; retries of an old health update arrive after the deployment finished; automation polls a stale deployment ID.","solutions":["Check deployment status before calling SetAllocHealth and skip if it is not 'running' (use the deployment's StatusIsTerminal or Active check)","Treat this error as a benign no-op: log it and stop sending health updates for that deployment ID","Refresh the deployment via the Deployment.Get RPC and reconcile local state; if terminal, update the alloc's deployment status locally without the RPC","Upgrade/patch client code to ignore ErrDeploymentTerminalNoSetHealth the same way other terminal-deployment errors are ignored"],"exampleFix":"// before\ndep, _, _ := client.Deployments().Info(depID, nil)\nclient.Deployments().SetAllocHealth(depID, healthy, unhealthy, nil)\n// after\ndep, _, _ := client.Deployments().Info(depID, nil)\nif dep != nil && dep.Active() {\n    client.Deployments().SetAllocHealth(depID, healthy, unhealthy, nil)\n}","handlingStrategy":"try-catch","validationCode":"dep, _, err := client.Deployments().Info(depID, nil)\nif err != nil || dep == nil || !dep.Active() {\n    return nil // skip health update, deployment is terminal\n}","typeGuard":"func deploymentActive(dep *api.Deployment) bool {\n    return dep != nil && dep.Active()\n}","tryCatchPattern":"err := client.Deployments().SetAllocHealth(depID, healthy, unhealthy, nil, nil)\nif err != nil && strings.Contains(err.Error(), \"terminal deployment\") {\n    log.Printf(\"deployment %s already terminal, skipping health update\", depID)\n    return nil\n}\nreturn err","preventionTips":["Always fetch deployment state and check Active() before SetAllocHealth","Ignore ErrDeploymentTerminalNoSetHealth as a benign outcome in retry loops","Refresh deployment state after any promote/fail/cancel operation before further updates"],"tags":["nomad","deployment","state-conflict"],"backgroundTag":"deployment-already-terminal","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}