{"record":{"id":"64cc98e7141528ff","repo":"hashicorp/nomad","slug":"deployment-q-has-terminal-status-q","errorCode":null,"errorMessage":"Deployment %q has terminal status %q:","messagePattern":"Deployment %q has terminal status %q:","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":4811,"sourceCode":"\t\tif err := s.nestedUpsertEval(txn, index, req.Eval); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn txn.Commit()\n}\n\n// updateDeploymentStatusImpl is used to make deployment status updates\nfunc (s *StateStore) updateDeploymentStatusImpl(index uint64, u *structs.DeploymentStatusUpdate, txn *txn) error {\n\t// Retrieve deployment\n\tws := memdb.NewWatchSet()\n\tdeployment, err := s.deploymentByIDImpl(ws, u.DeploymentID, txn)\n\tif err != nil {\n\t\treturn err\n\t} else if deployment == nil {\n\t\treturn fmt.Errorf(\"Deployment ID %q couldn't be updated as it does not exist\", u.DeploymentID)\n\t} else if !deployment.Active() {\n\t\treturn fmt.Errorf(\"Deployment %q has terminal status %q:\", deployment.ID, deployment.Status)\n\t}\n\n\t// Apply the new status\n\tcopy := deployment.Copy()\n\tcopy.Status = u.Status\n\tcopy.StatusDescription = u.StatusDescription\n\tcopy.ModifyIndex = index\n\tcopy.ModifyTime = u.UpdatedAt\n\n\t// check each TaskGroup for ProgressDeadline and reset RequireProgressBy\n\t// to ProgressDeadline if the deployment is running or paused. This is to\n\t// ensure that the RequireProgressBy is reset on a deployment that has been\n\t// paused and resumed.\n\tfor _, dState := range copy.TaskGroups {\n\t\tif dState == nil || dState.ProgressDeadline == 0 {\n\t\t\tcontinue\n\t\t}\n","sourceCodeStart":4793,"sourceCodeEnd":4829,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L4793-L4829","documentation":"Returned by UpdateDeploymentStatus when the deployment exists but has a terminal (non-active) status, so its status can no longer be changed. structs.Deployment.Active() is false for statuses like cancelled, paused-terminal, failed, or successful.","triggerScenarios":"DeploymentStatusUpdate applied against a deployment already in a terminal state (e.g. a late evaluation update arrives after the deployment was cancelled or completed).","commonSituations":"Stale evaluations processed after deployment completion; duplicate status updates; users manually cancelling a deployment while an agent still reports to it.","solutions":["Check deployment.Status before issuing an update; ignore updates to terminal deployments","Guard the caller: only send DeploymentStatusUpdate for active deployments","If an evaluation is stale, discard it instead of applying the status change","Reconcile with nomad deployment status to see the authoritative terminal state"],"exampleFix":"// before\nreq := &structs.DeploymentStatusUpdateRequest{Deployment: u}\nsrv.raft.Apply(req)\n// after\nif dep, _, _ := state.DeploymentByID(nil, u.DeploymentID); dep == nil || !dep.Active() {\n  return nil // skip stale/terminal deployment update\n}\nsrv.raft.Apply(req)","handlingStrategy":"validation","validationCode":"dep, _, err := state.DeploymentByID(nil, u.DeploymentID)\nif err != nil { return err }\nif dep != nil && !dep.Active() {\n  return nil // terminal deployment: nothing to update\n}","typeGuard":"func deploymentActive(s *state.StateStore, id string) bool {\n  d, _, err := s.DeploymentByID(nil, id)\n  return err == nil && d != nil && d.Active()\n}","tryCatchPattern":"if err := srv.raft.Apply(req).Error(); err != nil {\n  if strings.Contains(err.Error(), \"terminal status\") {\n    // stale update; discard rather than retry\n  }\n  return err\n}","preventionTips":["Check structs.Deployment.Active() before updating","Discard stale evaluations referencing finished deployments","Never retry terminal-status failures"],"tags":["nomad","state-store","deployment","invalid-state"],"backgroundTag":"deployment-terminal-state","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}