hashicorp/nomad · error

alloc %q is not part of deployment %q

Error message

alloc %q is not part of deployment %q

What it means

Thrown in setAllocHealth when the allocation exists but its Allocation.DeploymentID does not match the DeploymentID of the request. The deployment-health endpoint only allows updating allocs that belong to the referenced deployment, preventing cross-deployment health contamination. The entire Raft transaction aborts with no changes applied.

Source

Thrown at nomad/state/state_store.go:5167

		return fmt.Errorf("Deployment ID %q couldn't be updated as it does not exist", req.DeploymentID)
	} else if !deployment.Active() {
		return fmt.Errorf("Deployment %q has terminal status %q:", deployment.ID, deployment.Status)
	}

	// Update the health status of each allocation
	if total := len(req.HealthyAllocationIDs) + len(req.UnhealthyAllocationIDs); total != 0 {
		setAllocHealth := func(id string, healthy bool, ts time.Time) error {
			existing, err := txn.First("allocs", "id", id)
			if err != nil {
				return fmt.Errorf("alloc %q lookup failed: %v", id, err)
			}
			if existing == nil {
				return fmt.Errorf("unknown alloc %q", id)
			}

			old := existing.(*structs.Allocation)
			if old.DeploymentID != req.DeploymentID {
				return fmt.Errorf("alloc %q is not part of deployment %q", id, req.DeploymentID)
			}

			// Set the health
			copy := old.Copy()
			if copy.DeploymentStatus == nil {
				copy.DeploymentStatus = &structs.AllocDeploymentStatus{}
			}
			copy.DeploymentStatus.Healthy = new(healthy)
			copy.DeploymentStatus.Timestamp = ts
			copy.DeploymentStatus.ModifyIndex = index
			copy.ModifyTime = req.Timestamp.UnixNano()
			copy.ModifyIndex = index

			if err := s.updateDeploymentWithAlloc(index, copy, old, txn); err != nil {
				return fmt.Errorf("error updating deployment: %v", err)
			}

			if err := txn.Insert("allocs", copy); err != nil {

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Re-resolve allocs via the deployment itself (allocs where DeploymentID == req.DeploymentID) instead of job-wide alloc lists
2. Filter the alloc list before submitting: drop any alloc whose DeploymentID differs from the target deployment
3. After a job resubmit, fetch the fresh deployment ID (latest deployment for the job) and rebuild the health request
4. Check for races between evaluation completion and health submission; re-read state at submission time

Example fix

// before
for _, a := range allocsForJob(jobID) { ids = append(ids, a.ID) }
// after
for _, a := range allocsForJob(jobID) {
	if a.DeploymentID == depID { // only allocs of THIS deployment
		ids = append(ids, a.ID)
	}
}

func allocsOfDeployment(allocs []*structs.Allocation, depID string) []string {
	var out []string
	for _, a := range allocs {
		if a.DeploymentID == depID { out = append(out, a.ID) }
	}
	return out
}

func belongsToDeployment(a *structs.Allocation, depID string) bool { return a != nil && a.DeploymentID == depID }

if err := store.UpsertDeploymentAllocHealth(idx, req); err != nil {
	if strings.Contains(err.Error(), "is not part of deployment") {
		// rebuild request from allocs where DeploymentID == req.DeploymentID
	}
	return err
}

When it happens

Trigger: A DeploymentAllocHealthRequest lists an alloc ID that belongs to a different (typically older) deployment of the same job — e.g. after a resubmit, canary failover, or job update that created a new deployment while the caller still sends alloc IDs gathered from the old one.

Common situations: Job stopped and resubmitted: new deployment ID but tooling caches old alloc list; monitoring scripts polling allocs of the job rather than of the specific deployment; canary promotion flows mixing allocs across deployments.

Related errors

• deployment promotion cannot be undone
• deployment id not found: %q
• deployment %q references unknown job %q
• unknown deployment %q
• failed to retrieve latest deployment: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/37a7809b6d757e2a. Report an issue: GitHub.