temporalio/temporal · warning

worker deployment deleted

Error message

worker deployment deleted

What it means

The Worker Deployment workflow registers a DescribeDeployment query handler; if the deployment has been marked deleted (d.deleteDeployment set), the handler returns errDeploymentDeleted instead of state. This tells callers the deployment workflow is draining/destroyed and no longer serves its state.

Source

Thrown at service/worker/workerdeployment/workflow.go:319

		d.metrics.Counter(metrics.WorkerDeploymentCreated.Name()).Inc(1)
	}
	if d.State.Versions == nil {
		d.State.Versions = make(map[string]*deploymentspb.WorkerDeploymentVersionSummary)
	}

	// TODO(carlydf): remove verbose logging
	d.logger.Info("Starting workflow run",
		"create_time", d.State.GetCreateTime(),
		"routing_config", d.State.GetRoutingConfig(),
		//nolint:staticcheck // SA1019: worker versioning v0.31
		"current_version", d.State.GetRoutingConfig().GetCurrentVersion(),
		//nolint:staticcheck // SA1019: worker versioning v0.31
		"ramping_version", d.State.GetRoutingConfig().GetRampingVersion())

	err := workflow.SetQueryHandler(ctx, QueryDescribeDeployment, func() (*deploymentspb.QueryDescribeWorkerDeploymentResponse, error) {
		if d.deleteDeployment {
			return nil, errors.New(errDeploymentDeleted)
		}
		return &deploymentspb.QueryDescribeWorkerDeploymentResponse{
			State: d.State,
		}, nil
	})
	if err != nil {
		d.logger.Info("SetQueryHandler failed for WorkerDeployment workflow with error: " + err.Error())
		return err
	}

	err = workflow.SetQueryHandler(ctx, QueryCreateRequestID, func() (*deploymentspb.CreateRequestIDQueryResponse, error) {
		if d.deleteDeployment {
			return nil, errors.New(errDeploymentDeleted)
		}
		return &deploymentspb.CreateRequestIDQueryResponse{
			RequestId:     d.GetState().GetCreateRequestId(),
			ConflictToken: d.State.GetConflictToken(),
		}, nil

View on GitHub (pinned to bde624efd1)

Solutions

  1. Stop describing the deployment once it is deleted; treat this error as confirmation of deletion
  2. Filter out deleted deployments before describing (check deployment state via list APIs first)
  3. In polling code, break the loop when this error is returned instead of retrying
  4. If the deployment should not be deleted, audit recent DeleteWorkerDeployment calls / bad deploy automation

Example fix

// before
for _, name := range deployments {
    d, err := client.DescribeWorkerDeployment(ctx, ns, name)
    use(d)
}
// after
for _, name := range deployments {
    d, err := client.DescribeWorkerDeployment(ctx, ns, name)
    if err != nil && strings.Contains(err.Error(), "worker deployment deleted") {
        continue // deleted; workflow is shutting down
    }
    use(d)
}
Defensive patterns

Strategy: type-guard

Validate before calling

// skip deployments already known to be deleted
if deletedDeployments[name] { continue }
// or inspect list output before describing

Type guard

func isDeploymentDeletedErr(err error) bool {
    return err != nil && strings.Contains(err.Error(), "worker deployment deleted")
}

Try / catch

d, err := client.DescribeWorkerDeployment(ctx, ns, name)
if isDeploymentDeletedErr(err) {
    log.Printf("deployment %s deleted, skipping", name)
    return nil
}
if err != nil { return err }

Prevention

When it happens

Trigger: Sending the DescribeDeployment query to a deployment workflow after a delete of the deployment was processed (d.deleteDeployment == true), e.g. via DescribeWorkerDeployment while the workflow is still finishing shutdown.

Common situations: Describing a deployment right after deleting it; automation that lists deployments and immediately describes each (TOCTOU race); polling loops that keep describing a deleted deployment until the workflow record disappears.

Related errors


AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/3b944d0ece5f8cdb. Report an issue: GitHub.