hashicorp/terraform · critical

failed to upload state

Error message

failed to upload state: %w

What it means

Thrown by the S3 remote-state backend when the AWS multipart uploader fails to write the Terraform state object to the configured bucket (PutObject/Upload). The wrapped error is the raw AWS SDK v2 response, so the underlying cause (NoSuchBucket, AccessDenied, KMSNotFound, etc.) is preserved in the %w chain. This is the primary persistence path: if it fails, the new state is not saved remotely.

Solutions

1. Read the wrapped AWS error: run `terraform apply` again with TF_LOG=DEBUG and look for the ErrorCode/HTTP status in the chain (e.g. NoSuchBucket, AccessDenied, KMSNotFoundException) to pinpoint the cause.
2. Verify the bucket: `aws s3api head-bucket --bucket <bucket>` from the same environment/credentials Terraform uses; confirm it lives in the configured region.
3. Verify IAM permissions: ensure the effective identity has s3:PutObject on arn:aws:s3:::<bucket>/<key> (and kms:GenerateDataKey if SSE-KMS). Use the IAM policy simulator against the state key ARN.
4. Reconcile encryption settings: if backend config sets kms_key_id, confirm the key is enabled and the principal can use it; if using SSE-C, confirm the customer key is stable across runs.
5. For transient/network errors, simply retry the apply; multipart uploads are idempotent at the object key level.

Example fix

# before (wrong region / missing kms)
backend "s3" {
  bucket = "tf-state-prod"
  key    = "prod/terraform.tfstate"
  region = "us-east-1"
}
# after
terraform {
  backend "s3" {
    bucket     = "tf-state-prod"
    key        = "prod/terraform.tfstate"
    region     = "us-west-2"        # match the bucket's actual region
    kms_key_id = "arn:aws:kms:us-west-2:111122223333:key/abcd-1234"
  }
}

// Pre-flight: confirm the principal can write the state key before applying.
import (
  "context"
  "github.com/aws/aws-sdk-go-v2/service/s3"
)

func canPutState(ctx context.Context, c *s3.Client, bucket, key string) error {
  // cheap reachability + bucket-existence check
  if _, err := c.HeadBucket(ctx, &s3.HeadBucketInput{Bucket: &bucket}); err != nil {
    return fmt.Errorf("bucket %s not reachable: %w", bucket, err)
  }
  return nil
}
// Call before terraform apply; surface HeadBucket failures to the operator early.

// Wrap Put in a short retry for transient transport errors, surface AWS error codes.
var lastErr error
for i := 0; i < 3; i++ {
  if _, err := uploader.Upload(ctx, input); err == nil {
    return nil
  } else {
    lastErr = err
    var apiErr smithy.APIError
    if errors.As(err, &apiErr) {
      switch apiErr.ErrorCode() {
      case "NoSuchBucket", "AccessDenied", "KMSNotFoundException":
        return fmt.Errorf("failed to upload state: %w", err) // non-retryable
      }
    }
    time.Sleep(backoff(i))
  }
}
return fmt.Errorf("failed to upload state: %w", lastErr)

When it happens

Trigger: The S3 manager.Uploader.Upload call at client.go:238 returns a non-nil error. Concrete triggers: bucket does not exist in the configured region, IAM principal lacks s3:PutObject on the state key, KMS key id is wrong/disabled/uses a different account, SSE-C customer key MD5 mismatches the stored object metadata, a bucket policy enforces a specific ACL/checksum the request violates, object-lock retention blocks the overwrite, or a network/transport error mid-multipart-upload.

Common situations: Freshly referenced bucket name typo, AWS credentials expired or scoped to the wrong account, region mismatch between the AWS client and the bucket, switched SSE configuration (KMS vs SSE-C vs AES256) without updating backend config, bucket versioning/object-lock enabled with deny-overwrite policy, running from CI with short-lived role credentials that expired mid-apply.

Related errors

• unable to retrieve file from S3 bucket
• failed to access object
• failed to delete the lock file
• failed to read the body of the S3 object
• failed to store state MD5

AI-assisted analysis of hashicorp/terraform@d32a084675 (2026-08-11). Data as JSON: /api/errors/97df555c594cbdb6. Report an issue: GitHub.

Appendix: source

Thrown at internal/backend/remote-state/s3/client.go:240

			input.SSECustomerAlgorithm = aws.String(string(s3EncryptionAlgorithm))
			input.SSECustomerKeyMD5 = aws.String(c.getSSECustomerKeyMD5())
		} else {
			input.ServerSideEncryption = s3EncryptionAlgorithm
		}
	}

	if c.acl != "" {
		input.ACL = s3types.ObjectCannedACL(c.acl)
	}

	log.Info("Uploading remote state")

	uploader := manager.NewUploader(c.s3Client, func(u *manager.Uploader) {
		u.ClientOptions = optFns
	})
	_, err := uploader.Upload(ctx, input)
	if err != nil {
		return fmt.Errorf("failed to upload state: %w", err)
	}

	if err := c.putMD5(ctx, sum[:]); err != nil {
		// if this errors out, we unfortunately have to error out altogether,
		// since the next Get will inevitably fail.
		return fmt.Errorf("failed to store state MD5: %w", err)
	}

	return nil
}

func (c *RemoteClient) Delete() tfdiags.Diagnostics {
	var diags tfdiags.Diagnostics
	ctx := context.TODO()
	log := c.logger(operationClientDelete)

	ctx, baselog := baselogging.NewHcLogger(ctx, log)
	ctx = baselogging.RegisterLogger(ctx, baselog)

View on GitHub (pinned to d32a084675)