hashicorp/terraform · error

Failed to upload state to %v: %v

Error message

Failed to upload state to %v: %v

What it means

remoteClient.Put() opens a writer, writes data, and closes; any failure in Write or Close is wrapped here with the gs:// destination URL. Close is where GCS commits the upload, so most real failures (auth expired mid-flight, quota, precondition) surface on Close.

Source

Thrown at internal/backend/remote-state/gcs/client.go:77

	return result, diags
}

func (c *remoteClient) Put(data []byte) tfdiags.Diagnostics {
	var diags tfdiags.Diagnostics
	ctx := context.TODO()
	err := func() error {
		stateFileWriter := c.stateFile().NewWriter(ctx)
		if len(c.kmsKeyName) > 0 {
			stateFileWriter.KMSKeyName = c.kmsKeyName
		}
		if _, err := stateFileWriter.Write(data); err != nil {
			return err
		}
		return stateFileWriter.Close()
	}()
	if err != nil {
		return diags.Append(fmt.Errorf("Failed to upload state to %v: %v", c.stateFileURL(), err))
	}

	return diags
}

func (c *remoteClient) Delete() tfdiags.Diagnostics {
	var diags tfdiags.Diagnostics

	ctx := context.TODO()
	if err := c.stateFile().Delete(ctx); err != nil {
		return diags.Append(fmt.Errorf("Failed to delete state file %v: %v", c.stateFileURL(), err))
	}

	return diags
}

// Lock writes to a lock file, ensuring file creation. Returns the generation
// number, which must be passed to Unlock().

View on GitHub (pinned to c9def3e214)

Solutions

1. Inspect %v: 401 → refresh credentials; 403 → IAM/quota; 429 → throttle/retry; KMS error → enable key.
2. Grant SA 'roles/storage.objectAdmin' and (if KMS) 'roles/cloudkms.cryptoKeyEncrypter' on the key.
3. For token-expiry, ensure ADC/workload-identity refreshes automatically and avoid hours-long single applies; split or use state push separately.
4. If concurrent writers, enable and rely on state locking (it uses a separate .tflock) — confirm the lock is being honored.

Example fix

// before: kms key disabled
kms_encryption_key = "projects/p/locations/global/keyRings/kr/cryptoKeys/disabled-key"

// after
gcloud kms keys enable disabled-key --location global --keyring kr
terraform apply

// Confirm SA can write before apply
ctx := context.Background()
w := client.Bucket(bucket).Object(prefix+"/probe").If(storage.Conditions{DoesNotExist: true}).NewWriter(ctx)
if _, err := w.Write([]byte("probe")); err != nil { return err }
return w.Close()

diags := client.Put(data)
if diags.HasErrors() {
    if isAuthExpired(diags) { refreshCredentials(); /* retry once */ }
    if isQuota(diags) { return fmt.Errorf("GCS write quota exceeded, retry later") }
}

When it happens

Trigger: 'terraform apply' / 'state push' persisting state: the access token expired between plan and apply; CSEK provided for read but not write; bucket is full (quota), in a region under maintenance; KMS key disabled; precondition (ifGenerationMatch) failed due to concurrent writer.

Common situations: Long-running apply where the OAuth token lifetime (<1h) elapses; SA lacks storage.objects.create; kms_encryption_key references a disabled/destroyed key; bucket retention policy conflict.

Related errors

• storage.NewClient() failed: %v
• Failed to read state file from %v: %v
• Failed to read state file attrs from %v: %v
• querying Cloud Storage failed: %v
• Failed to open state file at %v: %v

AI-assisted analysis of hashicorp/terraform@c9def3e214 (2026-08-07). Data as JSON: /api/errors/a7125eb95ec26bf6. Report an issue: GitHub.