What causes "upload %v to target bucket" in thanos-io/thanos?

Verify the target bucket exists and the credentials have write permission

upload to target bucket

Error message

upload %v to target bucket

What it means

In ensureObjectReplicated, the objstore Upload of an object (block file) from the origin bucket to the target bucket failed after confirming the object was missing on the target — network or object-store error — so block replication is incomplete and the object must be retried.

Solutions

Verify the target bucket exists and the credentials have write permission
Retry replication — ensureObjectReplicated is idempotent (skips objects already present)
Check target storage quota and rate limits
Confirm network connectivity to the target endpoint

Defensive patterns

Strategy: retry

Validate before calling

// verify target bucket is writable before starting replication
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
probe := "__replication_probe__"
err := toBkt.Upload(ctx, probe, strings.NewReader("ok"))
_ = toBkt.Delete(ctx, probe)
cancel()
if err != nil { return fmt.Errorf("target bucket not writable: %w", err) }

Try / catch

if err := rs.toBkt.Upload(ctx, objectName, r); err != nil {
    if errors.Is(err, context.DeadlineExceeded) {
        // large object: retry with backoff
    }
    return errors.Wrapf(err, "upload %v to target bucket", objectName)
}

Prevention

Pre-create the target bucket and grant write IAM permissions
Set generous timeouts for large-object uploads
Monitor target bucket quota and rate-limit metrics

When it happens

Trigger: toBkt.Upload returns an error: target bucket does not exist, write permission denied, quota exceeded, network interruption mid-stream, or context canceled during the transfer.

Common situations: Target bucket misconfigured or not pre-created; IAM role on target lacks putObject; storage quota/billing limits hit; transient S3/GCS 5xx or throttling during large object copy.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/6a9a5f262a9a9386. Report an issue: GitHub.

Appendix: source

Thrown at pkg/replicate/scheme.go:304

	}

	// skip if already exists.
	if exists {
		level.Debug(rs.logger).Log("msg", "skipping object as already replicated", "object", objectName)
		return nil
	}

	level.Debug(rs.logger).Log("msg", "object not present in target bucket, replicating", "object", objectName)

	r, err := rs.fromBkt.Get(ctx, objectName)
	if err != nil {
		return errors.Wrapf(err, "get %v from origin bucket", objectName)
	}

	defer r.Close()

	if err = rs.toBkt.Upload(ctx, objectName, r); err != nil {
		return errors.Wrapf(err, "upload %v to target bucket", objectName)
	}

	level.Info(rs.logger).Log("msg", "object replicated", "object", objectName)
	rs.metrics.objectsReplicated.Inc()

	return nil
}

View on GitHub (pinned to 35b8b99117)