kopia/kopia · error

error writing blob

Error message

error writing blob

What it means

Wrapper in the concurrent PutBlob worker of the provider validation stress phase: PutBlob failed while writing a generated blob to the storage under test; the write worker returns and the validation reports the backend as unreliable under concurrency.

Solutions

1. Read the wrapped cause — if it is a rate-limit error, reduce concurrency or add retries with backoff in the provider.
2. Verify credentials/quota are valid for the whole ConcurrencyTestDuration.
3. Ensure the provider implementation is goroutine-safe.
4. Enable provider-level retry for transient HTTP errors.

Example fix

// before
if err := c.st.pickOne().PutBlob(ctx, id, gather.FromSlice(data), blob.PutOptions{}); err != nil {
    return errors.Wrap(err, "error writing blob")
}
// after
err := retry(3, func() error {
    return c.st.pickOne().PutBlob(ctx, id, gather.FromSlice(data), blob.PutOptions{})
})
if err != nil {
    return errors.Wrap(err, "error writing blob")
}

if err := ctx.Err(); err != nil {
    return err // fail fast if the test context is already cancelled
}

err := st.PutBlob(ctx, id, data, blob.PutOptions{})
if err != nil {
    if isTransient(err) { err = retryWithBackoff(3, put) }
    if err != nil { return fmt.Errorf("error writing blob: %w", err) }
}

When it happens

Trigger: st.pickOne().PutBlob(ctx, id, gather.FromSlice(data), blob.PutOptions{}) returns a non-nil error during the concurrent write phase.

Common situations: Rate limiting (HTTP 429/503) from cloud storage under parallel load; expired credentials mid-test; adapter not safe for concurrent use.

Related errors

• deleting
• error deleting blobs
• error listing uncompacted epoch
• error validating concurrency
• invalid data read for

AI-assisted analysis of kopia/kopia@82495e54b5 (2026-09-07). Data as JSON: /api/errors/927f4646fa490d7d. Report an issue: GitHub.

Appendix: source

Thrown at internal/providervalidation/providervalidation.go:337

		for clock.Now().Before(c.deadline) {
			seed := rand.Int63() //nolint:gosec
			data := c.dataFromSeed(seed, data0)

			id := c.prefix + blob.ID(hex.EncodeToString(data[0:16]))

			c.mu.Lock()
			c.blobSeeds[id] = seed
			c.blobIDs = append(c.blobIDs, id)
			c.mu.Unlock()

			// sleep for a short time so that readers can start getting the blob when it's still
			// not written.
			c.randomSleep()

			log(ctx).Debugf("PutBlob worker %v writing %v (%v bytes)", worker, id, len(data))

			if err := c.st.pickOne().PutBlob(ctx, id, gather.FromSlice(data), blob.PutOptions{}); err != nil {
				return errors.Wrap(err, "error writing blob")
			}

			c.mu.Lock()
			c.blobWritten[id] = true
			c.mu.Unlock()

			log(ctx).Debugf("PutBlob worker %v finished writing %v (%v bytes)", worker, id, len(data))
		}

		return nil
	}
}

func (c *concurrencyTest) randomSleep() {
	time.Sleep(time.Duration(rand.Intn(int(500 * time.Millisecond)))) //nolint:gosec,mnd
}

func (c *concurrencyTest) pickBlob() (blob.ID, int64, bool) {

View on GitHub (pinned to 82495e54b5)