juicedata/juicefs · error

(max tries) upload block %s: %s (after %d tries)

Error message

(max tries) upload block %s: %s (after %d tries)

What it means

cached_store's block uploader retries `store.put` (upload of a chunk block to object storage) up to `max` times; if every attempt fails, it wraps the last underlying error as "(max tries) upload block <key>: <cause> (after <N> tries)". It signals that a block could not be persisted to the object store despite internal retries, so the write path (uploadStagingFile -> upload) aborts and the pending slice is left un-uploaded.

Source

Thrown at pkg/chunk/cached_store.go:383

	buf.Data = buf.Data[:n]

	try, max := 0, 3
	if sync {
		max = store.conf.MaxRetries + 1
	}
	for ; try < max; try++ {
		time.Sleep(time.Second * time.Duration(try*try))
		if s != nil && s.uploadError != nil {
			err = fmt.Errorf("(cancelled) upload block %s: %s (after %d tries)", key, err, try)
			break
		}
		if err = store.put(ctx, key, buf); err == nil {
			break
		}
		logger.Debugf("Upload %s: %s (try %d)", key, err, try+1)
	}
	if err != nil && try >= max {
		err = fmt.Errorf("(max tries) upload block %s: %s (after %d tries)", key, err, try)
	}
	return err
}

func (s *wSlice) upload(indx int) {
	blen := s.blockSize(indx)
	key := s.key(indx)
	pages := s.pages[indx]
	s.pages[indx] = nil
	s.pendings++

	go func() {
		var block *Page
		var off int
		if len(pages) == 1 {
			block = pages[0]
			off = len(block.Data)
		} else {

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Check connectivity and credentials to the object storage endpoint (e.g. `juicefs status <meta-url>` or aws cli against the same bucket).
2. Inspect the wrapped cause after 'upload block <key>:' — it contains the actual storage error (403, timeout, DNS, etc.) and fix that root cause.
3. Increase upload retry tolerance by lowering upload pressure (reduce `--upload-limit`, `--max-uploads`) or fixing throttling on the provider side.
4. Verify bucket name/region/endpoint configuration and that the bucket exists and is writable.
5. If failures persist transiently, remount or restart the client so the pending slice is re-uploaded after the storage recovers.

Example fix

// before: storage endpoint misconfigured
//   juicefs mount --storage-url error ...
// after: verify and correct storage config before mounting
//   juicefs format --storage s3 --bucket https://s3.us-east-1.amazonaws.com/mybucket ...
//   juicefs mount redis://localhost/1 /mnt/jfs

if err := storeHealthCheck(ctx); err != nil {
    return fmt.Errorf("object storage unreachable, deferring upload: %w", err)
}

if err != nil && strings.Contains(err.Error(), "(max tries) upload block") {
    // storage persistently unavailable; schedule retry with backoff
    time.Sleep(backoff)
    return retryUpload(key, buf)
}

When it happens

Trigger: Calling the write path (wSlice upload via uploadStagingFile) while every `store.put` attempt to object storage fails: storage endpoint unreachable, expired/invalid credentials, bucket missing, per-request throttling, or a client whose retry budget (max tries) is exhausted within one call.

Common situations: Object storage outage or network partition during heavy writes; misconfigured S3 credentials or wrong endpoint in the mount config; rate limiting from the provider under parallel upload load; DNS or proxy problems in containerized/Kubernetes deployments.

Related errors

• Failed to create bucket %s: %s, previous error: %s
• get %s: %s
• ceph: can't put empty file
• Failed to put: %s
• Failed to create bucket %s: %s, previous error: %s Please cr

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/a7d34dac2c52d814. Report an issue: GitHub.