What causes "recovered from %s" in juicedata/juicefs?

Look at the wrapped panic value after 'recovered from' — it identifies the actual panicking component.

recovered from %s

Error message

recovered from %s

What it means

cachedStore.load wraps its body in a recover() and converts any panic (e.g. a runtime panic in decompression, page handling, or a storage client) into a regular error "recovered from <panic value>". This keeps a panicking download path from crashing the process; callers like writeFiles/readFiles (and the bench command at cmd/bench.go) see it as an ordinary error return.

Source

Thrown at pkg/chunk/cached_store.go:754

	if errors.Is(err, context.Canceled) || errors.Is(err, utils.ErrFuncTimeout) {
		return 0, err
	}
	store.objectDataBytes.WithLabelValues("GET", res.sc).Add(float64(res.n))
	store.objectReqsHistogram.WithLabelValues("GET", res.sc).Observe(used.Seconds())
	if err == nil {
		store.fetcher.fetch(key)
		return res.n, nil
	}
	store.objectReqErrors.Add(1)
	// fall back to full read
	return 0, errTryFullRead
}

func (store *cachedStore) load(ctx context.Context, key string, page *Page, cache bool, forceCache bool) (err error) {
	defer func() {
		e := recover()
		if e != nil {
			err = fmt.Errorf("recovered from %s", e)
		}
	}()
	store.currentDownload <- struct{}{}
	defer func() { <-store.currentDownload }()
	needed := store.compressor.CompressBound(len(page.Data))
	compressed := needed > len(page.Data)
	// we don't know the actual size for compressed block
	if store.downLimit != nil && !compressed {
		store.downLimit.Wait(int64(len(page.Data)))
	}
	var (
		p     *Page
		start = time.Now()
	)
	if compressed {
		c := NewOffPage(needed)
		defer c.Release()
		p = c

View on GitHub (pinned to c9a67b23e8)

Solutions

Look at the wrapped panic value after 'recovered from' — it identifies the actual panicking component.
Delete the affected object from the object store / clear the local cache and re-read to rule out corrupted data.
Check compressor settings in the volume format (e.g. zstd) — try without compression to isolate decompressor panics.
Capture a goroutine dump and report the stack to juicefs if the panic is reproducible (library bug, not caller error).
Pin/upgrade to a fixed juicefs version if the panic matches a known issue.

Example fix

// before: corrupted cached object causes panic in load
//   err = store.load(ctx, key, page, true, false)
// after: clear cache and retry read once
if err != nil && strings.HasPrefix(err.Error(), "recovered from") {
    store.bcache.remove(key)
    err = store.load(ctx, key, page, true, false)
}

Defensive patterns

Strategy: try-catch

Try / catch

if err != nil {
    if strings.HasPrefix(err.Error(), "recovered from") {
        logger.Errorf("internal panic during load: %v", err)
        // invalidate cache/object and retry once, else fail over
    }
    return err
}

Prevention

Keep the client version consistent with the volume format (compressor/codec) to avoid panics on unexpected data.
Clear local cache and re-read when encountering a panic-derived error; corrupted cached objects are a common trigger.
Report reproducible panics upstream with a goroutine dump — this error indicates a library bug, not caller misuse.
Run `juicefs bench`/read workloads on a staging mount before production to surface panic paths early.

When it happens

Trigger: Any panic inside cachedStore.load during a block download — typically triggered via readFiles/writeFiles paths (including `juicefs bench` reading/writing temp files through the store): nil deref or slice bound issues in page handling, a compressor bug, or a panicking object-storage client.

Common situations: See trigger scenarios.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/36bedafc36e442da. Report an issue: GitHub.