{"record":{"id":"b1f6ca42b574a1da","repo":"juicedata/juicefs","slug":"chunk-pipeline-exec-err-w","errorCode":null,"errorMessage":"chunk pipeline exec err: %w","messagePattern":"chunk pipeline exec err: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/meta/redis_bak.go","lineNumber":561,"sourceCode":"\tpipe := m.rdb.Pipeline()\n\tinos := make([]uint64, 0, len(keys))\n\tidxs := make([]uint32, 0, len(keys))\n\tfor _, key := range keys {\n\t\tps := strings.Split(key, \"_\")\n\t\tif len(ps) != 2 {\n\t\t\tlogger.Warnf(\"invalid chunk key: %s\", key)\n\t\t\tcontinue\n\t\t}\n\t\tino, _ := strconv.ParseUint(ps[0][len(m.prefix)+1:], 10, 64)\n\t\tidx, _ := strconv.ParseUint(ps[1], 10, 32)\n\t\tpipe.LRange(ctx, m.chunkKey(Ino(ino), uint32(idx)), 0, -1)\n\t\tinos = append(inos, ino)\n\t\tidxs = append(idxs, uint32(idx))\n\t}\n\n\tcmds, err := pipe.Exec(ctx)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"chunk pipeline exec err: %w\", err)\n\t}\n\n\tchunks := make([]*pb.Chunk, 0, len(cmds))\n\tfor k, cmd := range cmds {\n\t\tvals, err := cmd.(*redis.StringSliceCmd).Result()\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"get chunk result err: %w\", err)\n\t\t}\n\t\tif len(vals) == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\tpc := pools[0].Get().(*pb.Chunk)\n\t\tpc.Inode = inos[k]\n\t\tpc.Index = idxs[k]\n\n\t\tpc.Slices = pools[1].Get().([]byte)\n\t\tif len(pc.Slices) < len(vals)*sliceBytes {","sourceCodeStart":543,"sourceCodeEnd":579,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/meta/redis_bak.go#L543-L579","documentation":"During `juicefs dump` of a Redis-backed metadata engine, dumpChunks batches SCAN-followup reads of chunk records into a Redis pipeline and calls pipe.Exec. This error wraps any transport/protocol failure returned by the pipeline itself (connection dropped, cluster redirection, timeout), aborting the metadata dump.","triggerScenarios":"Calling dump (or dumpChunks) against Redis when the connection fails mid-pipeline: server closed the connection, MOVED/ASK cluster redirection issues, read timeout, or Redis restarted during the dump.","commonSituations":"Dumping a large volume so the pipeline takes long enough for Redis to close an idle connection or for a proxy/LB (e.g. cloud-managed Redis) to time it out; pointing at a ClusterClient with a misconfigured node; network flakiness between client and Redis.","solutions":["Retry the dump; transient connection drops are the most common cause.","Check redis-cli connectivity and server logs for OOM/restart/connection-limit events during the dump window.","If behind a load balancer/proxy, raise its idle timeout or connect directly to the Redis endpoint.","For ClusterClient, verify the cluster is healthy (cluster info) and all nodes reachable.","Upgrade go-redis / JuiceFS if errors are protocol-related with a newer Redis version."],"exampleFix":"// before\nif err := dumpChunks(...); err != nil { return err }\n// after\nvar lastErr error\nfor i := 0; i < 3; i++ {\n    if err := dumpChunks(...); err == nil { lastErr = nil; break }\n    else { lastErr = err; time.Sleep(time.Second) }\n}\nreturn lastErr","handlingStrategy":"retry","validationCode":"if err := rdb.Ping(ctx).Err(); err != nil { return fmt.Errorf(\"redis unreachable before dump: %w\", err) }","typeGuard":null,"tryCatchPattern":"for attempt := 0; attempt < 3; attempt++ {\n    err := dumpMeta(ctx, src, dst)\n    if err == nil { break }\n    var netErr net.Error\n    if errors.As(err, &netErr) || isRedisConnErr(err) { time.Sleep(backoff(attempt)); continue }\n    return err\n}","preventionTips":["Ping Redis immediately before starting a dump.","Avoid LB/proxy idle timeouts by keeping dumps short or raising idle timeout.","Monitor Redis memory and connection limits (maxmemory, maxclients) before bulk operations.","Pin a stable Redis endpoint rather than a DNS name behind flaky resolution."],"tags":["redis","pipeline","metadata-dump","network"],"backgroundTag":"database-query-failed","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}