{"record":{"id":"95bbeb55c106890a","repo":"juicedata/juicefs","slug":"get-v-s","errorCode":null,"errorMessage":"get %v: %s","messagePattern":"get %v: %s","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/meta/tkv_etcd.go","lineNumber":51,"sourceCode":"\tetcd \"go.etcd.io/etcd/client/v3\"\n\t\"go.etcd.io/etcd/pkg/transport\"\n)\n\ntype etcdTxn struct {\n\tctx      context.Context\n\tkv       etcd.KV\n\tobserved map[string]int64\n\tbuffer   map[string][]byte\n}\n\nfunc (tx *etcdTxn) get(key []byte) []byte {\n\tk := string(key)\n\tif v, ok := tx.buffer[k]; ok {\n\t\treturn v\n\t}\n\tresp, err := tx.kv.Get(tx.ctx, k, etcd.WithLimit(1))\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"get %v: %s\", k, err))\n\t}\n\tif resp.Count == 0 {\n\t\ttx.observed[k] = 0\n\t\treturn nil\n\t}\n\tif resp.Count > 1 {\n\t\tpanic(fmt.Errorf(\"expect 1 keys but got %d\", resp.Count))\n\t}\n\tfor _, pair := range resp.Kvs {\n\t\tif bytes.Equal(pair.Key, key) {\n\t\t\ttx.observed[k] = pair.ModRevision\n\t\t\treturn pair.Value\n\t\t} else {\n\t\t\tpanic(fmt.Errorf(\"expect key %v, but got %v\", k, string(pair.Key)))\n\t\t}\n\t}\n\tpanic(\"unreachable\")\n}","sourceCodeStart":33,"sourceCodeEnd":69,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/meta/tkv_etcd.go#L33-L69","documentation":"Inside an etcd transaction wrapper (`etcdTxn.get`), a point Get RPC to etcd failed. Because the txn layer cannot know whether the read happened, the code panics with this message wrapping the underlying etcd error; the panic is recovered by JuiceFS and reported as a transaction failure.","triggerScenarios":"Any transaction that reads a key (`cache`, `load`, `exist`, `add` paths) when the etcd Get returns an error: connection lost, etcd down, context deadline exceeded, auth failure, or etcd returning a non-OK gRPC status.","commonSituations":"etcd cluster unreachable or leader election in progress; network partition between client and etcd; etcd restarting during compaction; wrong endpoints or credentials; request context timeout exceeded under load.","solutions":["Check etcd cluster health: `etcdctl endpoint health` and inspect client logs for the wrapped gRPC error","Fix connectivity/endpoint issues (endpoints, TLS certs, credentials) in the metadata URL","Increase client timeouts / review `--metacache` and network latency; retry the failed operation once etcd is reachable","If errors persist, verify etcd disk latency and compaction settings (slow disk causes request timeouts)"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Before mounting, verify etcd reachability\netcdctl --endpoints=host:2379 endpoint health","typeGuard":null,"tryCatchPattern":"// JuiceFS recovers the panic and surfaces it as a tx error; wrap mount/load with retry\nfor i := 0; i < 3; i++ {\n    err := mountVolume()\n    if err == nil { break }\n    log.Printf(\"etcd get failed (%v), retrying\", err)\n    time.Sleep(time.Duration(1<<i) * time.Second)\n}","preventionTips":["Monitor etcd with endpoint health checks and alert on leader changes","Right-size etcd disks (low fsync latency) and tune election/heartbeat timeouts","Keep client and etcd versions supported by JuiceFS","Use stable endpoints with TLS and auth credentials verified up front"],"tags":["etcd","network","grpc","transaction","juicefs"],"backgroundTag":"database-query-failed","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}