{"record":{"id":"ac527d429650f779","repo":"juicedata/juicefs","slug":"expect-key-v-but-got-v","errorCode":null,"errorMessage":"expect key %v, but got %v","messagePattern":"expect key (.+?), but got (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/meta/tkv_etcd.go","lineNumber":65,"sourceCode":"\t\treturn v\n\t}\n\tresp, err := tx.kv.Get(tx.ctx, k, etcd.WithLimit(1))\n\tif err != nil {\n\t\tpanic(fmt.Errorf(\"get %v: %s\", k, err))\n\t}\n\tif resp.Count == 0 {\n\t\ttx.observed[k] = 0\n\t\treturn nil\n\t}\n\tif resp.Count > 1 {\n\t\tpanic(fmt.Errorf(\"expect 1 keys but got %d\", resp.Count))\n\t}\n\tfor _, pair := range resp.Kvs {\n\t\tif bytes.Equal(pair.Key, key) {\n\t\t\ttx.observed[k] = pair.ModRevision\n\t\t\treturn pair.Value\n\t\t} else {\n\t\t\tpanic(fmt.Errorf(\"expect key %v, but got %v\", k, string(pair.Key)))\n\t\t}\n\t}\n\tpanic(\"unreachable\")\n}\n\nfunc (tx *etcdTxn) gets(keys ...[]byte) [][]byte {\n\tif len(keys) > 128 {\n\t\tvar rs = make([][]byte, 0, len(keys))\n\t\tfor i := 0; i < len(keys); i += 128 {\n\t\t\trs = append(rs, tx.gets(keys[i:min(i+128, len(keys))]...)...)\n\t\t}\n\t\treturn rs\n\t}\n\tops := make([]etcd.Op, len(keys))\n\tfor i, key := range keys {\n\t\tops[i] = etcd.OpGet(string(key))\n\t}\n\tr, err := tx.kv.Do(tx.ctx, etcd.OpTxn(nil, ops, nil))","sourceCodeStart":47,"sourceCodeEnd":83,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/meta/tkv_etcd.go#L47-L83","documentation":"`etcdTxn.get` fetched a key but etcd returned a KV whose key differs from the requested one (since Get with a key acts as a range from that key). The wrapper panics because the transaction observed the wrong key, which would break optimistic-concurrency (observed revision) bookkeeping.","triggerScenarios":"The etcd Get returns a successor key instead of the exact requested key — happens when the requested key does not exist but the Get was issued without proper range end (WithRange/WithPrefix), so the next key in order is returned.","commonSituations":"Reading a deleted/nonexistent key with a misconfigured txn implementation; a forked or modified tkv_etcd.go; extremely rare with stock code, points to implementation or etcd-server anomaly.","solutions":["Check whether the key should exist; dump related keys with `etcdctl get --prefix` to see what etcd actually returned","Verify you are running an unmodified JuiceFS build and a supported etcd version","If the key was deleted concurrently, retry the operation; if it reproduces, report to JuiceFS with versions and key bytes"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"if strings.Contains(err.Error(), \"expect key \") {\n    log.Fatalf(\"etcd txn observed wrong key, likely deleted concurrently or a bug: %v\", err)\n}","preventionTips":["Avoid sharing one metadata DB across unrelated tools that delete keys","Use a dedicated prefix for the volume","Report reproducible cases to JuiceFS with etcd version and stack trace"],"tags":["etcd","invariant","panic","transaction","juicefs"],"backgroundTag":"internal-invariant-violation","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}