What causes "get %v: %s" in juicedata/juicefs?

Check etcd cluster health: `etcdctl endpoint health` and inspect client logs for the wrapped gRPC error

get %v: %s

Error message

get %v: %s

What it means

Inside an etcd transaction wrapper (`etcdTxn.get`), a point Get RPC to etcd failed. Because the txn layer cannot know whether the read happened, the code panics with this message wrapping the underlying etcd error; the panic is recovered by JuiceFS and reported as a transaction failure.

Source

Thrown at pkg/meta/tkv_etcd.go:51

	etcd "go.etcd.io/etcd/client/v3"
	"go.etcd.io/etcd/pkg/transport"
)

type etcdTxn struct {
	ctx      context.Context
	kv       etcd.KV
	observed map[string]int64
	buffer   map[string][]byte
}

func (tx *etcdTxn) get(key []byte) []byte {
	k := string(key)
	if v, ok := tx.buffer[k]; ok {
		return v
	}
	resp, err := tx.kv.Get(tx.ctx, k, etcd.WithLimit(1))
	if err != nil {
		panic(fmt.Errorf("get %v: %s", k, err))
	}
	if resp.Count == 0 {
		tx.observed[k] = 0
		return nil
	}
	if resp.Count > 1 {
		panic(fmt.Errorf("expect 1 keys but got %d", resp.Count))
	}
	for _, pair := range resp.Kvs {
		if bytes.Equal(pair.Key, key) {
			tx.observed[k] = pair.ModRevision
			return pair.Value
		} else {
			panic(fmt.Errorf("expect key %v, but got %v", k, string(pair.Key)))
		}
	}
	panic("unreachable")
}

View on GitHub (pinned to c9a67b23e8)

Solutions

Check etcd cluster health: `etcdctl endpoint health` and inspect client logs for the wrapped gRPC error
Fix connectivity/endpoint issues (endpoints, TLS certs, credentials) in the metadata URL
Increase client timeouts / review `--metacache` and network latency; retry the failed operation once etcd is reachable
If errors persist, verify etcd disk latency and compaction settings (slow disk causes request timeouts)

Defensive patterns

Strategy: retry

Validate before calling

// Before mounting, verify etcd reachability
etcdctl --endpoints=host:2379 endpoint health

Try / catch

// JuiceFS recovers the panic and surfaces it as a tx error; wrap mount/load with retry
for i := 0; i < 3; i++ {
    err := mountVolume()
    if err == nil { break }
    log.Printf("etcd get failed (%v), retrying", err)
    time.Sleep(time.Duration(1<<i) * time.Second)
}

Prevention

Monitor etcd with endpoint health checks and alert on leader changes
Right-size etcd disks (low fsync latency) and tune election/heartbeat timeouts
Keep client and etcd versions supported by JuiceFS
Use stable endpoints with TLS and auth credentials verified up front

When it happens

Trigger: Any transaction that reads a key (`cache`, `load`, `exist`, `add` paths) when the etcd Get returns an error: connection lost, etcd down, context deadline exceeded, auth failure, or etcd returning a non-OK gRPC status.

Common situations: etcd cluster unreachable or leader election in progress; network partition between client and etcd; etcd restarting during compaction; wrong endpoints or credentials; request context timeout exceeded under load.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/95bbeb55c106890a. Report an issue: GitHub.