vitessio/vitess · error
GetBackups(%s/%s): %w
Error message
GetBackups(%s/%s): %w
What it means
The per-shard GetBackups vtctld RPC failed after a pool slot was obtained. The error is wrapped with keyspace/shard so the operator can identify which shard's backup inventory could not be read. Other shards proceed independently.
Source
Thrown at go/vt/vtadmin/cluster/cluster.go:896
span.Annotate("keyspace", keyspace)
span.Annotate("shard", shard)
if err := c.backupReadPool.Acquire(ctx); err != nil {
rec.RecordError(fmt.Errorf("GetBackups(%s/%s) failed to acquire backupReadPool: %w", keyspace, shard, err))
return
}
resp, err := c.Vtctld.GetBackups(ctx, &vtctldatapb.GetBackupsRequest{
Keyspace: keyspace,
Shard: shard,
Limit: req.RequestOptions.Limit,
Detailed: req.RequestOptions.Detailed,
DetailedLimit: req.RequestOptions.DetailedLimit,
})
c.backupReadPool.Release()
if err != nil {
rec.RecordError(fmt.Errorf("GetBackups(%s/%s): %w", keyspace, shard, err))
return
}
shardBackups := make([]*vtadminpb.ClusterBackup, len(resp.Backups))
for i, backup := range resp.Backups {
shardBackups[i] = &vtadminpb.ClusterBackup{
Cluster: clusterProto,
Backup: backup,
}
}
m.Lock()
defer m.Unlock()
backups = append(backups, shardBackups...)
}(ks, shard)
}
}View on GitHub (pinned to 01a25a7d17)
Solutions
- Check the wrapped cause and vtctld logs for that shard
- Verify backup storage configuration and connectivity
- Confirm the shard exists in the topology
- Retry the listing
Defensive patterns
Strategy: retry
Validate before calling
// confirm shard exists in topology before listing backups
sr, err := vtctldClient.GetShard(ctx, &vtctldatapb.GetShardRequest{Keyspace: ks, Shard: shard})
if err != nil { return fmt.Errorf("shard %s/%s not queryable: %w", ks, shard, err) } Try / catch
backups, err := cluster.GetBackups(ctx, req)
if err != nil {
log.Warn("backup listing failed for shard", slog.Any("error", err))
// inspect wrapped cause; retry transient failures
} Prevention
- Verify backup storage config and connectivity
- Keep topology records for active shards intact
- Set per-request timeouts above vtctld latency
- Alert on vtctld restarts
When it happens
Trigger: c.Vtctld.GetBackups RPC returning an error for a specific keyspace/shard — vtctld unreachable, backup directory unreadable on the backup storage, or shard missing in topology.
Common situations: Backup storage (NFS/S3) misconfigured or temporarily unavailable; shard recently created/removed; vtctld restarting during the sweep; network flake.
Related errors
- invalid key:value pair
- ReloadSchemas(cluster = %s) failed: %w
- Error setting tablet to read-only: %w
- Error setting tablet to read-write: %w
- GetKeyspaces(cluster = %s) failed: %w
AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01).
Data as JSON: /api/errors/fe2acd48f662576c.
Report an issue: GitHub.