vitessio/vitess · error

ReloadSchemaShard(%s/%s) failed: %w

Error message

ReloadSchemaShard(%s/%s) failed: %w

What it means

In reloadKeyspaceSchemas, when the request targets specific shards, each shard-level ReloadSchema call's error is wrapped as 'ReloadSchemaShard(%s/%s) failed' and recorded in the errgroup collector. The underlying cause is the vtctld ReloadSchemaShard RPC failure for that keyspace/shard.

Source

Thrown at go/vt/vtadmin/cluster/cluster.go:2320

				span, ctx := trace.NewSpan(ctx, "Cluster.reloadShardSchema")
				defer span.Finish()

				AnnotateSpan(c, span)
				span.Annotate("keyspace", keyspace)
				span.Annotate("shard", shard)
				span.Annotate("concurrency", req.Concurrency)
				span.Annotate("include_primary", req.IncludePrimary)
				span.Annotate("wait_position", req.WaitPosition)

				resp, err := c.Vtctld.ReloadSchemaShard(ctx, &vtctldatapb.ReloadSchemaShardRequest{
					Keyspace:       keyspace,
					Shard:          shard,
					Concurrency:    req.Concurrency,
					IncludePrimary: req.IncludePrimary,
					WaitPosition:   req.WaitPosition,
				})
				if err != nil {
					rec.RecordError(fmt.Errorf("ReloadSchemaShard(%s/%s) failed: %w", keyspace, shard, err))
					return
				}

				m.Lock()
				defer m.Unlock()
				results = append(results, &vtadminpb.ReloadSchemasResponse_ShardResult{
					Shard: &vtadminpb.Shard{
						Cluster: cpb,
						Shard: &vtctldatapb.Shard{
							Keyspace: keyspace,
							Name:     shard,
						},
					},
					Events: resp.Events,
				})
			}(ks, shard)
		}
	}

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Read the wrapped cause to identify the failing tablet in the shard
2. Verify all tablets in the shard are serving and reachable (vtctldclient GetTablets)
3. Retry the shard reload after restoring tablet health
4. Relax WaitPosition or IncludePrimary options if they cannot be satisfied

Example fix

// before
req := &vtadminpb.ReloadSchemasRequest{Keyspaces: []string{"ks"}, Shards: []string{"-"}}
// after (retry only the failing shard once the tablet recovers)
req := &vtadminpb.ReloadSchemasRequest{KeyspacesAndShards: []*vtadminpb.ReloadSchemasRequest_KeyspaceShard{{Keyspace: "ks", Shard: "-"}}}

res, err := c.ReloadSchemas(ctx, req)
if err != nil {
    if strings.Contains(err.Error(), "ReloadSchemaShard(") {
        // parse keyspace/shard from message and retry that shard only
    }
}

When it happens

Trigger: vtctld ReloadSchema for a specific keyspace/shard fails — tablets in that shard unreachable, RPC timeout, reload with WaitPosition/IncludePrimary options failing on a tablet, or context cancellation.

Common situations: Reload after DDL on a shard whose replica is lagging (WaitPosition unmet); shard with a down tablet; cell isolation preventing RPCs to that shard's tablets.

Related errors

• ReloadSchemaKeyspace(%s) failed: %w
• %w: cannot create shard in %s
• %w: cannot delete shards in %s
• %w: shard name is required
• ReloadSchemas: failed to acquire topoReadPool: %w

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/e9b2d5ffa4e2e1bd. Report an issue: GitHub.