vitessio/vitess · error

schema diffs: %v

Error message

schema diffs: %v

What it means

Aggregate error returned by ValidateSchemaShard when any per-tablet schema diff (or GetSchema error recorded by diffSchema) was recorded in the ErrorRecorder. It wraps all recorded errors, which may include actual schema differences between replicas and the primary as well as schema-fetch failures.

Source

Thrown at go/vt/wrangler/schema.go:102

	aliases, err := wr.ts.FindAllTabletAliasesInShard(ctx, keyspace, shard)
	if err != nil {
		return fmt.Errorf("FindAllTabletAliasesInShard(%v, %v) failed: %v", keyspace, shard, err)
	}

	// then diff with all replicas
	er := concurrency.AllErrorRecorder{}
	wg := sync.WaitGroup{}
	for _, alias := range aliases {
		if topoproto.TabletAliasEqual(alias, si.PrimaryAlias) {
			continue
		}

		wg.Add(1)
		go wr.diffSchema(ctx, primarySchema, si.PrimaryAlias, alias, excludeTables, includeViews, &wg, &er)
	}
	wg.Wait()
	if er.HasErrors() {
		return fmt.Errorf("schema diffs: %v", er.Error().Error())
	}
	return nil
}

// ValidateSchemaKeyspace will diff the schema from all the tablets in the keyspace.
func (wr *Wrangler) ValidateSchemaKeyspace(ctx context.Context, keyspace string, excludeTables []string, includeViews, skipNoPrimary bool, includeVSchema bool) error {
	res, err := wr.VtctldServer().ValidateSchemaKeyspace(ctx, &vtctldatapb.ValidateSchemaKeyspaceRequest{
		Keyspace:       keyspace,
		ExcludeTables:  excludeTables,
		IncludeViews:   includeViews,
		IncludeVschema: includeVSchema,
		SkipNoPrimary:  skipNoPrimary,
	})

	for _, result := range res.Results {
		wr.Logger().Printf("%s\n", result)
	}

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Inspect the wrapped error list to see which tables and tablets differ
2. Re-apply the primary's schema to drifting replicas (vtctldclient ApplySchema --skip-verify or manually syncing the drifted tablets)
3. Restore replicas from a fresh backup of the primary if drift is extensive
4. Fix any underlying GetSchema errors first, then re-run ValidateSchemaShard to confirm clean

Example fix

// before: replica drifted
mysql> ALTER TABLE customer ADD COLUMN x INT; -- run only on replica
// after: apply schema through vitess to all tablets
vtctldclient ApplySchema --sql "ALTER TABLE customer ADD COLUMN x INT" commerce

// compare table definitions yourself before validating
primaryDefs := map[string]string{}
for _, t := range primarySchema.TableDefinitions { primaryDefs[t.Name] = t.Schema }
for _, t := range replicaSchema.TableDefinitions {
    if primaryDefs[t.Name] != t.Schema { return fmt.Errorf("table %s differs on replica", t.Name) }
}

func schemasMatch(primary, replica *tabletmanagerdatapb.SchemaDefinition) bool {
    if len(primary.TableDefinitions) != len(replica.TableDefinitions) { return false }
    for i, td := range primary.TableDefinitions {
        if td.Name != replica.TableDefinitions[i].Name || td.Schema != replica.TableDefinitions[i].Schema { return false }
    }
    return true
}

if err := wr.ValidateSchemaShard(ctx, ks, shard, nil, true, false); err != nil {
    if strings.Contains(err.Error(), "schema diffs") {
        // parse per-tablet diffs from err and re-apply canonical schema
    }
}

When it happens

Trigger: Any tablet in the shard has a schema differing from the primary (different CREATE TABLE definition, missing/extra tables or views), or diffSchema recorded a GetSchema failure for some tablet.

Common situations: Manual ALTER TABLE applied to a replica but not replicated (e.g. schema applied outside of vtctldclient ApplySchema with replication broken); restored replica from an older backup; drifted table definitions after a failed migration.

Related errors

• table %v not found in vschema
• GetSchema(%v, nil, %v, %v) failed: %v
• GetShard(%v, %v) failed: %v
• no primary in shard %v/%v
• FindAllTabletAliasesInShard(%v, %v) failed: %v

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/beadfddca924467d. Report an issue: GitHub.