{"record":{"id":"cc53f5e60ff90a48","repo":"vitessio/vitess","slug":"streams-are-mismatched-across-source-shards-s-vs","errorCode":null,"errorMessage":"streams are mismatched across source shards: %s vs %s","messagePattern":"streams are mismatched across source shards: (.+?) vs (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtctl/workflow/stream_migrator.go","lineNumber":524,"sourceCode":"\t\tstreams2[k] = append([]*VReplicationStream(nil), v...)\n\t}\n\n\tfor shard, tabletStreams := range streams2 {\n\t\tfor _, refStream := range reference {\n\t\t\terr := func() error {\n\t\t\t\tfor i := 0; i < len(tabletStreams); i++ {\n\t\t\t\t\tvrs := tabletStreams[i]\n\n\t\t\t\t\tif refStream.Workflow == vrs.Workflow &&\n\t\t\t\t\t\trefStream.BinlogSource.Keyspace == vrs.BinlogSource.Keyspace &&\n\t\t\t\t\t\trefStream.BinlogSource.Shard == vrs.BinlogSource.Shard {\n\t\t\t\t\t\t// Delete the matched item and scan for the next stream.\n\t\t\t\t\t\ttabletStreams = append(tabletStreams[:i], tabletStreams[i+1:]...)\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\treturn fmt.Errorf(\"streams are mismatched across source shards: %s vs %s\", refshard, shard)\n\t\t\t}()\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\n\t\tif len(tabletStreams) != 0 {\n\t\t\treturn nil, fmt.Errorf(\"streams are mismatched across source shards: %s vs %s\", refshard, shard)\n\t\t}\n\t}\n\n\treturn streams, nil\n}\n\nfunc (sm *StreamMigrator) readSourceStreams(ctx context.Context, cancelMigrate bool) (map[string][]*VReplicationStream, error) {\n\tvar (\n\t\tmu      sync.Mutex\n\t\tstreams = make(map[string][]*VReplicationStream)","sourceCodeStart":506,"sourceCodeEnd":542,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtctl/workflow/stream_migrator.go#L506-L542","documentation":"legacyReadSourceStreams validates that every source shard has an identical set of VReplication streams (matched by workflow name and BinlogSource keyspace/shard). After matching each reference stream, if a stream from the reference shard cannot be found on this shard, the streams are inconsistent across the reshard's source shards and migration is aborted, naming the two shards.","triggerScenarios":"Running MigrateStreams on a reshard source whose shards have different sets of workflows/streams — e.g. one shard has workflow 'sales' replicating from ks1/-80 but another shard lacks it or sources it from a different shard.","commonSituations":"Partial creation of a workflow across shards (creation failed mid-way); operator manually created streams on some shards only; source keyspace shards replicate from different keyspaces/shards after ad-hoc operations.","solutions":["Compare streams per shard: run `select id, workflow, source from _vt.vreplication` on each source shard primary and diff the (workflow, source keyspace/shard) tuples.","Create the missing workflow/stream on the shard that lacks it (vtctldclient MoveTables/VDiff-equivalent creation or Workflow Create) so all shards match.","Remove the extra stream from the shard that has it if it should not exist, then retry the migration.","Re-create the whole workflow consistently across all shards using the standard workflow tooling instead of manual stream creation."],"exampleFix":"// before: shard -80 has stream for ks1/-80, shard 80- does not\n// after: create matching stream on 80- or remove from -80, then\n$ vtctldclient Workflow --keyspace customer Migrate ...","handlingStrategy":"validation","validationCode":"streamSets := map[string]map[string]bool{} // shard -> (workflow|keyspace|shard) set\nfor shard, prim := range sourcePrimaries {\n    qr, _ := tmclient.VReplicationExec(ctx, prim, \"select workflow, source from _vt.vreplication\")\n    streamSets[shard] = collectKeys(qr)\n}\nif !allEqual(streamSets) {\n    return fmt.Errorf(\"align stream sets across source shards before migrating\")\n}","typeGuard":null,"tryCatchPattern":"err := migrateStreams(ctx, cfg)\nif err != nil && strings.Contains(err.Error(), \"streams are mismatched across source shards\") {\n    // diff _vt.vreplication across the two named shards, add missing or remove extra streams\n}","preventionTips":["Create workflows via vtctldclient so all shards get identical streams","Diff stream sets across source shards before any migration","Never hand-edit _vt.vreplication on individual shards"],"tags":["vitess","vreplication","workflow-migration","shard-consistency"],"backgroundTag":"stream-mismatch-across-shards","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}