{"record":{"id":"1304a87b8fc5ece1","repo":"vitessio/vitess","slug":"could-not-restart-source-streams-v","errorCode":null,"errorMessage":"could not restart source streams: %v","messagePattern":"could not restart source streams: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtctl/workflow/stream_migrator.go","lineNumber":226,"sourceCode":"\t}\n\terrs := &concurrency.AllErrorRecorder{}\n\n\tif err := sm.deleteTargetStreams(ctx); err != nil {\n\t\terrs.RecordError(fmt.Errorf(\"could not delete target streams: %v\", err))\n\t}\n\n\t// Restart the source streams, but leave the Reshard workflow's reverse\n\t// variant stopped.\n\terr := sm.ts.ForAllSources(func(source *MigrationSource) error {\n\t\t// We intend to update all but our workflow's reverse streams, so we\n\t\t// indicate that it's safe in this case using the comment diretive.\n\t\tquery := fmt.Sprintf(\"update /*vt+ %s */ _vt.vreplication set state='Running', stop_pos=null, message='' where db_name=%s and workflow != %s\",\n\t\t\tvreplication.AllowUnsafeWriteCommentDirective, encodeString(source.GetPrimary().DbName()), encodeString(sm.ts.ReverseWorkflowName()))\n\t\t_, err := sm.ts.VReplicationExec(ctx, source.GetPrimary().Alias, query)\n\t\treturn err\n\t})\n\tif err != nil {\n\t\terrs.RecordError(fmt.Errorf(\"could not restart source streams: %v\", err))\n\t\tsm.logger.Errorf(\"Cancel stream migrations failed: could not restart source streams: %v\", err)\n\t}\n\tif errs.HasErrors() {\n\t\treturn errs.AggrError(vterrors.Aggregate)\n\t}\n\treturn nil\n}\n\n// MigrateStreams migrates N streams\nfunc (sm *StreamMigrator) MigrateStreams(ctx context.Context) error {\n\tif sm.streams == nil {\n\t\treturn nil\n\t}\n\n\tif err := sm.deleteTargetStreams(ctx); err != nil {\n\t\treturn err\n\t}\n","sourceCodeStart":208,"sourceCodeEnd":244,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtctl/workflow/stream_migrator.go#L208-L244","documentation":"The second step of CancelStreamMigrations restarts the source streams by running an UPDATE on _vt.vreplication setting state='Running' on every source primary, excluding the workflow's reverse streams. If this VReplicationExec fails on any source, the error is recorded and aggregated with any earlier cleanup errors.","triggerScenarios":"CancelStreamMigration while a source shard primary is unreachable, rejects the update, or the SQL execution on the tablet fails; the reverse workflow name excluded in the query does not match rows as expected (that only affects which rows restart, not the error).","commonSituations":"Source primary tablet down or restarted mid-cancel; permissions/db_name mismatch causing zero-effect or failing updates; network issues between control plane and source tablets during a rolling maintenance.","solutions":["Verify each source shard has a healthy, reachable primary and retry the cancel.","Check tablet logs for the underlying VReplicationExec failure (the aggregated error includes it via %v).","If needed, run the equivalent UPDATE on the affected source tablets manually, then re-run cancel.","Complete any in-progress tablet restarts/maintenance before cancelling."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Check source primaries accept writes before cancelling\nfor _, shard := range sourceShards {\n    if !tabletHealthy(shard.Primary) { return fmt.Errorf(\"source primary %v unhealthy\", shard) }\n}","typeGuard":null,"tryCatchPattern":"err := client.CancelStreamMigration(ctx, keyspace, workflow)\nif err != nil && strings.Contains(err.Error(), \"could not restart source streams\") {\n    // inspect tablet logs for the VReplicationExec failure, fix, retry\n}","preventionTips":["Avoid cancelling during tablet restarts or maintenance windows","Verify source primary health and write access before cancel","Check the underlying error in the aggregate before manual intervention"],"tags":["vreplication","cancel","tablet","update"],"backgroundTag":"vreplication-cancel-cleanup-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}