{"record":{"id":"bb7fa39685c765bc","repo":"vitessio/vitess","slug":"shard-s-is-still-serving","errorCode":null,"errorMessage":"shard %s is still serving","messagePattern":"shard (.+?) is still serving","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtctl/workflow/utils.go","lineNumber":558,"sourceCode":"\thasher := fnv.New64()\n\thasher.Write([]byte(targetKeyspace))\n\n\tfor _, s := range expanded {\n\t\thasher.Write([]byte(s))\n\t}\n\n\t// Convert to int64 after dropping the highest bit.\n\treturn int64(hasher.Sum64() & math.MaxInt64)\n}\n\nfunc doValidateWorkflowHasCompleted(ctx context.Context, ts *trafficSwitcher) error {\n\twg := sync.WaitGroup{}\n\trec := concurrency.AllErrorRecorder{}\n\tif ts.MigrationType() == binlogdatapb.MigrationType_SHARDS {\n\t\t_ = ts.ForAllSources(func(source *MigrationSource) error {\n\t\t\twg.Add(1)\n\t\t\tif source.GetShard().IsPrimaryServing {\n\t\t\t\trec.RecordError(fmt.Errorf(\"shard %s is still serving\", source.GetShard().ShardName()))\n\t\t\t}\n\t\t\twg.Done()\n\t\t\treturn nil\n\t\t})\n\t} else {\n\t\t_ = ts.ForAllTargets(func(target *MigrationTarget) error {\n\t\t\twg.Add(1)\n\t\t\tdefer wg.Done()\n\t\t\tres, err := ts.ws.tmc.ReadVReplicationWorkflow(ctx, target.GetPrimary().Tablet, &tabletmanagerdatapb.ReadVReplicationWorkflowRequest{\n\t\t\t\tWorkflow: ts.WorkflowName(),\n\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\trec.RecordError(err)\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tfor _, stream := range res.Streams {\n\t\t\t\tif stream.Message != Frozen {\n\t\t\t\t\trec.RecordError(fmt.Errorf(\"vreplication streams are not frozen on tablet %d\", target.GetPrimary().Alias.Uid))","sourceCodeStart":540,"sourceCodeEnd":576,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtctl/workflow/utils.go#L540-L576","documentation":"During workflow validation (check for whether a migration has fully completed, e.g. before finalizing a shard migration), this check walks each migration source shard and records an error if the shard's primary is still marked IsPrimaryServing. A shard that still serves traffic is not safe to consider migrated/decommissioned. The error is recorded per shard via an AllErrorRecorder inside ForAllSources.","triggerScenarios":"Running validateWorkflowHasCompleted (traffic validation for MigrateTables/Reshard of type SHARDS) while the source shards' primaries still have IsPrimaryServing=true in the topo — i.e. PRS/traffic switch away from those shards has not happened.","commonSituations":"Attempting to complete a ReshardWorkflow before SwitchTraffic drained reads/writes; a failed or skipped traffic switch step; manual topo edits leaving the serving flag set.","solutions":["Complete the traffic switch (SwitchTraffic) for the source shards so serving is disabled before finalizing.","Check the shard record (`vtctldclient GetTablets` / topo) and confirm IsPrimaryServing=false on migrated source primaries.","If serving was already drained but the flag is stale, refresh/repair the topo record via the proper vtctldclient commands rather than finalizing the workflow."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// verify source primaries are no longer serving before completing the workflow\nfor _, s := range sourceShards {\n    if shardPrimaryInfo(ks, s).IsPrimaryServing {\n        return fmt.Errorf(\"shard %s still serving; run SwitchTraffic first\", s)\n    }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always run SwitchTraffic (serving disabled) before Complete/finalization.","Monitor serving flags via GetTablets during shard migrations.","Avoid manual topo edits that can leave IsPrimaryServing inconsistent."],"tags":["go","topology","reshard","validation"],"backgroundTag":"source-shard-still-serving","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}