{"record":{"id":"899d11e28ace529d","repo":"vitessio/vitess","slug":"failed-to-get-post-copy-actions-for-the-q-table","errorCode":null,"errorMessage":"failed to get post copy actions for the %q table: %v","messagePattern":"failed to get post copy actions for the %q table: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vttablet/tabletmanager/vreplication/vreplicator.go","lineNumber":1123,"sourceCode":"\t\t// the same action, then we skip executing it as an individual\n\t\t// action on a table should only be done by the last vreplicator\n\t\t// to finish. We use a transaction because we select matching\n\t\t// rows with FOR UPDATE in order to serialize the execution of\n\t\t// the post copy actions for the same workflow and table.\n\t\t// This ensures that the actions are only performed once after\n\t\t// all streams have completed the copy phase for the table.\n\t\tredundant := false\n\t\t_, err = dbClient.ExecuteFetch(\"start transaction\", 1)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tvrsq, err := sqlparser.ParseAndBind(sqlGetAndLockPostCopyActionsForTable, sqltypes.StringBindVariable(tableName))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tvrsres, err := dbClient.ExecuteFetch(vrsq, -1)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to get post copy actions for the %q table: %v\", tableName, err)\n\t\t}\n\t\tif vrsres != nil && len(vrsres.Rows) > 1 {\n\t\t\t// We have more than one planned post copy action on the table.\n\t\t\tfor _, row := range vrsres.Named().Rows {\n\t\t\t\tvrid, err := row[\"vrepl_id\"].ToInt32()\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tctlaction := row[\"action\"].ToString()\n\t\t\t\t// Let's make sure that it's a different controller/vreplicator\n\t\t\t\t// and that the action is the same.\n\t\t\t\tif vrid != vr.id && strings.EqualFold(ctlaction, string(actionBytes)) {\n\t\t\t\t\t// We know that there's another controller/vreplicator yet\n\t\t\t\t\t// to finish its copy phase for the table and it will perform\n\t\t\t\t\t// the same action on the same table when it completes, so we\n\t\t\t\t\t// skip doing the action and simply delete our action record\n\t\t\t\t\t// to mark this controller/vreplicator's post copy action work\n\t\t\t\t\t// as being done for the table before it finishes the copy","sourceCodeStart":1105,"sourceCodeEnd":1141,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vttablet/tabletmanager/vreplication/vreplicator.go#L1105-L1141","documentation":"Before executing each post copy action, execPostCopyActions opens a transaction and re-selects the action rows for the table with FOR UPDATE to serialize execution between concurrent vreplicators (e.g. during shard merges). This error wraps any MySQL failure of that locked select, including the table name. Without this check, duplicate ALTERs by multiple controllers could race.","triggerScenarios":"The bound sqlGetAndLockPostCopyActionsForTable query fails on dbClient.ExecuteFetch — e.g. the connection was killed by the interruption goroutine (CRServerLost/EOF), a lock wait timeout on the row lock, or the tablet went read-only mid-transaction.","commonSituations":"Workflow cancelled/stopped while the FOR UPDATE select is blocked by another controller's transaction (innodb_lock_wait_timeout exceeded); connection killed during engine shutdown; shard-merge concurrency contention on _vt.post_copy_action.","solutions":["Inspect the wrapped MySQL error: for lock wait timeout, check for competing vreplication controllers/transactions on the same table and retry after they finish","If the connection was killed (2013/EOF) during a workflow stop, simply restart the workflow — actions are retried on the new primary","Check innodb_lock_wait_timeout is adequate if shard merges run many concurrent controllers","Verify _vt.post_copy_action is healthy/consistent; clean orphaned rows with DBA access if needed"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Before restarting, check for competing/stranded lock holders:\nSHOW FULL PROCESSLIST; -- look for transactions holding locks on _vt.post_copy_action\nSELECT @@innodb_lock_wait_timeout;","typeGuard":null,"tryCatchPattern":"vrsres, err := dbClient.ExecuteFetch(vrsq, -1)\nif err != nil {\n    if sqlErr, ok := err.(*sqlerror.SQLError); ok && sqlErr.Number() == 1205 {\n        // lock wait timeout: wait for competing vreplicator, then retry\n    }\n    return fmt.Errorf(\"failed to get post copy actions for the %q table: %v\", tableName, err)\n}","preventionTips":["Avoid running workflow updates/deletes concurrently with shard-merge copy phases on the same tables","Set an adequate innodb_lock_wait_timeout when many controllers run concurrently","Restart the workflow after a killed connection — actions are idempotent-guarded (duplicate index detection) and retried","Keep _vt.post_copy_action free of manual long-running transactions"],"tags":["vreplication","mysql","locking","go"],"backgroundTag":"row-lock-contention","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}