vitessio/vitess · error

failed to delete post copy action for the %q table with id %

Error message

failed to delete post copy action for the %q table with id %d: %v

What it means

After executing a post copy action (typically a deferred ALTER TABLE adding secondary indexes), VReplication deletes the corresponding row from the _vt.post_copy_action table. This error wraps any MySQL failure of that DELETE, including the workflow/table name and action id for context. If it fails, the action row remains and could be re-executed on a later copy attempt.

Source

Thrown at go/vt/vttablet/tabletmanager/vreplication/vreplicator.go:993

		return err
	}
	// qr should never be nil, but check anyway to be extra safe.
	if idqr == nil || len(idqr.Rows) != 1 {
		return fmt.Errorf("unexpected number of rows returned (%d) from connection_id() query", len(idqr.Rows))
	}
	connID, err := idqr.Rows[0][0].ToInt64()
	if err != nil || connID == 0 {
		return fmt.Errorf("unexpected result (%d) from connection_id() query, error: %v", connID, err)
	}

	deleteAction := func(dbc *vdbClient, id int64, vid int32, tn string) error {
		delq, err := sqlparser.ParseAndBind(sqlDeletePostCopyAction, sqltypes.Int32BindVariable(vid),
			sqltypes.StringBindVariable(tn), sqltypes.Int64BindVariable(id))
		if err != nil {
			return err
		}
		if _, err := dbc.ExecuteFetch(delq, 1); err != nil {
			return fmt.Errorf("failed to delete post copy action for the %q table with id %d: %v",
				tableName, id, err)
		}
		return nil
	}

	// This could take hours so we start a monitoring goroutine to
	// listen for the cancellations which indicate that the controller
	// is stopping: engine shutdown (tablet shutdown or transition) or
	// controller stop (the workflow is being stopped, deleted, or
	// updated). If either happens we KILL the connection being used
	// to execute the actions, using a DBA connection, so that any
	// in-flight statement (e.g. an ALTER) is aborted and any
	// subsequent statement on the connection fails immediately.
	// If we don't do this then we could e.g. cause a PRS to fail as
	// the running ALTER will block setting [super_]read_only, or cause
	// a workflow delete to time out as the engine cannot process it
	// until the controller has stopped.
	// A failed/killed ALTER will be tried again when the copy

View on GitHub (pinned to 01a25a7d17)

Solutions

  1. Check the wrapped cause (%v suffix) for the underlying MySQL error and address it (read-only mode, connection killed, lock timeout)
  2. Re-run/refresh the workflow — an already-executed action is detected as duplicate (ERDupKeyName handling) and skipped, or the row is cleaned up on the next copy
  3. Verify the target tablet is writable and the _vt schema is intact (post_copy_action table exists)
  4. If the action row is orphaned but the schema change is already applied, clean up _vt.post_copy_action manually with DBA access
Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-check writability and row existence before retrying the workflow
qr, err := dbaConn.ExecuteFetch("SELECT @@read_only, (SELECT COUNT(*) FROM _vt.post_copy_action) ", 1, false)

Try / catch

if _, err := dbc.ExecuteFetch(delq, 1); err != nil {
    if sqlErr, ok := err.(*sqlerror.SQLError); ok {
        switch sqlErr.Number() {
        case sqlerror.ERServerIsReadOnly:
            // wait for read-only window (e.g. PRS) to end, then retry
        case 1205: // lock wait timeout
            // retry after competing controllers finish
        }
    }
    return fmt.Errorf("failed to delete post copy action for the %q table with id %d: %v", tableName, id, err)
}

Prevention

When it happens

Trigger: The deleteAction closure inside execPostCopyActions runs the bound DELETE on _vt.post_copy_action (via sqlparser.ParseAndBind + dbClient.ExecuteFetch) and the ExecuteFetch returns an error — e.g. the connection was killed, the tablet is read-only, or the table is missing/locked.

Common situations: Workflow interrupted mid-action (KILL of the connection due to engine shutdown or controller stop); target tablet flipped to read-only (e.g. during PRS); long-running DDL lock contention on _vt.post_copy_action; shard merges with concurrent controllers.

Related errors


AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/5643c8796e678696. Report an issue: GitHub.