{"record":{"id":"1612de6e98052d7c","repo":"vitessio/vitess","slug":"could-not-set-state-v-v-1612de","errorCode":null,"errorMessage":"could not set state: %v: %v","messagePattern":"could not set state: (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vttablet/tabletmanager/vreplication/vreplicator.go","lineNumber":588,"sourceCode":"\tinsertLog(vr.dbClient, typ, vr.id, vr.state.String(), message)\n}\n\nfunc (vr *vreplicator) setState(state binlogdatapb.VReplicationWorkflowState, message string) error {\n\tif message != \"\" {\n\t\tvr.stats.History.Add(&binlogplayer.StatsHistoryRecord{\n\t\t\tTime:    time.Now(),\n\t\t\tMessage: message,\n\t\t})\n\t}\n\tvr.stats.State.Store(state.String())\n\tquery := fmt.Sprintf(\"update _vt.vreplication set state=%v, message=left(%v, 1000) where id=%v\", encodeString(state.String()), encodeString(binlogplayer.MessageTruncate(message)), vr.id)\n\t// If we're batching a transaction, then include the state update\n\t// in the current transaction batch.\n\tif vr.dbClient.InTransaction && vr.dbClient.maxBatchSize > 0 {\n\t\tvr.dbClient.AddQueryToTrxBatch(query)\n\t} else { // Otherwise, send it down the wire\n\t\tif _, err := vr.dbClient.ExecuteFetch(query, 1); err != nil {\n\t\t\treturn fmt.Errorf(\"could not set state: %v: %v\", query, err)\n\t\t}\n\t}\n\tif state == vr.state {\n\t\treturn nil\n\t}\n\tinsertLog(vr.dbClient, LogStateChange, vr.id, state.String(), message)\n\tvr.state = state\n\n\treturn nil\n}\n\nfunc encodeString(in string) string {\n\treturn sqltypes.EncodeStringSQL(in)\n}\n\nfunc (vr *vreplicator) getSettingFKCheck() error {\n\tqr, err := vr.dbClient.Execute(\"select @@foreign_key_checks\")\n\tif err != nil {","sourceCodeStart":570,"sourceCodeEnd":606,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vttablet/tabletmanager/vreplication/vreplicator.go#L570-L606","documentation":"setState persists the stream state (Copying, Running, Error, etc.) via an UPDATE on _vt.vreplication. When not batched inside a transaction and the ExecuteFetch fails, this error wraps the query and MySQL error. A stream whose state cannot be persisted cannot make reliable progress, so it typically halts.","triggerScenarios":"The state UPDATE is sent outside an open transaction batch and MySQL returns an error: row gone (concurrent workflow cancel), read-only mysqld, dead connection, lock timeout. Called by runBlp, play, updatePos, applyEvent and copy-phase code.","commonSituations":"Concurrent vtctldclient workflow cancellation deleting the row; MySQL failover/read-only mode; network blips between tablet and mysqld; lock contention on the _vt.vreplication row.","solutions":["Inspect the wrapped MySQL error: 1290 -> clear read_only/super_read_only; 1146/1032 -> the row was deleted, so re-create or re-register the workflow.","Check tablet-to-MySQL connectivity and failover logs around the failure time.","If workflows were cancelled concurrently, do not force the old stream — re-run MoveTables/Reshard to start a fresh stream.","Retry by restarting the workflow; vreplication streams are resumable from their recorded position."],"exampleFix":"// before: state update fails on read-only target during a failover\n// after: ensure target writable before resuming the stream\nSET GLOBAL read_only = OFF;\n// then restart the workflow:\n// vtctldclient Workflow --keyspace=customer start <workflow>","handlingStrategy":"retry","validationCode":"// Before resuming a stream, verify target writability and row presence:\n// SELECT id, state FROM _vt.vreplication WHERE id = <id>;\n// SHOW GLOBAL VARIABLES LIKE 'super_read_only';","typeGuard":null,"tryCatchPattern":"if _, err := vr.dbClient.ExecuteFetch(query, 1); err != nil {\n\tswitch mysqlErr.Number() {\n\tcase 1290:\n\t\t// target read-only: clear then retry state update\n\tcase 1146, 1032:\n\t\t// row deleted: workflow cancelled elsewhere, do not retry\n\tdefault:\n\t\t// transient: retry with backoff\n\t}\n}","preventionTips":["Sequence maintenance: make target writable before starting/resuming workflows.","Serialize workflow cancel/start operations via vtctldclient.","Watch for failovers and restart streams afterwards.","Keep _vt.vreplication free of manual edits that can race state updates."],"tags":["vreplication","mysql","state-update"],"backgroundTag":"vreplication-state-update-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}