{"record":{"id":"13d3c21a6deace9b","repo":"vitessio/vitess","slug":"err-error-readtwopcinflight-failure","errorCode":null,"errorMessage":"err.Error() (ReadTwopcInflight failure)","messagePattern":"err\\.Error\\(\\) \\(ReadTwopcInflight failure\\)","errorType":"http","errorClass":null,"httpStatus":500,"severity":"error","filePath":"go/vt/vttablet/tabletserver/twopcz.go","lineNumber":160,"sourceCode":"\tswitch action {\n\tcase \"Discard\", \"Rollback\":\n\t\terr = txe.RollbackPrepared(dtid, 0)\n\tcase \"Commit\":\n\t\terr = txe.CommitPrepared(dtid)\n\tcase \"Conclude\":\n\t\terr = txe.ConcludeTransaction(dtid)\n\t}\n\tvar msg string\n\tif action != \"\" {\n\t\tif err != nil {\n\t\t\tmsg = fmt.Sprintf(\"%s(%s): %v\", r.FormValue(\"Action\"), dtid, err)\n\t\t} else {\n\t\t\tmsg = fmt.Sprintf(\"%s(%s): completed.\", r.FormValue(\"Action\"), dtid)\n\t\t}\n\t}\n\tdistributed, prepared, failed, err := txe.ReadTwopcInflight()\n\tif err != nil {\n\t\thttp.Error(w, err.Error(), http.StatusInternalServerError)\n\t\treturn\n\t}\n\tformat := r.FormValue(\"format\")\n\tif format == \"json\" {\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")\n\t\tjs, err := json.Marshal(struct {\n\t\t\tDistributed      []*tx.DistributedTx\n\t\t\tPrepared, Failed []*tx.PreparedTx\n\t\t}{\n\t\t\tDistributed: distributed,\n\t\t\tPrepared:    prepared,\n\t\t\tFailed:      failed,\n\t\t})\n\t\tif err != nil {\n\t\t\thttp.Error(w, err.Error(), http.StatusInternalServerError)\n\t\t\treturn\n\t\t}\n\t\tw.Header().Set(\"Content-Type\", \"application/json\")","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vttablet/tabletserver/twopcz.go#L142-L178","documentation":"This is the HTTP 500 returned by the /twopcz debug UI handler when txe.ReadTwopcInflight() fails to enumerate in-flight distributed transactions. ReadTwopcInflight queries the underlying transaction store (and MySQL) for unresolved prepared/distributed transactions; a failure here means the dtid transaction metadata could not be read (storage/backend error). The raw err.Error() text is written as the HTTP 500 body.","triggerScenarios":"An HTTP GET to the vttablet /twopcz endpoint (acl DEBUGGING required) where txe.ReadTwopcInflight() returns an error — e.g. the dtid table in the transaction metadata storage is unreadable/corrupt, the backing MySQL query fails, or the transaction store is not available.","commonSituations":"Investigating unresolved distributed transactions after a crash and the transaction metadata storage is damaged; probing /twopcz on a tablet whose MySQL backend is down; scripting /twopcz during a failover when the transaction store is temporarily inaccessible.","solutions":["Read the error body — it is the raw ReadTwopcInflight error — and fix the underlying storage/MySQL issue it names.","Check vttablet logs for the full error with context around the /twopcz request.","Verify the MySQL backend is healthy and the transaction metadata table (dtid store) exists and is readable; run a manual SELECT against it to confirm.","Retry after restoring MySQL connectivity; if metadata is corrupt, follow the 2PC recovery runbook (ResolveTransactions/ConcludeTransaction) carefully to avoid data loss."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Confirm MySQL backend reachable before calling /twopcz\nif err := mysqlPing(ctx, tabletDSN); err != nil {\n    return fmt.Errorf(\"backend down, /twopcz will fail: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"resp, err := http.Get(tabletURL + \"/twopcz\")\nif err != nil {\n    return err\n}\nif resp.StatusCode == http.StatusInternalServerError {\n    body, _ := io.ReadAll(resp.Body)\n    return fmt.Errorf(\"ReadTwopcInflight failed: %s\", body)\n}","preventionTips":["Monitor the dtid/transaction metadata storage for corruption or unavailability.","Resolve in-flight distributed transactions promptly instead of accumulating unbounded 2PC state.","Only operate /twopcz during controlled recovery windows with MySQL confirmed healthy.","Back up and test recovery of the transaction metadata store."],"tags":["vttablet","twopc","transactions","debug-endpoint"],"backgroundTag":"transaction-metadata-read-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}