{"record":{"id":"2420f46c8e8f5ad7","repo":"temporalio/temporal","slug":"transaction-rollback-failed-w","errorCode":null,"errorMessage":"transaction rollback failed: %w","messagePattern":"transaction rollback failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"common/persistence/sql/sqlplugin/mysql/visibility.go","lineNumber":109,"sourceCode":"\tfinalRow := mdb.prepareRowForDB(row)\n\tdefer func() {\n\t\tretError = mdb.handle.ConvertError(retError)\n\t}()\n\tdb, err := mdb.handle.DB()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\ttx, err := db.BeginTxx(ctx, nil)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer func() {\n\t\terr := tx.Rollback()\n\t\t// If the error is sql.ErrTxDone, it means the transaction already closed, so ignore error.\n\t\tif err != nil && !errors.Is(err, sql.ErrTxDone) {\n\t\t\t// Transaction rollback error should never happen, unless db connection was lost.\n\t\t\tretError = fmt.Errorf(\"transaction rollback failed: %w\", retError)\n\t\t}\n\t}()\n\tresult, err = tx.NamedExecContext(ctx, templateInsertWorkflowExecution, finalRow)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"unable to insert workflow execution: %w\", err)\n\t}\n\t_, err = tx.NamedExecContext(ctx, templateInsertCustomSearchAttributes, finalRow)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"unable to insert custom search attributes: %w\", err)\n\t}\n\t_, err = tx.NamedExecContext(ctx, templateInsertChasmSearchAttributes, finalRow)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"unable to insert chasm search attributes: %w\", err)\n\t}\n\terr = tx.Commit()\n\tif err != nil {\n\t\treturn nil, err\n\t}","sourceCodeStart":91,"sourceCodeEnd":127,"githubUrl":"https://github.com/temporalio/temporal/blob/bde624efd13fbd3843654058db6d9c716166318b/common/persistence/sql/sqlplugin/mysql/visibility.go#L91-L127","documentation":"In InsertIntoVisibility, a deferred rollback runs for every exit path; if tx.Rollback() fails with anything other than sql.ErrTxDone (which just means the tx was already committed/closed), the function's return error is overwritten with 'transaction rollback failed: %w'. Per the in-code comment this should never happen unless the DB connection was lost. Note the wrapped variable is retError, so the original failure (if any) is carried inside the wrap chain.","triggerScenarios":"InsertIntoVisibility: deferred tx.Rollback() returns a non-ErrTxDone error — the MySQL connection backing the transaction dropped or was killed between BeginTxx and the deferred rollback.","commonSituations":"MySQL server restart or failover mid-insert; connection killed by wait_timeout or proxy idle timeout; network partition between history service and MySQL; DBA killing a long-running connection.","solutions":["Check the wrapped cause (%w chain) with errors.Unwrap / %v logging to see 'bad connection' or 'driver: bad connection'.","Confirm MySQL connectivity and inspect server logs around the incident time for restarts or killed connections.","Tune MySQL wait_timeout / proxy idle timeouts to exceed worst-case transaction durations.","Retry the visibility insert — a fresh transaction on a healthy pooled connection should succeed.","If frequent, enable driver-side connection liveness checks (e.g. connMaxLifetime shorter than server timeout)."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := db.InsertIntoVisibility(ctx, row); err != nil {\n    if strings.Contains(err.Error(), \"transaction rollback failed\") {\n        logger.Warn(\"connection lost during visibility insert tx; retrying\", tag.Error(err))\n        return retryOp(ctx) // bounded retry with backoff\n    }\n    return err\n}","preventionTips":["Set MySQL wait_timeout and proxy idle timeouts well above maximum transaction duration.","Configure driver MaxConnLifetime shorter than server-side idle timeout.","Monitor MySQL restarts and connection kill events; alert on spikes.","Keep visibility transactions short; avoid heavy work inside the persistence call path."],"tags":["mysql","transaction","rollback","database"],"backgroundTag":"transaction-rollback-failed","analyzedSha":"bde624efd13fbd3843654058db6d9c716166318b","analyzedAt":"2026-09-01T07:18:39.080Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}