temporalio/temporal · error
unable to upsert workflow execution: %w
Error message
unable to upsert workflow execution: %w
What it means
ReplaceIntoVisibility performs an upsert (INSERT ... ON DUPLICATE KEY UPDATE) into the executions visibility table as the first statement of its transaction. Failure here is wrapped as 'unable to upsert workflow execution' and aborts the whole visibility write. It wraps whatever MySQL error occurred — constraints, schema mismatch, dead connection.
Source
Thrown at common/persistence/sql/sqlplugin/mysql/visibility.go:158
db, err := mdb.handle.DB()
if err != nil {
return nil, err
}
tx, err := db.BeginTxx(ctx, nil)
if err != nil {
return nil, err
}
defer func() {
err := tx.Rollback()
// If the error is sql.ErrTxDone, it means the transaction already closed, so ignore error.
if err != nil && !errors.Is(err, sql.ErrTxDone) {
// Transaction rollback error should never happen, unless db connection was lost.
retError = fmt.Errorf("transaction rollback failed: %w", retError)
}
}()
result, err = tx.NamedExecContext(ctx, templateUpsertWorkflowExecution, finalRow)
if err != nil {
return nil, fmt.Errorf("unable to upsert workflow execution: %w", err)
}
_, err = tx.NamedExecContext(ctx, templateUpsertCustomSearchAttributes, finalRow)
if err != nil {
return nil, fmt.Errorf("unable to upsert custom search attributes: %w", err)
}
_, err = tx.NamedExecContext(ctx, templateUpsertChasmSearchAttributes, finalRow)
if err != nil {
return nil, fmt.Errorf("unable to upsert chasm search attributes: %w", err)
}
err = tx.Commit()
if err != nil {
return nil, err
}
return result, nil
}
// DeleteFromVisibility deletes a row from visibility table if it exist
func (mdb *db) DeleteFromVisibility(View on GitHub (pinned to bde624efd1)
Solutions
- Read the wrapped MySQL error: lock wait timeout vs data truncation vs schema errors require different fixes.
- Apply visibility schema migrations to match the server version.
- For lock timeouts, check for long transactions/contended rows on visibility_workflow_executions (SHOW ENGINE INNODB STATUS, innodb_lock_wait_timeout).
- Check column size limits against the row values (RunID, WorkflowType, search attribute blobs).
- Verify DB connectivity if the cause is a driver error.
Example fix
// before: schema behind server // after temporal-sql-tool -plugin mysql -ep $DB_HOST -u $DB_USER -p $DB_PWD update-schema -d temporal_visibility
Defensive patterns
Strategy: try-catch
Validate before calling
// Go: preflight schema + connectivity check
var v string
if err := visDB.GetContext(ctx, &v, "SELECT version_ref FROM schema_version"); err != nil {
logger.Fatal("cannot read visibility schema version; check DB connectivity/migrations")
} Try / catch
if err := db.ReplaceIntoVisibility(ctx, row); err != nil {
if strings.Contains(err.Error(), "unable to upsert workflow execution") {
var mysqlErr *mysql.MySQLError
if errors.As(err, &mysqlErr) && mysqlErr.Number == 1205 {
logger.Warn("lock wait timeout on visibility upsert", tag.Error(err))
}
return convertPersistenceErr(err)
}
return err
} Prevention
- Run update-schema on the visibility store after every server upgrade.
- Watch for lock contention on hot visibility rows; raise innodb_lock_wait_timeout only after investigating.
- Keep column data (run IDs, types, SAs) within schema size limits.
- Alert on persistence error metrics for the visibility store.
When it happens
Trigger: Calling ReplaceIntoVisibility (workflow close/update) where the upsert into visibility_workflow_executions fails: lock wait timeout on a contended row, data-too-long on a column, unknown column from stale schema, or connection loss.
Common situations: High update rate on the same visibility row causing lock contention/timeouts (e.g. many updates to one workflow); visibility schema behind the server version; oversized payload from very long run IDs or search attributes.
Related errors
- unable to upsert chasm search attributes: %w
- unable to insert workflow execution: %w
- unable to insert custom search attributes: %w
- unable to insert chasm search attributes: %w
- unable to upsert custom search attributes: %w
AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01).
Data as JSON: /api/errors/573a9004e71d2675.
Report an issue: GitHub.