temporalio/temporal · critical

corrupted history event batch, eventID is not contiguous

Error message

corrupted history event batch, eventID is not contiguous

What it means

errNonContiguousEventID is declared in common/persistence/history_manager.go and returned by readHistoryBranch, readHistoryBranchReverse, and ValidateBatch when a loaded history event batch has a gap in event IDs — events must be contiguous within a batch/branch. This is treated as corrupted history data, indicating a storage or replication problem, and guards against building workflows on incomplete history.

Source

Thrown at common/persistence/history_manager.go:30

	"go.temporal.io/api/serviceerror"
	persistencespb "go.temporal.io/server/api/persistence/v1"
	"go.temporal.io/server/common"
	"go.temporal.io/server/common/log/tag"
	"go.temporal.io/server/common/primitives/timestamp"
	"go.temporal.io/server/common/softassert"
)

const (
	defaultLastNodeID        = common.FirstEventID - 1
	defaultLastTransactionID = int64(0)

	// TrimHistoryBranch will only dump metadata, relatively cheap
	trimHistoryBranchPageSize = 1000
	dataLossMsg               = "Potential data loss"
)

var (
	errNonContiguousEventID = errors.New("corrupted history event batch, eventID is not contiguous")
	errWrongVersion         = errors.New("corrupted history event batch, wrong version and IDs")
	errEmptyEvents          = errors.New("corrupted history event batch, empty events")
)

var _ ExecutionManager = (*executionManagerImpl)(nil)

// ForkHistoryBranch forks a new branch from a old branch
func (m *executionManagerImpl) ForkHistoryBranch(
	ctx context.Context,
	request *ForkHistoryBranchRequest,
) (*ForkHistoryBranchResponse, error) {

	if request.ForkNodeID <= 1 {
		return nil, &InvalidPersistenceRequestError{
			Msg: "ForkNodeID must be > 1",
		}
	}

View on GitHub (pinned to bde624efd1)

Solutions

1. Verify persistence integrity for the affected workflow's history rows/blobs and restore from backup if corrupted
2. Compare the branch token/branchID and version against the events table to find the missing range
3. Check history service logs for the correlated dataLossMsg ('Potential data loss') to identify the exact batch
4. If caused by a failed migration/restore, re-run the restore and confirm event ID continuity

func batchesContiguous(events []*historypb.HistoryEvent) bool {
    for i := 1; i < len(events); i++ {
        if events[i].EventId != events[i-1].EventId+1 { return false }
    }
    return true
}

if len(batch.Events) == 0 || batch.Events[0].EventId != expectedFirstEventID { return errNonContiguousEventID }

events, err := executionManager.ReadHistoryBranch(ctx, req)
if errors.Is(err, persistence.ErrNonContiguousEventID) {
    // alert, quarantine the workflow, and investigate persistence/restore integrity
}

When it happens

Trigger: Reading a history branch whose next/first event IDs leave a gap between batches; ValidateBatch detecting batch event IDs that do not run contiguously; database rows or Blobstore data for a branch that were partially written or deleted.

Common situations: Underlying persistence (DB rows or blobs) corrupted or partially deleted; version/branch metadata pointing at the wrong shard of events; bugs in history archival/trimming; disaster-recovery restores that missed some events.

Related errors

• history task from queue has nil blob
• membershipExpiry duration should be atleast 1 second
• membership upserts require all fields
• corrupted history event batch, wrong version and IDs
• corrupted history event batch, empty events

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/a98cd553c50d847e. Report an issue: GitHub.