temporalio/temporal · critical

corrupted history event batch, empty events

Error message

corrupted history event batch, empty events

What it means

errEmptyEvents is a sentinel in common/persistence/history_manager.go:32 returned when a history event batch fetched from persistence deserializes into an empty event list. A stored history batch must contain at least one event; an empty batch means the stored blob is corrupt or was written incorrectly. Like errWrongVersion, it is raised via softassert.UnexpectedDataLoss, logging a 'Potential data loss' warning, from readHistoryBranch and readHistoryBranchReverse.

Source

Thrown at common/persistence/history_manager.go:32

	"go.temporal.io/server/common"
	"go.temporal.io/server/common/log/tag"
	"go.temporal.io/server/common/primitives/timestamp"
	"go.temporal.io/server/common/softassert"
)

const (
	defaultLastNodeID        = common.FirstEventID - 1
	defaultLastTransactionID = int64(0)

	// TrimHistoryBranch will only dump metadata, relatively cheap
	trimHistoryBranchPageSize = 1000
	dataLossMsg               = "Potential data loss"
)

var (
	errNonContiguousEventID = errors.New("corrupted history event batch, eventID is not contiguous")
	errWrongVersion         = errors.New("corrupted history event batch, wrong version and IDs")
	errEmptyEvents          = errors.New("corrupted history event batch, empty events")
)

var _ ExecutionManager = (*executionManagerImpl)(nil)

// ForkHistoryBranch forks a new branch from a old branch
func (m *executionManagerImpl) ForkHistoryBranch(
	ctx context.Context,
	request *ForkHistoryBranchRequest,
) (*ForkHistoryBranchResponse, error) {

	if request.ForkNodeID <= 1 {
		return nil, &InvalidPersistenceRequestError{
			Msg: "ForkNodeID must be > 1",
		}
	}

	forkBranch, err := m.GetHistoryBranchUtil().ParseHistoryBranchInfo(request.ForkBranchToken)
	if err != nil {

View on GitHub (pinned to bde624efd1)

Solutions

1. Inspect the offending history blob row for the workflow/branch and restore it from backup if empty
2. Check for recent writes by the history service that may have persisted an empty batch (serializer or event-list bug)
3. Run ValidateBatch / readHistoryBranch on the shard to enumerate all empty batches for repair
4. Escalate as potential data loss; correlate with the data-loss tags emitted by the soft-assert log

Example fix

// history_manager.go:939
// before (corrupt blob):
if len(events) == 0 {
    return nil, nil, nil, nil, dataSize, softassert.UnexpectedDataLoss(m.logger, dataLossMsg, errEmptyEvents, dataLossTags(errEmptyEvents)...)
}
// after (caller-side handling):
history, _, _, token, _, err := mgr.ReadHistoryBranch(req)
if err != nil && strings.Contains(err.Error(), "empty events") {
    logger.Error("empty history batch detected; workflow history needs repair", tag.WorkflowID(req.RunID))
}

events, err := serializer.DeserializeEvents(blob)
if err == nil && len(events) == 0 {
    // would trigger errEmptyEvents on read: repair blob first
}

func isEmptyEventsErr(err error) bool {
    return err != nil && strings.Contains(err.Error(), "corrupted history event batch, empty events")
}

res, err := mgr.ReadHistoryBranchReverse(ctx, req)
if err != nil {
    if isEmptyEventsErr(err) {
        logger.Critical("empty history batch: potential data loss", tag.Error(err))
        return err // not retryable
    }
    return err
}

When it happens

Trigger: Reading a history branch (ReadHistoryBranch / ReadHistoryBranchReverse) where one of the returned data blobs passes deserialization but yields len(events) == 0.

Common situations: A batch row that was written empty due to a producer bug; truncation of serialized blob contents by storage-layer issues; manual DB cleanup that emptied rows; encoding/serializer mismatches yielding zero events.

Related errors

• corrupted history event batch, wrong version and IDs
• page size to read history tasks must be positive
• history task from queue has nil blob
• enqueue task request task is nil
• queue already exists

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/dc38b95249066a16. Report an issue: GitHub.