{"record":{"id":"dc38b95249066a16","repo":"temporalio/temporal","slug":"corrupted-history-event-batch-empty-events","errorCode":null,"errorMessage":"corrupted history event batch, empty events","messagePattern":"corrupted history event batch, empty events","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"common/persistence/history_manager.go","lineNumber":32,"sourceCode":"\t\"go.temporal.io/server/common\"\n\t\"go.temporal.io/server/common/log/tag\"\n\t\"go.temporal.io/server/common/primitives/timestamp\"\n\t\"go.temporal.io/server/common/softassert\"\n)\n\nconst (\n\tdefaultLastNodeID        = common.FirstEventID - 1\n\tdefaultLastTransactionID = int64(0)\n\n\t// TrimHistoryBranch will only dump metadata, relatively cheap\n\ttrimHistoryBranchPageSize = 1000\n\tdataLossMsg               = \"Potential data loss\"\n)\n\nvar (\n\terrNonContiguousEventID = errors.New(\"corrupted history event batch, eventID is not contiguous\")\n\terrWrongVersion         = errors.New(\"corrupted history event batch, wrong version and IDs\")\n\terrEmptyEvents          = errors.New(\"corrupted history event batch, empty events\")\n)\n\nvar _ ExecutionManager = (*executionManagerImpl)(nil)\n\n// ForkHistoryBranch forks a new branch from a old branch\nfunc (m *executionManagerImpl) ForkHistoryBranch(\n\tctx context.Context,\n\trequest *ForkHistoryBranchRequest,\n) (*ForkHistoryBranchResponse, error) {\n\n\tif request.ForkNodeID <= 1 {\n\t\treturn nil, &InvalidPersistenceRequestError{\n\t\t\tMsg: \"ForkNodeID must be > 1\",\n\t\t}\n\t}\n\n\tforkBranch, err := m.GetHistoryBranchUtil().ParseHistoryBranchInfo(request.ForkBranchToken)\n\tif err != nil {","sourceCodeStart":14,"sourceCodeEnd":50,"githubUrl":"https://github.com/temporalio/temporal/blob/bde624efd13fbd3843654058db6d9c716166318b/common/persistence/history_manager.go#L14-L50","documentation":"errEmptyEvents is a sentinel in common/persistence/history_manager.go:32 returned when a history event batch fetched from persistence deserializes into an empty event list. A stored history batch must contain at least one event; an empty batch means the stored blob is corrupt or was written incorrectly. Like errWrongVersion, it is raised via softassert.UnexpectedDataLoss, logging a 'Potential data loss' warning, from readHistoryBranch and readHistoryBranchReverse.","triggerScenarios":"Reading a history branch (ReadHistoryBranch / ReadHistoryBranchReverse) where one of the returned data blobs passes deserialization but yields len(events) == 0.","commonSituations":"A batch row that was written empty due to a producer bug; truncation of serialized blob contents by storage-layer issues; manual DB cleanup that emptied rows; encoding/serializer mismatches yielding zero events.","solutions":["Inspect the offending history blob row for the workflow/branch and restore it from backup if empty","Check for recent writes by the history service that may have persisted an empty batch (serializer or event-list bug)","Run ValidateBatch / readHistoryBranch on the shard to enumerate all empty batches for repair","Escalate as potential data loss; correlate with the data-loss tags emitted by the soft-assert log"],"exampleFix":"// history_manager.go:939\n// before (corrupt blob):\nif len(events) == 0 {\n    return nil, nil, nil, nil, dataSize, softassert.UnexpectedDataLoss(m.logger, dataLossMsg, errEmptyEvents, dataLossTags(errEmptyEvents)...)\n}\n// after (caller-side handling):\nhistory, _, _, token, _, err := mgr.ReadHistoryBranch(req)\nif err != nil && strings.Contains(err.Error(), \"empty events\") {\n    logger.Error(\"empty history batch detected; workflow history needs repair\", tag.WorkflowID(req.RunID))\n}","handlingStrategy":"validation","validationCode":"events, err := serializer.DeserializeEvents(blob)\nif err == nil && len(events) == 0 {\n    // would trigger errEmptyEvents on read: repair blob first\n}","typeGuard":"func isEmptyEventsErr(err error) bool {\n    return err != nil && strings.Contains(err.Error(), \"corrupted history event batch, empty events\")\n}","tryCatchPattern":"res, err := mgr.ReadHistoryBranchReverse(ctx, req)\nif err != nil {\n    if isEmptyEventsErr(err) {\n        logger.Critical(\"empty history batch: potential data loss\", tag.Error(err))\n        return err // not retryable\n    }\n    return err\n}","preventionTips":["Guard the producer against persisting empty event lists (reject len(events)==0 at write time)","Alert on softassert data-loss logs in history service","Validate blobs after storage migrations or restore operations","Pin serializer versions so blobs decode consistently"],"tags":["go","persistence","data-loss","history-corruption"],"backgroundTag":"history-batch-corruption","analyzedSha":"bde624efd13fbd3843654058db6d9c716166318b","analyzedAt":"2026-09-01T07:18:39.080Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}