{"record":{"id":"0b16882ad91b7ba9","repo":"temporalio/temporal","slug":"queue-slice-get-task-from-iterator-doesn-t-belong","errorCode":null,"errorMessage":"Queue slice get task from iterator doesn't belong to its range, range: %v, task key %v","messagePattern":"Queue slice get task from iterator doesn't belong to its range, range: (.+?), task key (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"service/history/queues/slice.go","lineNumber":392,"sourceCode":"\t}()\n\n\texecutables := make([]Executable, 0, batchSize)\n\tfor len(executables) < batchSize && len(s.iterators) != 0 {\n\t\tif s.iterators[0].HasNext() {\n\t\t\ttask, err := s.iterators[0].Next()\n\t\t\tif err != nil {\n\t\t\t\ts.iterators[0] = s.iterators[0].Remaining()\n\t\t\t\tif len(executables) != 0 {\n\t\t\t\t\t// NOTE: we must return the executables here\n\t\t\t\t\t// MoreTasks() will return true so queue reader will try to load again\n\t\t\t\t\treturn executables, nil\n\t\t\t\t}\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\ttaskKey := task.GetKey()\n\t\t\tif !s.scope.Range.ContainsKey(taskKey) {\n\t\t\t\tpanic(fmt.Sprintf(\"Queue slice get task from iterator doesn't belong to its range, range: %v, task key %v\",\n\t\t\t\t\ts.scope.Range, taskKey))\n\t\t\t}\n\n\t\t\tif !s.scope.Predicate.Test(task) {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\texecutable := s.executableFactory.NewExecutable(task, readerID)\n\t\t\ts.add(executable)\n\t\t\texecutables = append(executables, executable)\n\t\t} else {\n\t\t\ts.iterators = s.iterators[1:]\n\t\t}\n\t}\n\n\treturn executables, nil\n}\n","sourceCodeStart":374,"sourceCodeEnd":410,"githubUrl":"https://github.com/temporalio/temporal/blob/bde624efd13fbd3843654058db6d9c716166318b/service/history/queues/slice.go#L374-L410","documentation":"Queue slices are history-service task-queue abstractions that own a key Range; every task returned by a persistence iterator must fall inside the slice's scope range. This panic fires in SelectTasks when the DB iterator yields a task whose key lies outside that range — a broken internal invariant, since iterators are constructed from the slice's own range. It indicates stale iterators after scope changes or corrupted range/scope bookkeeping.","triggerScenarios":"SliceImpl.SelectTasks iterating after SplitByPredicate/ShrinkScope/Clear mutated the scope while an old iterator with the pre-split range is still in s.iterators; a persistence iterator implementation returning tasks beyond its declared range; race where the slice scope was updated concurrently with SelectTasks.","commonSituations":"Seen only in temporal-server history service internals; typically after a bug in queue split/merge logic, a version-upgrade mixing old and new slice logic, or a custom tasks.Iterator implementation that ignores its range bounds.","solutions":["Verify all iterators held by the slice are rebuilt whenever scope.Range changes (Clear/ShrinkScope already do this; check custom paths)","Ensure slices are not used concurrently from multiple goroutines without the queue's lock","Check the persistence iterator (paginationFnProvider) returns only keys within the range it was created for","Capture the panic message's range and task key and report to Temporal with shard/task details if reproducible"],"exampleFix":"// before: iterator created once at slice construction, reused after scope change\niterators := []Iterator{NewIterator(s.paginationFnProvider, oldRange)}\n// after: rebuild iterators whenever the scope range changes (as Clear does)\ns.ShrinkScope()\ns.iterators = []Iterator{NewIterator(s.paginationFnProvider, s.scope.Range)}","handlingStrategy":"validation","validationCode":"// Before relying on slice behavior, confirm any custom iterator stays in range:\nif !slice.Range().ContainsKey(task.GetKey()) {\n  return fmt.Errorf(\"iterator yielded task %v outside slice range %v\", task.GetKey(), slice.Range())\n}","typeGuard":"func taskInRange(r queues.Range, task tasks.Task) bool {\n  return r.ContainsKey(task.GetKey())\n}","tryCatchPattern":"// This is a panic, not an error. Wrap queue operations with recover only at\n// goroutine boundaries to convert an invariant bug into a logged failure:\nfunc safeSelect(slice queues.Slice) (executables []tasks.Executable, err error) {\n  defer func() { if r := recover(); r != nil { err = fmt.Errorf(\"queue slice panic: %v\", r) } }()\n  return slice.SelectTasks()\n}","preventionTips":["Rebuild iterators whenever a slice's scope/range changes","Never share queue slices across goroutines without the queue's lock","Keep custom Iterator implementations strictly within their declared Range","Report stock-build occurrences with range and task key to temporalio/temporal"],"tags":["go","history-service","task-queue","internal-invariant"],"backgroundTag":"queue-slice-range-violation","analyzedSha":"bde624efd13fbd3843654058db6d9c716166318b","analyzedAt":"2026-09-01T07:18:39.080Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}