{"record":{"id":"8d2f5996a9b349f0","repo":"thanos-io/thanos","slug":"populatewithdelchunkseriesiterator-unexpected-emp","errorCode":null,"errorMessage":"populateWithDelChunkSeriesIterator: unexpected empty chunk found while rewriting chunk","messagePattern":"populateWithDelChunkSeriesIterator: unexpected empty chunk found while rewriting chunk","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/compactv2/modifiers.go","lineNumber":334,"sourceCode":"\t\treturn true\n\t}\n\n\t// Re-encode the chunk if iterator is provider. This means that it has some samples to be deleted or chunk is opened.\n\tnewChunk := chunkenc.NewXORChunk()\n\tapp, err := newChunk.Appender()\n\tif err != nil {\n\t\tp.err = err\n\t\treturn false\n\t}\n\n\tif p.currDelIter.Next() == chunkenc.ValNone {\n\t\tif err := p.currDelIter.Err(); err != nil {\n\t\t\tp.err = errors.Wrap(err, \"iterate chunk while re-encoding\")\n\t\t\treturn false\n\t\t}\n\n\t\t// Empty chunk, this should not happen, as we assume full deletions being filtered before this iterator.\n\t\tp.err = errors.Wrap(err, \"populateWithDelChunkSeriesIterator: unexpected empty chunk found while rewriting chunk\")\n\t\treturn false\n\t}\n\n\tt, v := p.currDelIter.At()\n\tp.curr.MinTime = t\n\tapp.Append(t, v)\n\n\tfor p.currDelIter.Next() != chunkenc.ValNone {\n\t\tt, v = p.currDelIter.At()\n\t\tapp.Append(t, v)\n\t}\n\tif err := p.currDelIter.Err(); err != nil {\n\t\tp.err = errors.Wrap(err, \"iterate chunk while re-encoding\")\n\t\treturn false\n\t}\n\n\tp.curr.Chunk = newChunk\n\tp.curr.MaxTime = t","sourceCodeStart":316,"sourceCodeEnd":352,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/compactv2/modifiers.go#L316-L352","documentation":"While re-encoding a chunk under deletion modifiers, the iterator reached ValNone with no iterator error and an empty chunk was produced. The code explicitly treats this as an invariant violation: fully-deleted chunks should have been filtered out before this iterator, so an empty rewritten chunk means internal state is wrong.","triggerScenarios":"currDelIter.Next() == ValNone, currDelIter.Err() == nil, and err happens to be nil when wrapped — i.e. a chunk whose samples were all deleted (fully tombstoned) still reached the rewriting iterator instead of being dropped earlier.","commonSituations":"Deletion/time-range modifier selecting a window covering an entire chunk but the pre-filter not removing that chunk; block with tombstones not previously cleaned; edge-case chunk boundary conditions with modifiers.","solutions":["Ensure fully-deleted chunks are filtered before populating (run vertical compaction/tombstone cleanup, or check the deletion filtering in the modifier pipeline)","Check the modifier's deletion ranges — make sure minTime/maxTime handling drops chunks fully covered by deletions","Report/inspect as a bug if it reproduces with a specific modifier range; work around by adjusting the deletion window","Rerun compaction after tombstone cleanup (e.g. after a clean compaction level removes tombstones)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Drop chunks fully covered by the deletion range before rewriting\nchunks = filterOutFullyDeleted(chunks, delMinTime, delMaxTime)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always filter fully tombstoned chunks before the rewriting iterator","Run tombstone cleanup compaction before applying deletion modifiers","Test modifiers with deletion windows exactly covering whole chunks"],"tags":["go","tsdb","chunks","invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}