{"record":{"id":"557c40c424343dac","repo":"thanos-io/thanos","slug":"remove-tombstones","errorCode":null,"errorMessage":"remove tombstones","messagePattern":"remove tombstones","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/compact/compact.go","lineNumber":1306,"sourceCode":"\t}\n\tcg.compactions.Inc()\n\tif overlappingBlocks {\n\t\tcg.verticalCompactions.Inc()\n\t}\n\tcompIDStrings := make([]string, 0, len(compIDs))\n\tfor _, compID := range compIDs {\n\t\tcompIDStrings = append(compIDStrings, compID.String())\n\t}\n\tcompIDStrs := fmt.Sprintf(\"%v\", compIDStrings)\n\tlevel.Info(cg.logger).Log(\"msg\", \"compacted blocks\", \"new\", compIDStrs,\n\t\t\"duration\", time.Since(begin), \"duration_ms\", time.Since(begin).Milliseconds(), \"overlapping_blocks\", overlappingBlocks, \"blocks\", sourceBlockStr)\n\n\tfor _, compID := range compIDs {\n\t\tbdir := filepath.Join(dir, compID.String())\n\t\tindex := filepath.Join(bdir, block.IndexFilename)\n\n\t\tif err := os.Remove(filepath.Join(bdir, \"tombstones\")); err != nil {\n\t\t\treturn false, nil, errors.Wrap(err, \"remove tombstones\")\n\t\t}\n\n\t\tnewMeta, err := metadata.ReadFromDir(bdir)\n\t\tif err != nil {\n\t\t\treturn false, nil, errors.Wrap(err, \"read new meta\")\n\t\t}\n\n\t\tvar stats block.HealthStats\n\t\t// Ensure the output block is valid.\n\t\terr = tracing.DoInSpanWithErr(ctx, \"compaction_verify_index\", func(ctx context.Context) error {\n\t\t\tstats, err = block.GatherIndexHealthStats(ctx, cg.logger, index, newMeta.MinTime, newMeta.MaxTime)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn stats.AnyErr()\n\t\t})\n\t\tif !cg.acceptMalformedIndex && err != nil {\n\t\t\treturn false, nil, halt(errors.Wrapf(err, \"invalid result block %s\", bdir))","sourceCodeStart":1288,"sourceCodeEnd":1324,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/compact/compact.go#L1288-L1324","documentation":"Wraps os.Remove of the tombstone file inside a freshly compacted result block. The compactor deletes TSDB-internal tombstones before validating the output block; if removal fails (permissions, unexpected file state) the result block cannot be verified and the compaction is retried.","triggerScenarios":"Thrown at pkg/compact/compact.go:1306 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Check filesystem permissions on the block dir","Free disk/repair filesystem errors","Remove tombstones manually if the block is otherwise valid"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}