{"record":{"id":"aeb90b6b9f40ed2b","repo":"thanos-io/thanos","slug":"add-symbol","errorCode":null,"errorMessage":"add symbol","messagePattern":"add symbol","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/block/index.go","lineNumber":587,"sourceCode":"type seriesRepair struct {\n\tlset labels.Labels\n\tchks []chunks.Meta\n}\n\n// rewrite writes all data from the readers back into the writers while cleaning\n// up mis-ordered and duplicated chunks.\nfunc rewrite(\n\tctx context.Context,\n\tlogger log.Logger,\n\tindexr tsdb.IndexReader, chunkr tsdb.ChunkReader,\n\tindexw tsdb.IndexWriter, chunkw tsdb.ChunkWriter,\n\tmeta *metadata.Meta,\n\tignoreChkFns []ignoreFnType,\n) error {\n\tsymbols := indexr.Symbols()\n\tfor symbols.Next() {\n\t\tif err := indexw.AddSymbol(symbols.At()); err != nil {\n\t\t\treturn errors.Wrap(err, \"add symbol\")\n\t\t}\n\t}\n\tif symbols.Err() != nil {\n\t\treturn errors.Wrap(symbols.Err(), \"next symbol\")\n\t}\n\n\tkey, value := index.AllPostingsKey()\n\tall, err := indexr.Postings(ctx, key, value)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"postings\")\n\t}\n\tall = indexr.SortedPostings(all)\n\n\t// We fully rebuild the postings list index from merged series.\n\tvar (\n\t\tpostings = index.NewMemPostings()\n\t\tvalues   = map[string]stringset{}\n\t\ti        = storage.SeriesRef(0)","sourceCodeStart":569,"sourceCodeEnd":605,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/block/index.go#L569-L605","documentation":"This error wraps a failure from IndexWriter.AddSymbol() while block.Rewrite (pkg/block/index.go:587) copies all symbols from a source index reader to a new index file during block repair/compaction. AddSymbol fails when the underlying index file writer cannot serialize or flush the symbol table — typically disk I/O errors or a corrupted source symbol iterator. The 'add symbol' wrapper tells you the failure happened in the symbol-copying phase, not the postings or series phase.","triggerScenarios":"block.Rewrite() iterating indexr.Symbols() and calling indexw.AddSymbol(symbols.At()) when the destination index file cannot be written (disk full, I/O error, closed writer) or the symbol table serialization fails.","commonSituations":"Running promtool tsdb repair or block compaction on a disk that is full or failing; corrupted block directories after a crash; repair tooling copying blocks across filesystems with permission or space problems.","solutions":["Check disk space and filesystem health (df -h, dmesg) on the directory holding the block; free space or fix the disk and re-run Repair.","Verify the source block's index file is not truncated/corrupt; if it is, restore the block from backup or another replica instead of rewriting it.","Re-run the repair with a freshly opened destination writer; ensure the IndexWriter was not already closed before rewrite.","Inspect the wrapped inner error (errors.Wrap preserves cause) to pinpoint the underlying I/O failure."],"exampleFix":"// before: rewriting onto a nearly full disk\nerr := block.Rewrite(ctx, meta, blockDir, blockDir, indexr, chunkr, nil)\n// after: pre-check free space and source index readability before rewriting\nif err := checkDiskSpace(blockDir, requiredBytes); err != nil { return err }\nif err := validateIndexFile(filepath.Join(blockDirSrc, \"index\")); err != nil { return err }\nreturn block.Rewrite(ctx, meta, blockDirSrc, blockDirDst, indexr, chunkr, nil)","handlingStrategy":"try-catch","validationCode":"if freeDiskSpace(blockDir) < requiredBytes { return fmt.Errorf(\"insufficient disk space for block rewrite\") }\nif _, err := os.Stat(filepath.Join(blockDir, \"index\")); err != nil { return err }","typeGuard":"func isAddSymbolError(err error) bool { return err != nil && strings.Contains(err.Error(), \"add symbol\") }","tryCatchPattern":"if err := block.Rewrite(ctx, meta, src, dst, indexr, chunkr, nil); err != nil {\n    if strings.Contains(err.Error(), \"add symbol\") {\n        // check disk space / writer state, retry with fresh writer\n    }\n    return err\n}","preventionTips":["Monitor disk usage on the TSDB data directory before running repair/compaction","Keep the destination IndexWriter open only for the duration of rewrite; never reuse a closed writer","Always inspect the wrapped cause with errors.Unwrap to distinguish I/O failure from corruption"],"tags":["tsdb","index","repair","io-error"],"backgroundTag":"database-write-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}