{"record":{"id":"4b0db9687951dfb0","repo":"dgraph-io/badger","slug":"error-during-flatten-in-streamwriter-w","errorCode":null,"errorMessage":"error during flatten in StreamWriter: %w","messagePattern":"error during flatten in StreamWriter: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"stream_writer.go","lineNumber":121,"sourceCode":"\n\tisEmptyDB := true\n\tfor _, level := range sw.db.Levels() {\n\t\tif level.NumTables > 0 {\n\t\t\tsw.prevLevel = level.Level\n\t\t\tisEmptyDB = false\n\t\t\tbreak\n\t\t}\n\t}\n\tif isEmptyDB {\n\t\t// If DB is empty, we should allow doing incremental stream write.\n\t\treturn nil\n\t}\n\tif sw.prevLevel == 0 {\n\t\t// It seems that data is present in all levels from Lmax to L0. If we call flatten\n\t\t// on the tree, all the data will go to Lmax. All the levels above will be empty\n\t\t// after flatten call. Now, we should be able to use incremental stream writer again.\n\t\tif err := sw.db.Flatten(3); err != nil {\n\t\t\treturn fmt.Errorf(\"error during flatten in StreamWriter: %w\", err)\n\t\t}\n\t\tsw.prevLevel = len(sw.db.Levels()) - 1\n\t}\n\treturn nil\n}\n\n// Write writes KVList to DB. Each KV within the list contains the stream id which StreamWriter\n// would use to demux the writes. Write is thread safe and can be called concurrently by multiple\n// goroutines.\nfunc (sw *StreamWriter) Write(buf *z.Buffer) error {\n\tif buf.LenNoPadding() == 0 {\n\t\treturn nil\n\t}\n\n\t// closedStreams keeps track of all streams which are going to be marked as done. We are\n\t// keeping track of all streams so that we can close them at the end, after inserting all\n\t// the valid kvs.\n\tclosedStreams := make(map[uint32]struct{})","sourceCodeStart":103,"sourceCodeEnd":139,"githubUrl":"https://github.com/dgraph-io/badger/blob/2a001d466f6b71a917319a1db41f99860e16e269/stream_writer.go#L103-L139","documentation":"When preparing an incremental StreamWriter, the library detects that data spans all levels down to L0 (prevLevel==0) and calls db.Flatten(3) to compact everything to the max level before resuming incremental writes. If that flatten/compaction operation fails, the underlying error is wrapped with this message and returned.","triggerScenarios":"PrepareIncremental() on a DB where tables are present from Lmax up to L0 and the LSM tree cannot be flattened because compaction fails (e.g., disk full, I/O errors, table corruption).","commonSituations":"Disk out of space or slow/failing storage during compaction; corrupted SST files from an earlier crash; running PrepareIncremental on a DB in a degraded/compacting state.","solutions":["Inspect the wrapped %w cause with errors.Unwrap/errors.Is and fix it (e.g., free disk space, repair the DB)","Free disk capacity and ensure the compaction workers can run, then retry PrepareIncremental()","If corruption is suspected, run badger repair or restore from the last good backup before retrying"],"exampleFix":"// before\nerr := sw.PrepareIncremental() // \"error during flatten in StreamWriter: ...\"\n// after\nerr := sw.PrepareIncremental()\nif err != nil {\n    cause := errors.Unwrap(err) // inspect flatten failure, e.g. disk full\n    // free space / repair db, then retry\n}","handlingStrategy":"retry","validationCode":"// Ensure ample free disk space and no other compaction-stopping condition before PrepareIncremental.\nif !hasFreeDiskSpace(lsmDir, minRequiredBytes) { return errors.New(\"insufficient disk space for flatten\") }","typeGuard":null,"tryCatchPattern":"err := sw.PrepareIncremental()\nif err != nil {\n    var cause error\n    for e := err; e != nil; e = errors.Unwrap(e) { cause = e }\n    // inspect cause (e.g. disk full / I/O error), remediate, then retry once\n}","preventionTips":["Free disk space and monitor disk usage before running flatten-heavy operations","Unwrap and log the wrapped cause to distinguish transient I/O errors from corruption","If corruption is suspected, repair the DB or restore from a good backup before retrying"],"tags":["badger","streamwriter","compaction","flatten"],"backgroundTag":"compaction-failed","analyzedSha":"2a001d466f6b71a917319a1db41f99860e16e269","analyzedAt":"2026-09-05T13:00:02.264Z","contentChangedAt":"2026-09-05T13:00:02.264Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}