{"record":{"id":"1b3643a57bf2ab5c","repo":"dgraph-io/badger","slug":"request-size-offset-d-is-bigger-than-maximum-offs","errorCode":null,"errorMessage":"Request size offset %d is bigger than maximum offset %d","messagePattern":"Request size offset (.+?) is bigger than maximum offset (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"value.go","lineNumber":791,"sourceCode":"\tcurlf.lock.RUnlock()\n\treturn err\n}\n\nfunc (vlog *valueLog) woffset() uint32 {\n\treturn vlog.writableLogOffset.Load()\n}\n\n// validateWrites will check whether the given requests can fit into 4GB vlog file.\n// NOTE: 4GB is the maximum size we can create for vlog because value pointer offset is of type\n// uint32. If we create more than 4GB, it will overflow uint32. So, limiting the size to 4GB.\nfunc (vlog *valueLog) validateWrites(reqs []*request) error {\n\tvlogOffset := uint64(vlog.woffset())\n\tfor _, req := range reqs {\n\t\t// calculate size of the request.\n\t\tsize := estimateRequestSize(req)\n\t\testimatedVlogOffset := vlogOffset + size\n\t\tif estimatedVlogOffset > uint64(maxVlogFileSize) {\n\t\t\treturn fmt.Errorf(\"Request size offset %d is bigger than maximum offset %d\",\n\t\t\t\testimatedVlogOffset, maxVlogFileSize)\n\t\t}\n\n\t\tif estimatedVlogOffset >= uint64(vlog.opt.ValueLogFileSize) {\n\t\t\t// We'll create a new vlog file if the estimated offset is greater or equal to\n\t\t\t// max vlog size. So, resetting the vlogOffset.\n\t\t\tvlogOffset = 0\n\t\t\tcontinue\n\t\t}\n\t\t// Estimated vlog offset will become current vlog offset if the vlog is not rotated.\n\t\tvlogOffset = estimatedVlogOffset\n\t}\n\treturn nil\n}\n\n// estimateRequestSize returns the size that needed to be written for the given request.\nfunc estimateRequestSize(req *request) uint64 {\n\tsize := uint64(0)","sourceCodeStart":773,"sourceCodeEnd":809,"githubUrl":"https://github.com/dgraph-io/badger/blob/2a001d466f6b71a917319a1db41f99860e16e269/value.go#L773-L809","documentation":"validateWrites estimates each incoming request's size and rejects any request whose cumulative value-log offset would exceed maxVlogFileSize (the hard maximum vlog file size). Such a request could never fit in any value log file, so badger fails it before writing anything.","triggerScenarios":"db.Set/db.Write/txn.Commit with a single entry or batch whose estimated size exceeds maxVlogFileSize — commonly when opt.ValueLogFileSize was set above the allowed maximum, or a single value is enormous (hundreds of MB+).","commonSituations":"Users raising ValueLogFileSize beyond the hard cap and pushing one giant blob; storing large files/blobs in badger instead of object storage; building huge write batches that exceed the limit in aggregate.","solutions":["Shrink or chunk the large value so no single entry exceeds maxVlogFileSize, or store the blob externally and keep a reference key in badger.","Check opt.ValueLogFileSize: it must be <= maxVlogFileSize (default max ~2GB); set it within the supported range.","Split large batches into smaller requests; estimateRequestSize counts the whole request.","For genuinely huge values, use badger's Stream framework or an external store (S3/filesystem) with badger holding metadata."],"exampleFix":"// before\nopt.ValueLogFileSize = 4 << 30 // exceeds maxVlogFileSize\ndb.Set(key, hugeValue) // Request size offset X is bigger than maximum offset Y\n// after\nopt.ValueLogFileSize = 1 << 30 // within max\nif len(hugeValue) > 100<<20 {\n    ref := storeExternal(hugeValue)\n    db.Set(key, ref)\n} else {\n    db.Set(key, hugeValue)\n}","handlingStrategy":"validation","validationCode":"// Go: validate request size before writing\nconst maxVlogFileSize = 2 << 30 // badger hard cap\nfunc validateEntrySize(k, v []byte) error {\n    est := len(k) + len(v) + 64 // key + value + header overhead\n    if uint64(est) > uint64(maxVlogFileSize) {\n        return fmt.Errorf(\"entry too large: %d bytes\", est)\n    }\n    return nil\n}","typeGuard":"func isRequestTooLarge(err error) bool {\n    return err != nil && strings.Contains(err.Error(), \"bigger than maximum offset\")\n}","tryCatchPattern":"err := db.Set(key, value)\nif err != nil && isRequestTooLarge(err) {\n    ref, err2 := storeExternal(value)\n    if err2 != nil { return err2 }\n    return db.Set(key, ref)\n}","preventionTips":["Keep opt.ValueLogFileSize within badger's supported maximum.","Chunk or externalize values larger than ~100MB.","Split large write batches into smaller requests.","Use object storage for blobs; badger for metadata/references."],"tags":["badger","value-log","request-size","configuration"],"backgroundTag":"request-exceeds-max-vlog-size","analyzedSha":"2a001d466f6b71a917319a1db41f99860e16e269","analyzedAt":"2026-09-05T13:00:02.264Z","contentChangedAt":"2026-09-05T13:00:02.264Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}