{"record":{"id":"b372f7a13d52eb50","repo":"juicedata/juicefs","slug":"buffer-too-short-d-d","errorCode":null,"errorMessage":"buffer too short: %d < %d","messagePattern":"buffer too short: (.+?) < (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/compress/compress.go","lineNumber":57,"sourceCode":"func NewCompressor(algr string) Compressor {\n\talgr = strings.ToLower(algr)\n\tif algr == \"zstd\" {\n\t\treturn ZStandard{ZSTD_LEVEL}\n\t} else if algr == \"lz4\" {\n\t\treturn LZ4{}\n\t} else if algr == \"none\" || algr == \"\" {\n\t\treturn noOp{}\n\t}\n\treturn nil\n}\n\ntype noOp struct{}\n\nfunc (n noOp) Name() string            { return \"Noop\" }\nfunc (n noOp) CompressBound(l int) int { return l }\nfunc (n noOp) Compress(dst, src []byte) (int, error) {\n\tif len(dst) < len(src) {\n\t\treturn 0, fmt.Errorf(\"buffer too short: %d < %d\", len(dst), len(src))\n\t}\n\tcopy(dst, src)\n\treturn len(src), nil\n}\nfunc (n noOp) Decompress(dst, src []byte) (int, error) {\n\tif len(dst) < len(src) {\n\t\treturn 0, fmt.Errorf(\"buffer too short: %d < %d\", len(dst), len(src))\n\t}\n\tcopy(dst, src)\n\treturn len(src), nil\n}\n\n// ZStandard implements Compressor interface using zstd library\ntype ZStandard struct {\n\tlevel int\n}\n\n// Name returns name of the algorithm Zstd","sourceCodeStart":39,"sourceCodeEnd":75,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/compress/compress.go#L39-L75","documentation":"The Noop compressor's Compress copies src into dst unchanged and requires len(dst) >= len(src); if the destination buffer is too small it returns 'buffer too short: %d < %d'. It exists so callers that sized dst via CompressBound (which for Noop equals len(src)) can detect undersized buffers instead of truncating data.","triggerScenarios":"Calling noOp.Compress(dst, src) where len(dst) < len(src) — i.e. the caller allocated dst smaller than CompressBound(len(src)) or passed a nil/empty dst.","commonSituations":"Configuring compression 'none' in the volume format but writing code (or a caller like the chunk writer) that assumes compressed output is smaller and allocates dst as a fraction of input; tests passing wrong-size buffers.","solutions":["Allocate dst with at least compressor.CompressBound(len(src)) bytes before calling Compress.","For noOp specifically, ensure dst capacity equals len(src) (CompressBound(l) == l).","Check configuration: if volume compression is none, no shrinking occurs, so buffers must be input-sized."],"exampleFix":"// before\ndst := make([]byte, len(src)/2)\nn, err := comp.Compress(dst, src)\n\n// after\ndst := make([]byte, comp.CompressBound(len(src)))\nn, err := comp.Compress(dst, src)","handlingStrategy":"validation","validationCode":"if len(dst) < comp.CompressBound(len(src)) { dst = make([]byte, comp.CompressBound(len(src))) }","typeGuard":null,"tryCatchPattern":"n, err := comp.Compress(dst, src)\nif err != nil && strings.HasPrefix(err.Error(), \"buffer too short\") {\n    dst = make([]byte, comp.CompressBound(len(src)))\n    n, err = comp.Compress(dst, src)\n}","preventionTips":["Always size dst via CompressBound, never guess","Remember Noop compresses 1:1 — dst must equal input size","Add unit tests calling Compress with exactly-CompressBound buffers","Pool buffers with capacity >= max expected input"],"tags":["compression","buffer-size","noop"],"backgroundTag":"buffer-too-short","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}