{"record":{"id":"2ae736a4b0955829","repo":"valyala/fasthttp","slug":"error-when-compressing-file-q-w","errorCode":null,"errorMessage":"error when compressing file %q: %w","messagePattern":"error when compressing file %q: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"fs.go","lineNumber":1826,"sourceCode":"\tcase \"gzip\":\n\t\tzw := acquireStacklessGzipWriter(w, CompressDefaultCompression)\n\t\t_, err = copyZeroAlloc(zw, f)\n\t\tif errf := zw.Flush(); err == nil {\n\t\t\terr = errf\n\t\t}\n\t\treleaseStacklessGzipWriter(zw, CompressDefaultCompression)\n\tcase \"zstd\":\n\t\tzw := acquireStacklessZstdWriter(w, CompressZstdDefault)\n\t\t_, err = copyZeroAlloc(zw, f)\n\t\tif errf := zw.Flush(); err == nil {\n\t\t\terr = errf\n\t\t}\n\t\treleaseStacklessZstdWriter(zw, CompressZstdDefault)\n\t}\n\tdefer func() { _ = f.Close() }()\n\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error when compressing file %q: %w\", filePath, err)\n\t}\n\n\tseeker, ok := f.(io.Seeker)\n\tif !ok {\n\t\treturn nil, errors.New(\"seek is not implemented\")\n\t}\n\tif _, err = seeker.Seek(0, io.SeekStart); err != nil {\n\t\treturn nil, err\n\t}\n\n\text := fileExtension(fileInfo.Name(), false, h.compressedFileSuffixes[fileEncoding])\n\tcontentType := mime.TypeByExtension(ext)\n\tif contentType == \"\" {\n\t\tdata, err := readFileHeader(f, false, fileEncoding)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"cannot read header of the file %q: %w\", fileInfo.Name(), err)\n\t\t}\n\t\tcontentType = http.DetectContentType(data)","sourceCodeStart":1808,"sourceCodeEnd":1844,"githubUrl":"https://github.com/valyala/fasthttp/blob/c96f600972c6f4a7a30d664257b340ebe9d60124/fs.go#L1808-L1844","documentation":"This is the in-memory variant of compression (newCompressedFSFileCache): fasthttp compresses the file into a buffer and then wraps any error from the compression pipeline with the original file path. Unlike the on-disk variant there is no temp file; the error reflects a compression or buffer-write failure before the result is cached.","triggerScenarios":"First request for a file with an Accept-Encoding matching FS.Compress levels when the compressor (gzip/brotli/zstd) fails writing to its in-memory buffer — typically an out-of-memory/allocation failure for very large files or a writer-level error.","commonSituations":"Compressing huge (multi-GB) files in memory causing allocation failures under memory limits (cgroup OOM pressure); serving large dynamic-ish files with FS.Compress enabled instead of streaming compression.","solutions":["Limit in-memory compression to reasonably sized files; let the on-disk path handle large ones, or disable Compress for big assets.","Increase the container/process memory limit if the cause is allocation failure.","Pre-compress assets at build time so fasthttp serves existing compressed files instead of compressing in memory.","Retry: transient memory pressure often resolves after load drops."],"exampleFix":"// before: compressing huge files in memory\nfs := &fasthttp.FS{Root: \"/srv/bigfiles\", Compress: true}\n// after: only enable compression for small asset roots; serve big files uncompressed/streamed\nfs := &fasthttp.FS{Root: \"/srv/assets\", Compress: true} // keep bigfiles in a non-compressed handler","handlingStrategy":"fallback","validationCode":"if fi, err := os.Stat(path); err == nil && fi.Size() > 100<<20 {\n    // serve without in-memory compression (disable Compress for this root)\n}","typeGuard":null,"tryCatchPattern":"if strings.Contains(err.Error(), \"error when compressing file \") {\n    // fall back to serving uncompressed or to the on-disk compression path\n}","preventionTips":["Only enable in-memory compression for small-to-medium asset roots.","Pre-compress large assets at build time.","Set adequate memory limits for the server process."],"tags":["fasthttp","compression","memory","in-memory-cache"],"backgroundTag":"compression-write-failed","analyzedSha":"c96f600972c6f4a7a30d664257b340ebe9d60124","analyzedAt":"2026-08-31T22:48:28.265Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T05:18:18.240Z"}