valyala/fasthttp · error

error when compressing file %q: %w

Error message

error when compressing file %q: %w

What it means

This is the in-memory variant of compression (newCompressedFSFileCache): fasthttp compresses the file into a buffer and then wraps any error from the compression pipeline with the original file path. Unlike the on-disk variant there is no temp file; the error reflects a compression or buffer-write failure before the result is cached.

Source

Thrown at fs.go:1826

	case "gzip":
		zw := acquireStacklessGzipWriter(w, CompressDefaultCompression)
		_, err = copyZeroAlloc(zw, f)
		if errf := zw.Flush(); err == nil {
			err = errf
		}
		releaseStacklessGzipWriter(zw, CompressDefaultCompression)
	case "zstd":
		zw := acquireStacklessZstdWriter(w, CompressZstdDefault)
		_, err = copyZeroAlloc(zw, f)
		if errf := zw.Flush(); err == nil {
			err = errf
		}
		releaseStacklessZstdWriter(zw, CompressZstdDefault)
	}
	defer func() { _ = f.Close() }()

	if err != nil {
		return nil, fmt.Errorf("error when compressing file %q: %w", filePath, err)
	}

	seeker, ok := f.(io.Seeker)
	if !ok {
		return nil, errors.New("seek is not implemented")
	}
	if _, err = seeker.Seek(0, io.SeekStart); err != nil {
		return nil, err
	}

	ext := fileExtension(fileInfo.Name(), false, h.compressedFileSuffixes[fileEncoding])
	contentType := mime.TypeByExtension(ext)
	if contentType == "" {
		data, err := readFileHeader(f, false, fileEncoding)
		if err != nil {
			return nil, fmt.Errorf("cannot read header of the file %q: %w", fileInfo.Name(), err)
		}
		contentType = http.DetectContentType(data)

View on GitHub (pinned to c96f600972)

Solutions

1. Limit in-memory compression to reasonably sized files; let the on-disk path handle large ones, or disable Compress for big assets.
2. Increase the container/process memory limit if the cause is allocation failure.
3. Pre-compress assets at build time so fasthttp serves existing compressed files instead of compressing in memory.
4. Retry: transient memory pressure often resolves after load drops.

Example fix

// before: compressing huge files in memory
fs := &fasthttp.FS{Root: "/srv/bigfiles", Compress: true}
// after: only enable compression for small asset roots; serve big files uncompressed/streamed
fs := &fasthttp.FS{Root: "/srv/assets", Compress: true} // keep bigfiles in a non-compressed handler

if fi, err := os.Stat(path); err == nil && fi.Size() > 100<<20 {
    // serve without in-memory compression (disable Compress for this root)
}

if strings.Contains(err.Error(), "error when compressing file ") {
    // fall back to serving uncompressed or to the on-disk compression path
}

When it happens

Trigger: First request for a file with an Accept-Encoding matching FS.Compress levels when the compressor (gzip/brotli/zstd) fails writing to its in-memory buffer — typically an out-of-memory/allocation failure for very large files or a writer-level error.

Common situations: Compressing huge (multi-GB) files in memory causing allocation failures under memory limits (cgroup OOM pressure); serving large dynamic-ish files with FS.Compress enabled instead of streaming compression.

Related errors

• seek is not implemented
• fasthttp: unsupported content-encoding
• cannot create temporary file for %q: %w
• error when compressing file %q to %q: %w
• cannot change modification time to %v for tmp file %q: %v

AI-assisted analysis of valyala/fasthttp@c96f600972 (2026-08-31). Data as JSON: /api/errors/2ae736a4b0955829. Report an issue: GitHub.