rqlite/rqlite · error

failed to write decompressed data to file: %v

Error message

failed to write decompressed data to file: %v

What it means

The Dechunker reassembles gzip-compressed chunks into a single file. This error is wrapped by WriteChunk when io.Copy fails while writing decompressed bytes from the gzip reader into the underlying temp file. It indicates the disk write failed mid-stream, so the reassembled file is incomplete or unwritable.

Source

Thrown at command/chunking/dechunker.go:58

	} else if d.streamID != chunk.StreamId {
		return false, fmt.Errorf("chunk has unexpected stream ID: expected %s but got %s", d.streamID, chunk.StreamId)
	}

	if chunk.SequenceNum != d.seqNum+1 {
		return false, fmt.Errorf("chunks received out of order: expected %d but got %d", d.seqNum+1, chunk.SequenceNum)
	}
	d.seqNum = chunk.SequenceNum

	if chunk.Data != nil {
		buf := bytes.NewBuffer(chunk.Data)
		gzw, err := gzip.NewReader(buf)
		if err != nil {
			return false, fmt.Errorf("failed to create gzip reader: %v", err)
		}
		defer gzw.Close()

		if _, err := io.Copy(d.file, gzw); err != nil {
			return false, fmt.Errorf("failed to write decompressed data to file: %v", err)
		}
	}

	return chunk.IsLast, nil
}

// Close closes the Dechunker and returns the file path containing the reassembled data.
func (d *Dechunker) Close() (string, error) {
	if err := d.file.Close(); err != nil {
		return "", fmt.Errorf("failed to close file: %v", err)
	}
	return d.filePath, nil
}

// DechunkerManager manages Dechunkers.
type DechunkerManager struct {
	dir string
	mu  sync.Mutex

View on GitHub (pinned to 7586a4d1bd)

Solutions

1. Free disk space on the directory backing the Dechunker (usually the node's data dir) and retry the upload from the first chunk
2. Check filesystem permissions and mount health (df, dmesg for I/O errors)
3. Inspect the wrapped inner error (%v) to distinguish ENOSPC from EIO or a closed-file bug
4. If using containers, verify the volume quota and increase it

Example fix

// caller before/after: check space before a large chunked upload
// before
dechunker.WriteChunk(chunk)
// after
if free, _ := diskFree(dataDir); free < expectedSize {
    return fmt.Errorf("insufficient disk space in %s: need ~%d bytes", dataDir, expectedSize)
}
err := decomposer.Write(rl, dechunkerFn) // then handle error, delete partial file

// Go: check free space on the dechunker directory before large transfers
func hasFreeSpace(dir string, need uint64) bool {
    var st syscall.Statfs_t
    if err := syscall.Statfs(dir, &st); err != nil { return false }
    return uint64(st.Bavail)*uint64(st.Bsize) >= need
}

if _, err := dechunker.WriteChunk(chunk); err != nil {
    if strings.Contains(err.Error(), "failed to write decompressed data") {
        // clean up partial temp file, free space or fix fs, restart chunked transfer from chunk 0
    }
    return err
}

When it happens

Trigger: Calling Dechunker.WriteChunk with a chunk whose decompression succeeds but whose destination file write fails: disk full, I/O error, file descriptor closed/invalid, or underlying storage failure during io.Copy(d.file, gzw).

Common situations: Uploading very large compressed blobs (backups, restores) that exceed available disk space on the node's temp/data directory; read-only or full filesystem; container disk quota exceeded mid-transfer.

Related errors

• failed to close file: %v
• short write
• failed to create gzip reader: %v
• failed to create dechunker manager: %s
• failed to read file %s: %w

AI-assisted analysis of rqlite/rqlite@7586a4d1bd (2026-09-03). Data as JSON: /api/errors/8364a8adfc49b442. Report an issue: GitHub.