benbjohnson/litestream · error

close decoder

Error message

close decoder: %w

What it means

After writing pages and truncating, applyLTXFile closes the LTX decoder. dec.Close() verifies the LTX file's trailing checksum over everything decoded; an error here means the transaction stream failed integrity verification even though the data decoded fine. Litestream must fail the apply because the source file cannot be trusted.

Solutions

1. Treat the remote LTX file as corrupt: remove/quarantine it and let the replica re-upload
2. Run `litestream reset <db>` to clear local state and re-sync from a known-good snapshot
3. Inspect the storage object (etag/size) against what was uploaded to confirm corruption
4. Ensure nothing mutates LTX files after creation, including lifecycle/lambda processing

Example fix

// before: ignoring decoder close error
dec.Close()
return f.Sync()
// after
if err := dec.Close(); err != nil {
    return fmt.Errorf("close decoder: %w", err) // checksum verification failed
}

// verify an LTX object end-to-end before applying
func verifyLTX(ctx context.Context, rc io.ReadCloser) error {
    dec := ltx.NewDecoder(rc)
    if err := dec.DecodeHeader(); err != nil { return err }
    buf := make([]byte, dec.Header().PageSize)
    for {
        var ph ltx.PageHeader
        if err := dec.DecodePage(&ph, buf); err == io.EOF { break } else if err != nil { return err }
    }
    return dec.Close() // checksum verification happens here
}

if err := r.applyLTXFile(ctx, f, info, pageSize); err != nil {
    if strings.Contains(err.Error(), "close decoder") {
        // checksum verification failed: quarantine object, reset state
        quarantineObject(info)
        exec.Command("litestream", "reset", dbPath).Run()
    }
    return err
}

When it happens

Trigger: dec.Close() returns non-nil: LTX checksum mismatch at end-of-stream, trailing bytes after the last page, or the reader rc returned fewer/more bytes than the header declared (e.g., storage returned a corrupted or partial object).

Common situations: Bit rot or partial multipart uploads in object storage, a proxy truncating the HTTP body, LTX file modified after creation (LTX files are immutable by contract), version mismatch between writer and reader.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• apply gap-fill ltx file
• apply ltx to hydrated file
• calc pos
• close ltx encoder
• decode database

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/5d1c5dc22599d861. Report an issue: GitHub.

Appendix: source

Thrown at replica.go:996

		off := int64(phdr.Pgno-1) * int64(pageSize)
		if _, err := f.WriteAt(data, off); err != nil {
			return fmt.Errorf("write page %d: %w", phdr.Pgno, err)
		}
	}

	if hdr.Commit > 0 {
		if err := f.Sync(); err != nil {
			return fmt.Errorf("sync before truncate: %w", err)
		}
		newSize := int64(hdr.Commit) * int64(pageSize)
		if err := f.Truncate(newSize); err != nil {
			return fmt.Errorf("truncate: %w", err)
		}
	}

	if err := dec.Close(); err != nil {
		return fmt.Errorf("close decoder: %w", err)
	}

	return f.Sync()
}

// fillFollowGap attempts to bridge a gap in level 0 files by searching
// higher compaction levels for a file that covers the missing TXID range.
func (r *Replica) fillFollowGap(ctx context.Context, f *os.File, afterTXID ltx.TXID, gapMinTXID ltx.TXID, pageSize uint32) (ltx.TXID, error) {
	currentTXID := afterTXID

	for level := 1; level < SnapshotLevel; level++ {
		itr, err := r.Client.LTXFiles(ctx, level, 0, false)
		if err != nil {
			return currentTXID, fmt.Errorf("list level %d ltx files: %w", level, err)
		}
		closeLevel := func(retErr error) (ltx.TXID, error) {
			if closeErr := itr.Close(); closeErr != nil {
				closeErr = fmt.Errorf("close level %d ltx iterator: %w", level, closeErr)

View on GitHub (pinned to 4ed7a308f6)