errUnrefData

errUnrefData

Error message

archive/tar: sparse file contains unreferenced data

What it means

errUnrefData (unexported) is returned when the dense data stream contains MORE bytes than the sparse map references — i.e. there are bytes in the stream that no sparse data fragment covers, so they would be silently dropped. Like errMissData it signals an internally inconsistent sparse entry and is returned by both the reader and writer paths.

Source

Thrown at src/archive/tar/common.go:40

	"strconv"
	"strings"
	"time"
)

// BUG: Use of the Uid and Gid fields in Header could overflow on 32-bit
// architectures. If a large value is encountered when decoding, the result
// stored in Header will be the truncated version.

var tarinsecurepath = godebug.New("tarinsecurepath")

var (
	ErrHeader          = errors.New("archive/tar: invalid tar header")
	ErrWriteTooLong    = errors.New("archive/tar: write too long")
	ErrFieldTooLong    = errors.New("archive/tar: header field too long")
	ErrWriteAfterClose = errors.New("archive/tar: write after close")
	ErrInsecurePath    = errors.New("archive/tar: insecure file path")
	errMissData        = errors.New("archive/tar: sparse file references non-existent data")
	errUnrefData       = errors.New("archive/tar: sparse file contains unreferenced data")
	errWriteHole       = errors.New("archive/tar: write non-NUL byte in sparse hole")
	errSparseTooLong   = errors.New("archive/tar: sparse map too long")
)

type headerError []string

func (he headerError) Error() string {
	const prefix = "archive/tar: cannot encode header"
	var ss []string
	for _, s := range he {
		if s != "" {
			ss = append(ss, s)
		}
	}
	if len(ss) == 0 {
		return prefix
	}
	return fmt.Sprintf("%s: %v", prefix, strings.Join(ss, "; and "))

View on GitHub (pinned to b6b368adc5)

Solutions

Re-fetch the archive and validate checksum.
When writing sparse entries, ensure the total bytes you push equals sum(sparse fragments) and matches Header.Size minus the hole regions.
Quarantine the entry on errUnrefData; do not retry — the sparse map and data disagree.
For untrusted input, log and skip rather than attempting repair.

Example fix

// before
hdr := &tar.Header{
  Size: 20,
  Sparses: []tar.SparseEntry{{Offset: 0, NumBytes: 5}}, // under-declared
tw.WriteHeader(hdr)
tw.Write(make([]byte, 20)) // throws errUnrefData

// after
hdr := &tar.Header{
  Size: 20,
  Sparses: []tar.SparseEntry{{Offset: 0, NumBytes: 20}},
}
tw.WriteHeader(hdr)
tw.Write(make([]byte, 20))

Defensive patterns

Strategy: validation

Validate before calling

var sum int64
for _, s := range hdr.Sparses { sum += s.NumBytes }
if sum != hdr.Size {
  return fmt.Errorf("sparse fragments sum %d != Size %d (unref data risk)", sum, hdr.Size)
}

Try / catch

if err != nil && strings.Contains(err.Error(), "sparse file contains unreferenced data") {
  log.Printf("corrupt sparse entry %q; skipping", hdr.Name)
  continue
}

Prevention

Match the bytes you write to the sum of sparse data fragments exactly.
Validate sparse archives from untrusted sources via checksum before extraction.
Treat any unexported tar sparse error as corruption and skip the entry.

When it happens

Trigger: Reading a sparse tar entry whose dense data length exceeds the sum of sparse data fragments; writing a sparse entry where the bytes pushed exceed the sum of declared fragment lengths (extra trailing bytes).

Common situations: Corrupted sparse archive; truncated or extended download; hand-built sparse Header whose Sparses array underestimates the data to be written.

Related errors

AI-assisted analysis of golang/go@b6b368adc5 (2026-08-12). Data as JSON: /api/errors/30a8fb8dfbb92625. Report an issue: GitHub.