juicedata/juicefs · error

failed to unmarshal footer: %w

Error message

failed to unmarshal footer: %w

What it means

After successfully reading h.Len bytes of footer payload from the backup file, BakFooter.Unmarshal attempts to proto.Unmarshal them into a pb.Footer message. This error means the bytes are not a valid protobuf Footer encoding — the data is corrupt, wrong, or the length prefix pointed at the wrong bytes.

Source

Thrown at pkg/meta/backup.go:216

func (h *BakFooter) Unmarshal(r io.ReadSeeker) error {
	lenSize := int64(unsafe.Sizeof(h.Len))
	_, _ = r.Seek(-lenSize, io.SeekEnd)

	data := make([]byte, lenSize)
	if n, err := r.Read(data); err != nil && n != int(lenSize) {
		return fmt.Errorf("failed to read footer length: err %w, read len %d, expect len %d", err, n, lenSize)
	}

	h.Len = binary.BigEndian.Uint64(data)
	_, _ = r.Seek(-int64(h.Len)-lenSize, io.SeekEnd)
	data = make([]byte, h.Len)
	if n, err := r.Read(data); err != nil && n != int(h.Len) {
		return fmt.Errorf("failed to read footer: err %w, read len %d, expect len %d", err, n, h.Len)
	}

	h.Msg = &pb.Footer{}
	if err := proto.Unmarshal(data, h.Msg); err != nil {
		return fmt.Errorf("failed to unmarshal footer: %w", err)
	}
	return nil
}

type BakSegment struct {
	typ uint32
	len uint64
	val proto.Message
}

func (s *BakSegment) Name() string {
	if name, ok := SegType2Name[int(s.typ)]; ok {
		return name
	}
	return fmt.Sprintf("type-%d", s.typ)
}

func (s *BakSegment) String() string {

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Regenerate the backup with `juicefs dump` from the live metadata engine and retry — corrupt protobuf bytes cannot be repaired in place.
2. Check whether the dump that produced the file completed and was fsynced; discard backups written by processes that crashed mid-dump.
3. Verify the file is actually a JuiceFS metadata backup (correct magic/size) and not a different file accidentally passed to ReadFooter.
4. Compare versions: if the backup came from a different JuiceFS release, use a compatible client version to load it.
5. Inspect the wrapped err from proto.Unmarshal — a size/resource error hints the h.Len trailer is bogus and the file structure is shifted.

Example fix

// before: no integrity check before loading footer
data := downloadBackup("s3://bucket/meta.backup")
footer, err := format.ReadFooter(bytes.NewReader(data))

// after: verify integrity first
data := downloadBackup("s3://bucket/meta.backup")
if sha256.Sum256(data) != expectedChecksum {
    return fmt.Errorf("backup corrupt; re-run juicefs dump")
}
footer, err := format.ReadFooter(bytes.NewReader(data))

if sha256sum(backupPath) != expectedChecksum {
    return errors.New("backup content mismatch; regenerate with juicefs dump")
}
// sanity: last 8 bytes (footer len) must fit in file
sz := fileSize(backupPath)
flen := readUint64BEAt(backupPath, sz-8)
if 8+uint64(flen) > sz { return errors.New("footer length exceeds file size; file corrupt") }

func footerLenFitsInFile(f io.ReaderAt) bool {
    buf := make([]byte, 8)
    if _, err := f.ReadAt(buf, fileSize-8); err != nil { return false }
    flen := binary.BigEndian.Uint64(buf)
    return 8+flen <= fileSize
}

if err := footer.Unmarshal(f); err != nil {
    var perr *proto.UnmarshalError
    if errors.As(err, &perr) || strings.Contains(err.Error(), "failed to unmarshal footer") {
        return fmt.Errorf("backup footer corrupt — regenerate the backup: %w", err)
    }
    return err
}

When it happens

Trigger: Reading a backup file whose footer region was overwritten or partially written (crash during dump before fsync); byte-shifted files (footer length parsed from wrong offset); passing a non-backup file to ReadFooter; incompatible pb.Footer schema/round-trip through an older writer writing a different format.

Common situations: Manual edits or binary patches to a backup file; rsync/S3 download that altered or truncated content without changing the length trailer; attempting to restore a backup produced by a much older/newer JuiceFS version with a different footer layout; corrupted volume on which the backup resides.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

• failed to read footer: err %w, read len %d, expect len %d
• unknown message type %d
• failed to find message %s's type: %v
• failed to marshal segment %s: %v
• failed to marshal footer: %w

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/69b39141407e669c. Report an issue: GitHub.