juicedata/juicefs · error

failed to marshal segment %s: %v

Error message

failed to marshal segment %s: %v

What it means

writeSegment serializes a backup segment into the output stream via protobuf Marshal. If marshalling fails (nil descriptor, required field unset, oversized message), the segment cannot be written and DumpMetaV2 aborts with this wrapped error.

Source

Thrown at pkg/meta/backup.go:126

	return &BakFormat{
		Footer: &BakFooter{
			Msg: &pb.Footer{
				Magic:   BakMagic,
				Version: BakVersion,
				Infos:   make(map[string]*pb.Footer_SegInfo),
			},
		},
	}
}

func (f *BakFormat) writeSegment(w io.Writer, seg *BakSegment) error {
	if seg == nil {
		return nil
	}

	n, err := seg.Marshal(w)
	if err != nil {
		return fmt.Errorf("failed to marshal segment %s: %v", seg, err)
	}

	name := seg.Name()
	info, ok := f.Footer.Msg.Infos[name]
	if !ok {
		info = &pb.Footer_SegInfo{Offset: []uint64{}, Num: 0}
		f.Footer.Msg.Infos[name] = info
	}

	info.Offset = append(info.Offset, f.Pos)
	info.Num += seg.num()
	f.Pos += uint64(n)
	return nil
}

func (f *BakFormat) ReadSegment(r io.Reader) (*BakSegment, error) {
	seg := &BakSegment{}
	if err := seg.Unmarshal(r); err != nil {

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Retry the dump after upgrading all clients to a compatible version
2. Identify the offending segment (it is printed in the error) and inspect the corresponding metadata record
3. Report/repair the corrupt record in the metadata engine, then re-run the dump

err := dumpMetaV2(ctx, w)
if err != nil {
    if strings.Contains(err.Error(), "failed to marshal segment") {
        log.Printf("corrupt metadata record during dump: %v; check engine health", err)
    }
    return err
}

When it happens

Trigger: Calling DumpMetaV2 on a metadata engine whose record fails proto.Marshal — e.g. a segment message with fields that exceed protobuf limits or a corrupt in-memory record read from Redis/SQL/KV.

Common situations: Dumping metadata from an engine with records written by incompatible client versions containing unexpected data; extremely large single records (huge xattr/ACL) hitting protobuf size limits.

Related errors

• failed to marshal footer: %w
• failed to marshal segment message %s : %w
• unknown message type %d
• failed to find message %s's type: %v
• failed to read footer: err %w, read len %d, expect len %d

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/60d42a23045a1135. Report an issue: GitHub.