juicedata/juicefs · error

failed to write segment data %s: err %w, write len %d, expec

Error message

failed to write segment data %s: err %w, write len %d, expect len %d

What it means

After protobuf-marshaling and writing the length prefix, BakSegment.Marshal writes the segment's serialized payload bytes with w.Write. This error is thrown when w.Write returns an error or writes fewer bytes than the payload length (partial write). It indicates the destination stream failed or accepted only part of the segment data, so the backup footer would be corrupt.

Source

Thrown at pkg/meta/backup.go:343

func (s *BakSegment) Marshal(w io.Writer) (int, error) {
	if s == nil || s.val == nil {
		return 0, fmt.Errorf("segment %s is nil", s)
	}

	if err := binary.Write(w, binary.BigEndian, s.typ); err != nil {
		return 0, fmt.Errorf("failed to write segment type %s : %w", s, err)
	}
	data, err := proto.Marshal(s.val)
	if err != nil {
		return 0, fmt.Errorf("failed to marshal segment message %s : %w", s, err)
	}
	s.len = uint64(len(data))
	if err := binary.Write(w, binary.BigEndian, s.len); err != nil {
		return 0, fmt.Errorf("failed to write segment length %s: %w", s, err)
	}

	if n, err := w.Write(data); err != nil || n != len(data) {
		return 0, fmt.Errorf("failed to write segment data %s: err %w, write len %d, expect len %d", s, err, n, len(data))
	}

	return binary.Size(s.typ) + binary.Size(s.len) + len(data), nil
}

func (s *BakSegment) Unmarshal(r io.Reader) error {
	if err := binary.Read(r, binary.BigEndian, &s.typ); err != nil {
		return fmt.Errorf("failed to read segment type: %v", err)
	}

	if s.typ == BakEOS {
		return errBakEOF
	}

	if err := binary.Read(r, binary.BigEndian, &s.len); err != nil {
		return fmt.Errorf("failed to read segment %s length: %v", s, err)
	}
	data := make([]byte, s.len)

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Inspect the wrapped error (err %w) in the message to find the concrete I/O cause
2. Check disk space and file writability on the backup destination
3. If using a custom io.Writer, fix it to return a non-nil error on short writes or use io.Copy/io.WriteFull semantics
4. Retry the backup after resolving the storage/stream problem

Example fix

// before (custom writer)
func (w *myWriter) Write(p []byte) (int, error) { return w.n, nil } // short write, nil error
// after
func (w *myWriter) Write(p []byte) (int, error) {
    n, err := w.inner.Write(p)
    if err != nil { return n, err }
    if n < len(p) { return n, io.ErrShortWrite }
    return n, nil
}

if fi, err := outStat(); err == nil && fi.Size() >= expectedFooterSize { /* destination plausible */ }

if _, err := seg.Marshal(w); err != nil {
    if errors.Is(err, io.ErrShortWrite) || errors.Is(err, syscall.ENOSPC) {
        return fmt.Errorf("backup destination full or short write: %w", err)
    }
    return err
}

When it happens

Trigger: Calling writeFooter (invoking BakSegment.Marshal) where w.Write returns err != nil or n != len(data): disk full mid-write, closed/broken stream, or a short write from a custom io.Writer implementation.

Common situations: Filesystem filling up while dumping a large metadata backup; writing to a network connection or pipe that broke; custom writer that violates io.Writer contract by returning n < len(p) with nil error.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• failed to write EOS: err %w, write len %d, expect len 4
• failed to write footer data: err %w, write len %d, expect le
• failed to write footer length: err %w, write len %d, expect
• failed to write segment type %s : %w
• failed to write segment length %s: %w

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/a9b9d8d26fd8c72f. Report an issue: GitHub.