{"record":{"id":"a9b9d8d26fd8c72f","repo":"juicedata/juicefs","slug":"failed-to-write-segment-data-s-err-w-write-len","errorCode":null,"errorMessage":"failed to write segment data %s: err %w, write len %d, expect len %d","messagePattern":"failed to write segment data (.+?): err %w, write len (.+?), expect len (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/meta/backup.go","lineNumber":343,"sourceCode":"func (s *BakSegment) Marshal(w io.Writer) (int, error) {\n\tif s == nil || s.val == nil {\n\t\treturn 0, fmt.Errorf(\"segment %s is nil\", s)\n\t}\n\n\tif err := binary.Write(w, binary.BigEndian, s.typ); err != nil {\n\t\treturn 0, fmt.Errorf(\"failed to write segment type %s : %w\", s, err)\n\t}\n\tdata, err := proto.Marshal(s.val)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"failed to marshal segment message %s : %w\", s, err)\n\t}\n\ts.len = uint64(len(data))\n\tif err := binary.Write(w, binary.BigEndian, s.len); err != nil {\n\t\treturn 0, fmt.Errorf(\"failed to write segment length %s: %w\", s, err)\n\t}\n\n\tif n, err := w.Write(data); err != nil || n != len(data) {\n\t\treturn 0, fmt.Errorf(\"failed to write segment data %s: err %w, write len %d, expect len %d\", s, err, n, len(data))\n\t}\n\n\treturn binary.Size(s.typ) + binary.Size(s.len) + len(data), nil\n}\n\nfunc (s *BakSegment) Unmarshal(r io.Reader) error {\n\tif err := binary.Read(r, binary.BigEndian, &s.typ); err != nil {\n\t\treturn fmt.Errorf(\"failed to read segment type: %v\", err)\n\t}\n\n\tif s.typ == BakEOS {\n\t\treturn errBakEOF\n\t}\n\n\tif err := binary.Read(r, binary.BigEndian, &s.len); err != nil {\n\t\treturn fmt.Errorf(\"failed to read segment %s length: %v\", s, err)\n\t}\n\tdata := make([]byte, s.len)","sourceCodeStart":325,"sourceCodeEnd":361,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/meta/backup.go#L325-L361","documentation":"After protobuf-marshaling and writing the length prefix, BakSegment.Marshal writes the segment's serialized payload bytes with w.Write. This error is thrown when w.Write returns an error or writes fewer bytes than the payload length (partial write). It indicates the destination stream failed or accepted only part of the segment data, so the backup footer would be corrupt.","triggerScenarios":"Calling writeFooter (invoking BakSegment.Marshal) where w.Write returns err != nil or n != len(data): disk full mid-write, closed/broken stream, or a short write from a custom io.Writer implementation.","commonSituations":"Filesystem filling up while dumping a large metadata backup; writing to a network connection or pipe that broke; custom writer that violates io.Writer contract by returning n < len(p) with nil error.","solutions":["Inspect the wrapped error (err %w) in the message to find the concrete I/O cause","Check disk space and file writability on the backup destination","If using a custom io.Writer, fix it to return a non-nil error on short writes or use io.Copy/io.WriteFull semantics","Retry the backup after resolving the storage/stream problem"],"exampleFix":"// before (custom writer)\nfunc (w *myWriter) Write(p []byte) (int, error) { return w.n, nil } // short write, nil error\n// after\nfunc (w *myWriter) Write(p []byte) (int, error) {\n    n, err := w.inner.Write(p)\n    if err != nil { return n, err }\n    if n < len(p) { return n, io.ErrShortWrite }\n    return n, nil\n}","handlingStrategy":"try-catch","validationCode":"if fi, err := outStat(); err == nil && fi.Size() >= expectedFooterSize { /* destination plausible */ }","typeGuard":null,"tryCatchPattern":"if _, err := seg.Marshal(w); err != nil {\n    if errors.Is(err, io.ErrShortWrite) || errors.Is(err, syscall.ENOSPC) {\n        return fmt.Errorf(\"backup destination full or short write: %w\", err)\n    }\n    return err\n}","preventionTips":["Use file-based io.Writer implementations that honor the io.Writer contract (no silent short writes)","Monitor disk space during long backups","Validate the written backup afterward by reading it back with ReadFooter"],"tags":["io","write-failed","short-write","backup"],"backgroundTag":"file-write-failed","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}