What causes "write out of chunk boundary: %d > %d" in juicedata/juicefs?

Fix the caller to split writes so each write fits within chunkSize (end at off+len(p) <= chunkSize).

write out of chunk boundary: %d > %d

Error message

write out of chunk boundary: %d > %d

What it means

wSlice represents one slice of a chunk (chunkSize bytes) being staged for upload. Its WriteAt enforces that writes stay within the fixed chunk boundary; a write whose offset+length exceeds chunkSize is rejected. This guards the internal invariant that one wSlice maps to at most chunkSize bytes.

Source

Thrown at pkg/chunk/cached_store.go:257

		rSlice:    rSlice{id, 0, store},
		pages:     make([][]*Page, chunkSize/store.conf.BlockSize),
		errors:    make(chan error, chunkSize/store.conf.BlockSize),
		writeback: store.conf.Writeback,
		tierID:    tierID,
	}
}

func (s *wSlice) SetID(id uint64) {
	s.id = id
}

func (s *wSlice) SetWriteback(enabled bool) {
	s.writeback = enabled
}

func (s *wSlice) WriteAt(p []byte, off int64) (n int, err error) {
	if int(off)+len(p) > chunkSize {
		return 0, fmt.Errorf("write out of chunk boundary: %d > %d", int(off)+len(p), chunkSize)
	}
	if off < int64(s.uploaded) {
		return 0, fmt.Errorf("Cannot overwrite uploaded block: %d < %d", off, s.uploaded)
	}

	// Fill previous blocks with zeros
	if s.length < int(off) {
		zeros := make([]byte, int(off)-s.length)
		_, _ = s.WriteAt(zeros, int64(s.length))
	}

	for n < len(p) {
		indx := s.index(int(off) + n)
		boff := (int(off) + n) % s.store.conf.BlockSize
		var bs = pageSize
		if indx > 0 || bs > s.store.conf.BlockSize {
			bs = s.store.conf.BlockSize
		}

View on GitHub (pinned to c9a67b23e8)

Solutions

Fix the caller to split writes so each write fits within chunkSize (end at off+len(p) <= chunkSize).
Verify chunkSize used by the caller matches the store's configured BlockSize/chunkSize.
Clamp or split oversized writes across multiple wSlice/chunk boundaries before calling WriteAt.
If this arises during normal mounting (not custom code), report a bug with the workload reproducing it — it indicates an internal chunk-splitting defect.

Example fix

// before
slice.WriteAt(p, off) // off+len(p) may exceed chunkSize
// after
if int(off)+len(p) > chunkSize {
    n1 := chunkSize - int(off)
    slice.WriteAt(p[:n1], off)
    nextSlice.WriteAt(p[n1:], 0)
}

Defensive patterns

Strategy: validation

Validate before calling

if int(off)+len(p) > chunkSize {
    return errors.New("write exceeds chunk boundary; split before calling WriteAt")
}

Try / catch

n, err := slice.WriteAt(p, off)
if err != nil && strings.Contains(err.Error(), "out of chunk boundary") {
    // split the write at chunkSize-off and continue on next slice
}

Prevention

Compute block splits from the store's chunkSize, not hardcoded sizes
Unit-test write paths near chunk boundaries

When it happens

Trigger: Calling wSlice.WriteAt with p and off such that int(off)+len(p) > chunkSize — e.g. a caller splitting data into chunks incorrectly, or passing an offset near the end of a chunk with a buffer sized for a full block.

Common situations: Custom tooling writing to the chunk store directly with miscomputed block offsets; internal bugs in chunk splitting after a version change of chunkSize assumptions; replaying a write log with wrong offsets.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/2f26449354ce1ec8. Report an issue: GitHub.