What causes "writeend %s: %s" in juicedata/juicefs?

Check backend logs at the failure timestamp for the specific errno cause (connection, timeout, quota)

writeend %s: %s

Error message

writeend %s: %s

What it means

Reported when m.Write() — which persists a filled slice into the file's chunk index in the metadata engine — returns a non-zero errno. After NewSlice allocates an ID, mdtest bypasses the normal write path and calls meta.Write directly to register the slice at chunk index `indx`; a failure here means the metadata engine rejected the chunk-index update. Shown as `writeend <file>: <errno>` when running with -write > 0.

Source

Thrown at cmd/mdtest.go:84

	m := jfs.Meta()
	for i := 0; i < files; i++ {
		fn := path.Join(root, fmt.Sprintf("file.mdtest.%d.%d", np, i))
		f, err := jfs.Create(ctx, fn, 0666, umask)
		if err != 0 {
			return fmt.Errorf("create %s: %s", fn, err)
		}
		if bytes > 0 {
			for indx := 0; indx*meta.ChunkSize < bytes; indx++ {
				var id uint64
				if st := m.NewSlice(ctx, &id); st != 0 {
					return fmt.Errorf("writechunk %s: %s", fn, st)
				}
				size := meta.ChunkSize
				if bytes < (indx+1)*meta.ChunkSize {
					size = bytes - indx*meta.ChunkSize
				}
				if st := m.Write(ctx, f.Inode(), uint32(indx), 0, meta.Slice{Id: id, Size: uint32(size), Len: uint32(size)}, time.Now()); st != 0 {
					return fmt.Errorf("writeend %s: %s", fn, st)
				}
			}
		}
		f.Close(ctx)
		bar.Increment()
	}
	if d > 0 {
		dirs := make([]int, width)
		for i := 0; i < width; i++ {
			dirs[i] = i
		}
		rand.Shuffle(width, func(i, j int) {
			dirs[i], dirs[j] = dirs[j], dirs[i]
		})
		for i := range dirs {
			dn := path.Join(root, fmt.Sprintf("mdtest_tree.%d", dirs[i]))
			if err := createFile(jfs, bar, np, dn, d-1, width, files, bytes); err != nil {
				return err

View on GitHub (pinned to c9a67b23e8)

Solutions

Check backend logs at the failure timestamp for the specific errno cause (connection, timeout, quota)
Ensure session stability — avoid restarting the meta backend during mdtest; re-run after it recovers
Reduce --threads and file count to lower write pressure on the metadata engine
Confirm no quota is exceeded on the volume (`juicefs quota` / info) since quota denial fails Write
Retry with the same command once backend health is confirmed; errors here are mostly transient backend issues

Example fix

// before
// mdtest run continues against a restarting Redis -> writeend file.mdtest.0.3: connection refused
// after
// wait for backend healthy, then
$ juicefs mdtest --threads 8 -write 1048576 redis://localhost /t

Defensive patterns

Strategy: retry

Validate before calling

// confirm session alive and no quota exceeded before write phase
if err := m.Ping(meta.Background()); err != 0 {
    logger.Fatalf("session/backend unhealthy: %v", err)
}

Try / catch

// treat errno as transient and re-run or retry the write
if st := m.Write(ctx, f.Inode(), uint32(indx), 0, sl, time.Now()); st != 0 {
    if isTransient(st) { // EIO/EAGAIN/conn errors
        time.Sleep(retryBackoff)
        continue
    }
    return fmt.Errorf("writeend %s: %s", fn, st)
}

Prevention

Don't restart or fail over the meta backend mid-benchmark
Check volume quotas before write-heavy runs
Keep sessions alive; avoid client idle timeouts on long runs
Monitor backend error rates and stop the benchmark when they spike

When it happens

Trigger: `juicefs mdtest -write N` where the meta backend fails on the chunk write/persist step: session lost or revoked (EINVAL/EPERM on stale session), backend write error, timeout, or quota enforcement rejecting the write for the inode.

Common situations: Long-running mdtest where the client session expires mid-write; metadata backend restart or failover during the run; Redis async replication lagging/failing under write load; SQL backend connection pool exhausted under many threads.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/67b80ef63549d34a. Report an issue: GitHub.