What causes "invalid gc meta record size: %d" in juicedata/juicefs?

Delete stale contents of the sort work directory and re-run `juicefs gc` with --sort-dir pointing to a clean directory

invalid gc meta record size: %d

Error message

invalid gc meta record size: %d

What it means

Returned by gcMetaRecordFromBytes when a serialized gc meta record read back from the external sort spill files is not exactly 13 bytes (8-byte sliceID + 4-byte size + 1-byte state). This indicates corruption of the external sort intermediate data, guarded by the sorter's Checksum option.

Source

Thrown at cmd/gc_external.go:220

		return gcObjectRecord{}, false
	}
	return gcObjectRecord{sliceID: sliceID, index: index, blockSize: blockSize, objectSize: obj.Size(), key: obj.Key()}, true
}

type gcMetaRecord struct {
	sliceID uint64
	size    uint32
	state   uint8
}

const (
	gcMetaRecordSize        = 13
	gcObjectRecordFixedSize = 24
)

func gcMetaRecordFromBytes(data []byte) (gcMetaRecord, error) {
	if len(data) != gcMetaRecordSize {
		return gcMetaRecord{}, errors.Errorf("invalid gc meta record size: %d", len(data))
	}
	rb := utils.FromBuffer(data)
	return gcMetaRecord{
		sliceID: rb.Get64(),
		size:    rb.Get32(),
		state:   rb.Get8(),
	}, nil
}

func gcMetaRecordToBytes(r gcMetaRecord) ([]byte, error) {
	wb := utils.NewBuffer(gcMetaRecordSize)
	wb.Put64(r.sliceID)
	wb.Put32(r.size)
	wb.Put8(r.state)
	return wb.Bytes(), nil
}

func compareGcMetaRecord(a, b gcMetaRecord) int {

View on GitHub (pinned to c9a67b23e8)

Solutions

Delete stale contents of the sort work directory and re-run `juicefs gc` with --sort-dir pointing to a clean directory
Check the sort-dir filesystem for errors (dmesg, fsck) and free disk space
Run gc without external sort (smaller dataset) to bypass the extsort path
If reproducible, file a bug with the JuiceFS version — record sizes are fixed constants and should never mismatch

Example fix

// before
# corrupted leftovers in sort dir cause the error
juicefs gc --sort-dir /tmp/gc-sort sqlite3://test.db
// after
rm -rf /tmp/gc-sort/* && juicefs gc --sort-dir /tmp/gc-sort sqlite3://test.db

Defensive patterns

Strategy: fallback

Validate before calling

if fi, err := os.Stat(sortDir); err == nil && fi.IsDir() { /* start from a clean dir */ os.RemoveAll(filepath.Join(sortDir, "gc-meta-*")) }

Try / catch

if err := gcExternalSort(...); err != nil {
	if strings.Contains(err.Error(), "invalid gc meta record size") {
		os.RemoveAll(sortDir); /* clean and retry once */
	}
}

Prevention

Never reuse a sort directory after a crashed/killed gc run
Keep the sort volume from filling up (monitor df)
Run gc to completion or clean up spill files after abort

When it happens

Trigger: Only occurs when extsort deserializes spill segments during gc's external sort and a segment file is truncated or corrupted (partial write, disk issue, or a bug in record serialization), so the FromBytes codec receives a byte slice whose length differs from gcMetaRecordSize (13).

Common situations: Disk full causing truncated spill files; corrupted temp files in --sort-dir after a crash; I/O errors on the sort volume; tampered or manually cleaned work directory mid-run.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/24fecb581d472170. Report an issue: GitHub.