What causes "invalid gc object record size: %d" in juicedata/juicefs?

Clean the sort work directory and re-run `juicefs gc`

invalid gc object record size: %d

Error message

invalid gc object record size: %d

What it means

Returned by gcObjectRecordFromBytes when a serialized gc object record read from the external sort spill files is shorter than the fixed 24-byte prefix (8-byte sliceID + 4-byte index + 4-byte blockSize + 8-byte objectSize; variable key follows). Like its meta counterpart, this signals corrupted or truncated external sort intermediate data.

Source

Thrown at cmd/gc_external.go:255

func compareGcMetaRecord(a, b gcMetaRecord) int {
	if c := cmp.Compare(a.sliceID, b.sliceID); c != 0 {
		return c
	}
	return cmp.Compare(a.state, b.state)
}

type gcObjectRecord struct {
	sliceID    uint64
	index      int
	blockSize  int
	objectSize int64
	key        string
}

func gcObjectRecordFromBytes(data []byte) (gcObjectRecord, error) {
	if len(data) < gcObjectRecordFixedSize {
		return gcObjectRecord{}, errors.Errorf("invalid gc object record size: %d", len(data))
	}
	rb := utils.FromBuffer(data)
	return gcObjectRecord{
		sliceID:    rb.Get64(),
		index:      int(rb.Get32()),
		blockSize:  int(rb.Get32()),
		objectSize: int64(rb.Get64()),
		key:        string(rb.Get(len(data) - gcObjectRecordFixedSize)),
	}, nil
}

func gcObjectRecordToBytes(r gcObjectRecord) ([]byte, error) {
	wb := utils.NewBuffer(uint32(gcObjectRecordFixedSize + len(r.key)))
	wb.Put64(r.sliceID)
	wb.Put32(uint32(r.index))
	wb.Put32(uint32(r.blockSize))
	wb.Put64(uint64(r.objectSize))
	wb.Put([]byte(r.key))

View on GitHub (pinned to c9a67b23e8)

Solutions

Clean the sort work directory and re-run `juicefs gc`
Verify disk space and filesystem health on the --sort-dir volume
Bypass external sort for smaller datasets if possible
If reproducible on a clean directory, report a bug with JuiceFS version and gc command line

Example fix

// before
# disk full during previous run left truncated segments
juicefs gc --sort-dir /tmp/gc-sort redis://localhost:6379/1
// after
df -h /tmp && rm -rf /tmp/gc-sort/* && juicefs gc --sort-dir /tmp/gc-sort redis://localhost:6379/1

Defensive patterns

Strategy: fallback

Validate before calling

if fi, err := os.Stat(sortDir); err == nil && fi.IsDir() { /* start from a clean dir */ os.RemoveAll(filepath.Join(sortDir, "gc-object-*")) }

Try / catch

if err := gcExternalSort(...); err != nil {
	if strings.Contains(err.Error(), "invalid gc object record size") {
		os.RemoveAll(sortDir); /* clean and retry once */
	}
}

Prevention

Clean spill files after any aborted gc run
Ensure enough free space for object-record spill files (records are larger with keys)
Avoid killing gc mid-spill; use graceful cancellation

When it happens

Trigger: extsort deserializes gc-object spill segments during gc and a segment is truncated/corrupted so the codec gets fewer than gcObjectRecordFixedSize (24) bytes — partial spill write, disk-full, or I/O corruption in the sort work directory.

Common situations: Sort volume ran out of disk space mid-spill; crash or kill during gc leaving partial segment files reused later; faulty disk; manually deleted/edited files in --sort-dir.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/a15cc2efba199162. Report an issue: GitHub.