centered RQ mean length %d does not match input dimension %d

Error message

centered RQ mean length %d does not match input dimension %d

What it means

A validation (not I/O) error: readRQCenteredData compares the declared centering-mean length against the RQ data's InputDim and rejects mismatches before allocating the float32 slice. The mean must have exactly InputDim entries; anything else means the snapshot metadata is damaged or written by an incompatible format, and honoring the length could request an arbitrarily large allocation.

Source

Thrown at adapters/repos/db/vector/hnsw/compact/snapshot_reader.go:295

	var flags uint8
	if err := binary.Read(reader, binary.LittleEndian, &flags); err != nil {
		return errors.Wrap(err, "read centered RQ flags")
	}
	if err := r.readRQData(reader, res); err != nil {
		return err
	}
	if err := applyRQCenteredFlags(res.CompressionRQData(), flags); err != nil {
		return err
	}
	var meanLen uint32
	if err := binary.Read(reader, binary.LittleEndian, &meanLen); err != nil {
		return errors.Wrap(err, "read RQ mean length")
	}
	// The mean always has exactly InputDim entries; validating before the
	// allocation stops a damaged snapshot from requesting an arbitrarily
	// large slice.
	if inputDim := res.CompressionRQData().InputDim; meanLen != inputDim {
		return errors.Errorf("centered RQ mean length %d does not match input dimension %d", meanLen, inputDim)
	}
	mean := make([]float32, meanLen)
	for i := range mean {
		var bits uint32
		if err := binary.Read(reader, binary.LittleEndian, &bits); err != nil {
			return errors.Wrap(err, "read RQ mean")
		}
		mean[i] = math.Float32frombits(bits)
	}
	res.CompressionRQData().Mean = mean
	return nil
}

// readPQData reads Product Quantization data from the reader.
func (r *SnapshotReader) readPQData(reader io.Reader, res *ent.DeserializationResult) error {
	var dims, ks, m uint16
	var encoderType, dist uint8
	var useBitsEncoding uint8

View on GitHub (pinned to 75aa4b6d11)

Solutions

Delete the snapshot file and restart Weaviate so it regenerates from the commit log or a fresh compaction.
Confirm all cluster nodes run the same Weaviate version before restoring shard files.
Restore from a verified backup instead of copying raw files.
Run filesystem/disk health checks if the corruption repeats.

Example fix

// error: 'centered RQ mean length 943 does not match input dimension 1536'
// after
rm <shard>/vector_index/hnsw.snapshot* && systemctl restart weaviate

Defensive patterns

Strategy: validation

Validate before calling

// the reader itself validates; the caller cannot pre-check without parsing, but can sanity-check RQ config vs schema dimension
if rqCfg.InputDim != vectorDim { return fmt.Errorf("RQ InputDim %d != vector dimension %d", rqCfg.InputDim, vectorDim) }

Try / catch

if _, err := reader.Read(f); err != nil { if strings.Contains(err.Error(), "does not match input dimension") { os.Remove(snapshotPath); return regenerateSnapshot() } return err }

Prevention

Never edit snapshot bytes manually
Keep RQ compression settings stable across node versions in a cluster
Detect bit-rot with checksumming filesystems or backup-verified restores
If this error appears, treat the snapshot as corrupt — it is guarded to prevent huge allocations from damaged length fields

When it happens

Trigger: SnapshotReader.Read on a centered-RQ snapshot whose declared mean length differs from the InputDim stored in its RQ payload — corrupt/garbage bytes where the length field is, or writer/reader format skew.

Common situations: Bit-rot or partial write corrupting the length field; hand-edited or wrongly restored snapshot files; mixing Weaviate versions with different centered-RQ layouts across a cluster.

Related errors

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/8a6cde942fd9a303. Report an issue: GitHub.