{"record":{"id":"8a6cde942fd9a303","repo":"weaviate/weaviate","slug":"centered-rq-mean-length-d-does-not-match-input-di","errorCode":null,"errorMessage":"centered RQ mean length %d does not match input dimension %d","messagePattern":"centered RQ mean length (.+?) does not match input dimension (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"adapters/repos/db/vector/hnsw/compact/snapshot_reader.go","lineNumber":295,"sourceCode":"\tvar flags uint8\n\tif err := binary.Read(reader, binary.LittleEndian, &flags); err != nil {\n\t\treturn errors.Wrap(err, \"read centered RQ flags\")\n\t}\n\tif err := r.readRQData(reader, res); err != nil {\n\t\treturn err\n\t}\n\tif err := applyRQCenteredFlags(res.CompressionRQData(), flags); err != nil {\n\t\treturn err\n\t}\n\tvar meanLen uint32\n\tif err := binary.Read(reader, binary.LittleEndian, &meanLen); err != nil {\n\t\treturn errors.Wrap(err, \"read RQ mean length\")\n\t}\n\t// The mean always has exactly InputDim entries; validating before the\n\t// allocation stops a damaged snapshot from requesting an arbitrarily\n\t// large slice.\n\tif inputDim := res.CompressionRQData().InputDim; meanLen != inputDim {\n\t\treturn errors.Errorf(\"centered RQ mean length %d does not match input dimension %d\", meanLen, inputDim)\n\t}\n\tmean := make([]float32, meanLen)\n\tfor i := range mean {\n\t\tvar bits uint32\n\t\tif err := binary.Read(reader, binary.LittleEndian, &bits); err != nil {\n\t\t\treturn errors.Wrap(err, \"read RQ mean\")\n\t\t}\n\t\tmean[i] = math.Float32frombits(bits)\n\t}\n\tres.CompressionRQData().Mean = mean\n\treturn nil\n}\n\n// readPQData reads Product Quantization data from the reader.\nfunc (r *SnapshotReader) readPQData(reader io.Reader, res *ent.DeserializationResult) error {\n\tvar dims, ks, m uint16\n\tvar encoderType, dist uint8\n\tvar useBitsEncoding uint8","sourceCodeStart":277,"sourceCodeEnd":313,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/vector/hnsw/compact/snapshot_reader.go#L277-L313","documentation":"A validation (not I/O) error: readRQCenteredData compares the declared centering-mean length against the RQ data's InputDim and rejects mismatches before allocating the float32 slice. The mean must have exactly InputDim entries; anything else means the snapshot metadata is damaged or written by an incompatible format, and honoring the length could request an arbitrarily large allocation.","triggerScenarios":"SnapshotReader.Read on a centered-RQ snapshot whose declared mean length differs from the InputDim stored in its RQ payload — corrupt/garbage bytes where the length field is, or writer/reader format skew.","commonSituations":"Bit-rot or partial write corrupting the length field; hand-edited or wrongly restored snapshot files; mixing Weaviate versions with different centered-RQ layouts across a cluster.","solutions":["Delete the snapshot file and restart Weaviate so it regenerates from the commit log or a fresh compaction.","Confirm all cluster nodes run the same Weaviate version before restoring shard files.","Restore from a verified backup instead of copying raw files.","Run filesystem/disk health checks if the corruption repeats."],"exampleFix":"// error: 'centered RQ mean length 943 does not match input dimension 1536'\n// after\nrm <shard>/vector_index/hnsw.snapshot* && systemctl restart weaviate","handlingStrategy":"validation","validationCode":"// the reader itself validates; the caller cannot pre-check without parsing, but can sanity-check RQ config vs schema dimension\nif rqCfg.InputDim != vectorDim { return fmt.Errorf(\"RQ InputDim %d != vector dimension %d\", rqCfg.InputDim, vectorDim) }","typeGuard":null,"tryCatchPattern":"if _, err := reader.Read(f); err != nil { if strings.Contains(err.Error(), \"does not match input dimension\") { os.Remove(snapshotPath); return regenerateSnapshot() } return err }","preventionTips":["Never edit snapshot bytes manually","Keep RQ compression settings stable across node versions in a cluster","Detect bit-rot with checksumming filesystems or backup-verified restores","If this error appears, treat the snapshot as corrupt — it is guarded to prevent huge allocations from damaged length fields"],"tags":["hnsw","snapshot","rq","validation","corruption"],"backgroundTag":"snapshot-file-corrupted","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}