{"record":{"id":"6448d2a9449fcfec","repo":"weaviate/weaviate","slug":"transform-value-w","errorCode":null,"errorMessage":"transform value: %w","messagePattern":"transform value: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"adapters/repos/db/lsmkv/compactor_replace.go","lineNumber":273,"sourceCode":"\t\t\t// a secondary key is not guaranteed to be present for every index\n\t\t\t// position (e.g. segment written with a different secondary index\n\t\t\t// count). Skip positions beyond what the compactor expects.\n\t\t\tbreak\n\t\t}\n\t\tc.secIndexSizes[pos] += int64(segmentindex.TREE_KEY_STORE_OVERHEAD + len(sk))\n\t}\n}\n\nfunc (c *compactorReplace) writeIndividualNode(f *segmentindex.SegmentFile,\n\toffset int, key, value []byte, secondaryKeys [][]byte, tombstone bool,\n) (segmentindex.Key, error) {\n\t// Rewrite live values through the active edit transformer before they hit\n\t// the merged segment. Tombstones and empty payloads carry nothing to\n\t// transform, so they are skipped.\n\tif c.valueTransformer != nil && !tombstone && len(value) > 0 {\n\t\ttransformed, err := c.valueTransformer(value)\n\t\tif err != nil {\n\t\t\treturn segmentindex.Key{}, fmt.Errorf(\"transform value: %w\", err)\n\t\t}\n\t\tvalue = transformed\n\t}\n\n\t// Copy key bytes into stable arena memory. The reusable cursor reuses its\n\t// internal buffers on every next() call, so ki.Key / ki.SecondaryKeys stored\n\t// in the kis slice would otherwise be corrupted on the next iteration.\n\tkeyCopy := c.arena.CopyKey(key)\n\n\tvar secKeysCopy [][]byte\n\tif len(secondaryKeys) > 0 {\n\t\tsecKeysCopy = make([][]byte, len(secondaryKeys))\n\t\tfor i, sk := range secondaryKeys {\n\t\t\tsecKeysCopy[i] = c.arena.CopyKey(sk)\n\t\t}\n\t}\n\n\t// value is NOT arena-copied because it is written synchronously to the","sourceCodeStart":255,"sourceCodeEnd":291,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/lsmkv/compactor_replace.go#L255-L291","documentation":"writeIndividualNode optionally rewrites live (non-tombstone, non-empty) values through c.valueTransformer — the active edit transformer used e.g. when compression encoding changes across segment versions. If the transformer returns an error for the given value bytes, compaction aborts with this wrapped error. It means the on-disk value could not be converted to the new encoding/representation.","triggerScenarios":"A replace-bucket compaction runs with a valueTransformer configured and the transformer function errors on a specific live value — e.g. decompression/compression failure (BQ/PQ/SQ compression change), value produced by an older incompatible segment version, or bit-corrupted value bytes.","commonSituations":"Upgrading Weaviate with a changed default compression so old segments must be rewritten; manually downgrading/switching compression options on a bucket; corrupted segment data after a crash.","solutions":["Read the wrapped cause to identify whether it is a decompression failure (corrupt data) or an encode failure (incompatible value).","If compression settings were changed, verify the transformer supports values from all existing segment versions (upgrade path).","Restore the affected bucket from backup if the source segment is corrupt, then let compaction rebuild it.","Report the failing value pattern upstream if it reproduces on valid data — the transformer has a version-compatibility bug."],"exampleFix":"// before: transformer assumes new encoding and fails on old values\ntransformed, err := c.valueTransformer(value)\n// after (caller-side mitigation): re-vectorize/rewrite the collection\n// or revert the compression setting change that forced the transform\n//   client.schema.updateCollection({ compression: previousSetting })","handlingStrategy":"fallback","validationCode":"// before enabling a new compression on a collection, round-trip test:\n// transformed, err := transformer(existingValue); require.NoError(t, err)","typeGuard":"func isTransformErr(err error) bool {\n\treturn strings.Contains(err.Error(), \"transform value:\")\n}","tryCatchPattern":"if err := bucket.Compact(ctx); err != nil {\n\tif isTransformErr(err) {\n\t\tlog.Printf(\"value transform failed: %v — restore bucket from backup or revert compression setting\", err)\n\t}\n}","preventionTips":["Round-trip test the transformer against values from every existing segment version before upgrading.","Take a backup before changing compression/encoding settings on a collection.","Avoid downgrading across versions that changed value encodings.","Treat this error as data-integrity signal: verify segment checksums and restore from backup if corrupt."],"tags":["lsmkv","compaction","value-transformer","compression"],"backgroundTag":"value-transform-failed","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}