{"record":{"id":"62fa163eacb3ee61","repo":"weaviate/weaviate","slug":"unmarshal-new-reindex-payload-w","errorCode":null,"errorMessage":"unmarshal new reindex payload: %w","messagePattern":"unmarshal new reindex payload: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"adapters/repos/db/reindex_conflict.go","lineNumber":146,"sourceCode":"// CheckConflict implements [distributedtask.ConflictDetector] for the\n// reindex namespace. Called under [Manager.mu] from the RAFT-apply\n// AddTask path BEFORE the new task is appended to FSM-stored state.\n// Returns a non-nil error iff `newPayload` would conflict with an\n// already-STARTED task in `existingTasks`.\n//\n// FSM-determinism: every node applies the same RAFT log entry, sees\n// the same `existingTasks` snapshot, and runs this same function — so\n// every node reaches the same accept/reject decision. The function\n// must remain a pure transform of its arguments.\n//\n// Conflict rule: any two reindex migrations on overlapping properties\n// of the same collection conflict, regardless of which bucket type\n// they primarily write to. See [typesConflictReason] for the\n// rationale.\nfunc (p *ReindexProvider) CheckConflict(newPayload []byte, existingTasks []*distributedtask.Task) error {\n\tvar newP ReindexTaskPayload\n\tif err := json.Unmarshal(newPayload, &newP); err != nil {\n\t\treturn fmt.Errorf(\"unmarshal new reindex payload: %w\", err)\n\t}\n\tif newP.Collection == \"\" || newP.MigrationType == \"\" {\n\t\treturn fmt.Errorf(\"new reindex payload missing Collection or MigrationType\")\n\t}\n\n\tfor _, task := range existingTasks {\n\t\t// PREPARING and SWAPPING are the subtle ones: every unit has\n\t\t// reached terminal state, but the post-completion callbacks have\n\t\t// not yet committed. A new migration on the same property could\n\t\t// land before the schema flip commits, leaving it and the\n\t\t// unfinished swap racing on the same bucket pointers.\n\t\tif !task.Status.IsActive() {\n\t\t\tcontinue\n\t\t}\n\n\t\tvar existP ReindexTaskPayload\n\t\tif err := json.Unmarshal(task.Payload, &existP); err != nil {\n\t\t\t// Existing task has an unparseable payload. We can't prove","sourceCodeStart":128,"sourceCodeEnd":164,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/reindex_conflict.go#L128-L164","documentation":"ReindexProvider.CheckConflict — the distributed-task ConflictDetector run on every node during RAFT-apply of AddTask — first json.Unmarshals the new task's payload into ReindexTaskPayload. If that fails it returns 'unmarshal new reindex payload: %w'. Because the function must be a pure, FSM-deterministic transform of its arguments, a payload that cannot be parsed is rejected outright rather than guessed at.","triggerScenarios":"Submitting a reindex (distributed) task whose payload bytes are not valid JSON or do not match ReindexTaskPayload's shape (wrong field types, truncated payload, non-JSON bytes).","commonSituations":"Client tooling building payloads by hand with wrong JSON shape; version skew where a newer node writes payload fields an older decoder rejects; corrupted task payload in the RAFT log.","solutions":["Marshal the payload via the ReindexTaskPayload type (json.Marshal) instead of hand-building JSON, and validate the round-trip before submitting.","Log/print the wrapped json error — it names the exact offset/type mismatch; fix that field in the payload.","Check Weaviate version skew across the cluster; align versions so all nodes parse the same payload schema."],"exampleFix":"// before\npayload := []byte(fmt.Sprintf(`{\"collection\":%s}`, collection)) // malformed JSON\n// after\ntype ReindexTaskPayload struct{ Collection string; MigrationType string; Properties []string }\nnewP := ReindexTaskPayload{Collection: collection, MigrationType: mt, Properties: props}\npayload, err := json.Marshal(newP)\nif err != nil { return err }","handlingStrategy":"validation","validationCode":"func validReindexPayload(b []byte) error {\n    var p ReindexTaskPayload\n    if err := json.Unmarshal(b, &p); err != nil { return err }\n    if p.Collection == \"\" || p.MigrationType == \"\" { return errors.New(\"missing Collection or MigrationType\") }\n    return nil\n}","typeGuard":null,"tryCatchPattern":"if err := provider.CheckConflict(payload, tasks); err != nil {\n    var jsonErr *json.SyntaxError\n    if errors.As(err, &jsonErr) && strings.HasPrefix(err.Error(), \"unmarshal new reindex payload\") {\n        return fmt.Errorf(\"malformed reindex payload at offset %d: %w\", jsonErr.Offset, err)\n    }\n    return err\n}","preventionTips":["Always build reindex payloads with json.Marshal on the typed struct","Validate payloads with a client-side unmarshal round-trip before submitting","Align Weaviate versions cluster-wide so payload schemas match"],"tags":["weaviate","reindex","json","payload","deserialization"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}