{"record":{"id":"2cc69b17429ae857","repo":"weaviate/weaviate","slug":"data-save-error","errorCode":null,"errorMessage":"data save error","messagePattern":"data save error","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"usecases/classification/classifier_run_worker.go","lineNumber":140,"sourceCode":"}\n\nfunc (ws *runWorkers) work(ctx context.Context) runWorkerResults {\n\tws.batchWriter.Start()\n\n\twg := &sync.WaitGroup{}\n\tfor _, worker := range ws.workers {\n\t\tworker := worker\n\t\twg.Add(1)\n\t\tenterrors.GoWrapper(func() { worker.work(ctx, wg) }, ws.logger)\n\n\t}\n\n\twg.Wait()\n\n\tres := ws.batchWriter.Stop()\n\n\tif res.SuccessCount() != *ws.successCount || res.ErrorCount() != *ws.errorCount {\n\t\tws.ec.Add(errors.New(\"data save error\"))\n\t}\n\n\tif res.Err() != nil {\n\t\tws.ec.Add(res.Err())\n\t}\n\n\treturn runWorkerResults{\n\t\tsuccessCount: *ws.successCount,\n\t\terrorCount:   *ws.errorCount,\n\t\terr:          ws.ec.ToError(),\n\t}\n}\n\ntype runWorkerResults struct {\n\tsuccessCount int64\n\terrorCount   int64\n\terr          error\n}","sourceCodeStart":122,"sourceCodeEnd":158,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/usecases/classification/classifier_run_worker.go#L122-L158","documentation":"After all classification item workers finish (wg.Wait), the work() routine stops the batch writer and compares the counts it recorded (successCount/errorCount) against the counts the batch writer actually reported. A mismatch means some object saves silently succeeded/failed outside the accounting (e.g. results were dropped or double-counted), so a synthetic 'data save error' is added to the error counter to fail the classification run rather than report a wrong success rate.","triggerScenarios":"batchWriter.Stop() returns a result whose SuccessCount()/ErrorCount() differ from the tallies kept by the worker shard (ws.successCount/ws.errorCount) — typically when objects were added/removed or the shard state changed mid-run, or an internal accounting race in the batch writer.","commonSituations":"Objects deleted or schema changed while a classification was running on a large dataset; concurrent writes to the same collection during classification; an aborted shard write whose outcome never got counted.","solutions":["Re-run the classification once the cluster is quiescent (no concurrent imports/deletes on the target collection).","Check res.Err() and the error counter (ws.ec) for the underlying write failure and fix that first (disk, shard availability).","If reproducible without concurrent writes, file a bug with logs — it indicates an internal count-accounting defect."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Go — re-run classification when only accounting failed\nerr := classifier.Run(ctx, req)\nif err != nil && strings.Contains(err.Error(), \"data save error\") {\n    // check /v1/classifications/{id} status, then re-trigger once quiescent\n    return retryClassificationAfterQuiesce(ctx, req)\n}","preventionTips":["Avoid concurrent imports/deletes on a collection during classification.","Monitor classification error counters and batch-writer metrics.","Re-run failed classifications rather than trusting partial results."],"tags":["classification","batching","consistency"],"backgroundTag":"count-mismatch","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}