{"record":{"id":"47495effb5cc8fbd","repo":"weaviate/weaviate","slug":"write-split-file-for-shard-q-w","errorCode":null,"errorMessage":"write split file for shard %q: %w","messagePattern":"write split file for shard %q: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"usecases/backup/backend.go","lineNumber":716,"sourceCode":"\t\t\t// Close writes tar/gzip trailers and could fail if the pipe is closed.\n\t\t\t// Use CloseWithError to signal any producer error to the consumer,\n\t\t\t// so the consumer's read fails instead of seeing EOF.\n\t\t\tcloseErr := zip.CloseWithError(err)\n\t\t\terr = errors.Join(err, labelErr(\"close\", closeErr))\n\t\t}()\n\n\t\tif err := ctx.Err(); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tvar fileSizeExceededInfo *SplitFile\n\t\tif fileSizeExceededWrite != nil {\n\t\t\t// Only write the split file part in this chunk; remaining space is intentionally\n\t\t\t// left unused to keep the logic simple and avoid mixing split file parts with\n\t\t\t// regular files in the same chunk.\n\t\t\tfileSizeExceededInfo, err = zip.WriteSplitFile(ctx, shard, fileSizeExceededWrite, &preCompressionSize, chunkKey)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"write split file for shard %q: %w\", shard.Name, err)\n\t\t\t}\n\t\t} else {\n\t\t\t_, fileSizeExceededInfo, err = zip.WriteShard(ctx, shard, filesInShard, firstChunkForShard, &preCompressionSize, chunkKey)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"write files for shard %q: %w\", shard.Name, err)\n\t\t\t}\n\t\t}\n\t\tshard.ClearTemporary()\n\n\t\tif zip.compressorWriter != nil {\n\t\t\tif err := zip.compressorWriter.Flush(); err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"flush compressor: %w\", err)\n\t\t\t}\n\t\t}\n\n\t\treturn fileSizeExceededInfo, nil\n\t}\n","sourceCodeStart":698,"sourceCodeEnd":734,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/usecases/backup/backend.go#L698-L734","documentation":"Wraps a failure from zip.WriteSplitFile when a shard's data exceeds the max chunk size and must be written as a split-file part. If writing the split-file chunk to the destination (or compressing it) fails, the shard upload aborts with this error naming the shard.","triggerScenarios":"Backing up a shard whose files exceed the per-chunk size limit so the code takes the fileSizeExceededWrite branch, and WriteSplitFile returns an error — destination write failure, compression error, or destination backend error.","commonSituations":"Very large shards combined with a misconfigured or full backup destination (S3 bucket permissions, quota, network drops); an overly small backup chunk size setting forcing split files for most shards.","solutions":["Inspect the wrapped inner error to see whether it is a destination-backend or compression failure.","If chunk size is set very low, increase the backup chunk size limit so fewer shards need splitting.","Verify write permissions, quota, and connectivity to the backup destination (bucket policy, credentials).","Retry the backup; transient destination errors are common with large split writes."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// preflight: destination is writable and quota OK\nif err := probeDestinationWrite(bucket); err != nil {\n    return fmt.Errorf(\"destination not writable: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"if err := runBackup(ctx); err != nil {\n    if strings.Contains(err.Error(), \"write split file for shard\") {\n        // increase chunk size or fix destination, then retry\n        err = runBackupWithLargerChunks(ctx)\n    }\n}","preventionTips":["Set a chunk size large enough that most shards avoid split-file paths.","Verify destination credentials, quota, and permissions before large backups.","Retry large-shard backups on transient destination errors."],"tags":["backup","zip","shard","destination"],"backgroundTag":"shard-write-failed","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}