{"record":{"id":"01e4f181b63b5af9","repo":"jackwener/OpenCLI","slug":"short-read-on-part-partnumber-expected-chunk","errorCode":null,"errorMessage":"Short read on part ${partNumber}: expected ${chunkSize} bytes, got ${bytesRead}","messagePattern":"Short read on part (.+?): expected (.+?) bytes, got (.+?)","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/douyin/_shared/tos-upload.js","lineNumber":313,"sourceCode":"    // Calculate total parts\n    const totalParts = Math.ceil(fileSize / PART_SIZE);\n    // Track uploaded bytes for progress\n    let uploadedBytes = completedParts.length * PART_SIZE;\n    if (onProgress)\n        onProgress(Math.min(uploadedBytes, fileSize), fileSize);\n    const fd = fs.openSync(filePath, 'r');\n    try {\n        for (let partNumber = 1; partNumber <= totalParts; partNumber++) {\n            if (completedPartNumbers.has(partNumber)) {\n                continue; // already uploaded\n            }\n            const offset = (partNumber - 1) * PART_SIZE;\n            const chunkSize = Math.min(PART_SIZE, fileSize - offset);\n            const buffer = Buffer.allocUnsafe(chunkSize);\n            const readFn = _readSyncOverride ?? fs.readSync;\n            const bytesRead = readFn(fd, buffer, 0, chunkSize, offset);\n            if (bytesRead !== chunkSize) {\n                throw new CommandExecutionError(`Short read on part ${partNumber}: expected ${chunkSize} bytes, got ${bytesRead}`);\n            }\n            const crc32 = await uploadPart(tosUrl, partNumber, uploadId, buffer, auth, uploadHeader, userId);\n            completedParts.push({ partNumber, crc32 });\n            saveResumeState(resumePath, { uploadId, fileSize, parts: completedParts });\n            uploadedBytes = Math.min(offset + chunkSize, fileSize);\n            if (onProgress)\n                onProgress(uploadedBytes, fileSize);\n        }\n    }\n    finally {\n        fs.closeSync(fd);\n    }\n    const completedKey = await completeMultipartUpload(tosUrl, uploadId, completedParts, auth, uploadHeader, userId);\n    deleteResumeState(resumePath);\n    return completedKey;\n}\n// ── Internal exports for testing ─────────────────────────────────────────────\nexport { PART_SIZE, RESUME_DIR, extractRegionFromHost, getResumeFilePath, loadResumeState, saveResumeState, deleteResumeState, computeAws4Headers, extractUploadId, crc32Hex, gatewayBaseUrl, gatewayHeaders, };","sourceCodeStart":295,"sourceCodeEnd":331,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/douyin/_shared/tos-upload.js#L295-L331","documentation":"While reading each PART_SIZE chunk of the video, tosUpload compares fs.readSync's returned byte count with the expected chunkSize; this CommandExecutionError is thrown on a short read, since uploading a partial buffer would corrupt the part's CRC32 and data.","triggerScenarios":"readFn(fd, buffer, 0, chunkSize, offset) returns fewer bytes than chunkSize — file shrank between stat and read, I/O error, or a testing override (_readSyncOverride) misbehaving.","commonSituations":"File modified/deleted while the upload is in progress; NFS/network filesystem partial reads; custom read override in tests returning wrong counts.","solutions":["Ensure the file isn't modified or deleted during upload (pause writers, copy to stable location)","Retry the upload; transient filesystem errors often resolve","Re-run fs.statSync to confirm size matches the resumeState fileSize","If using setReadSyncOverride in tests, make it return the requested byte count"],"exampleFix":"// before\nconst bytesRead = fs.readSync(fd, buffer, 0, chunkSize, offset);\n// after (loop until full read)\nlet bytesRead = 0;\nwhile (bytesRead < chunkSize) {\n  const n = fs.readSync(fd, buffer, bytesRead, chunkSize - bytesRead, offset + bytesRead);\n  if (n <= 0) break;\n  bytesRead += n;\n}","handlingStrategy":"validation","validationCode":"const { size: sizeNow } = fs.statSync(filePath);\nif (sizeNow !== expectedSize) {\n  throw new Error(`file changed: expected ${expectedSize}, now ${sizeNow}`);\n}","typeGuard":null,"tryCatchPattern":"try { await tosUpload(options); }\ncatch (e) {\n  if (String(e.message).startsWith('Short read on part')) {\n    console.error('file changed during upload or fs error; retry with the file held stable');\n  }\n  throw e;\n}","preventionTips":["Never write to/delete the video while it is uploading","Upload from local disk rather than flaky network mounts when possible","Remove custom _readSyncOverride in production paths","Verify resume-state fileSize still matches the current file size"],"tags":["filesystem","io","upload"],"backgroundTag":"short-file-read","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}