{"record":{"id":"881d00ee4bba2b58","repo":"paperclipai/paperclip","slug":"remote-file-read-was-truncated-for-remotepath","errorCode":null,"errorMessage":"Remote file read was truncated for ${remotePath}: ${out.byteLength}/${totalBytes} bytes","messagePattern":"Remote file read was truncated for (.+?): (.+?)/(.+?) bytes","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/adapter-utils/src/command-managed-runtime.ts","lineNumber":315,"sourceCode":"      const decodedChunks: Buffer[] = [];\n      let decodedSoFar = 0;\n      if (totalBytes === 0) {\n        await options?.onProgress?.(0, 0);\n        return Buffer.alloc(0);\n      }\n      for (let chunkIndex = 0; decodedSoFar < totalBytes; chunkIndex++) {\n        const result = await runShell(\n          `dd if=${shellQuote(remotePath)} bs=${REMOTE_READ_CHUNK_BYTES} skip=${chunkIndex} count=1 2>/dev/null | base64`,\n        );\n        const chunk = Buffer.from(result.stdout.replace(/\\s+/g, \"\"), \"base64\");\n        if (chunk.byteLength === 0) break;\n        decodedChunks.push(chunk);\n        decodedSoFar += chunk.byteLength;\n        await options?.onProgress?.(Math.min(decodedSoFar, totalBytes), totalBytes);\n      }\n      const out = Buffer.concat(decodedChunks);\n      if (out.byteLength !== totalBytes) {\n        throw new Error(`Remote file read was truncated for ${remotePath}: ${out.byteLength}/${totalBytes} bytes`);\n      }\n      await options?.onProgress?.(out.byteLength, totalBytes);\n      return out;\n    },\n    listFiles: async (remotePath) => {\n      const result = await runShell(\n        `if [ -d ${shellQuote(remotePath)} ]; then ` +\n          `for entry in ${shellQuote(remotePath)}/*; do ` +\n          `[ -f \"$entry\" ] || continue; ` +\n          `basename \"$entry\"; ` +\n          `done; ` +\n        `fi`,\n      );\n      return result.stdout\n        .split(/\\r?\\n/)\n        .map((entry) => entry.trim())\n        .filter((entry) => entry.length > 0)\n        .sort((left, right) => left.localeCompare(right));","sourceCodeStart":297,"sourceCodeEnd":333,"githubUrl":"https://github.com/paperclipai/paperclip/blob/67001ec6eb96ae601aa27bc91d9b2415d665334a/packages/adapter-utils/src/command-managed-runtime.ts#L297-L333","documentation":"Thrown by the readFile method of the command-managed runtime client when the total decoded bytes from the chunked dd|base64 read loop do not equal the byte count reported by the initial 'wc -c' size probe. The loop reads REMOTE_READ_CHUNK_BYTES at a time via dd, base64-encodes each chunk, and accumulates decoded bytes. If the file was truncated, deleted, or modified during the read, or if dd/base64 produced fewer bytes than expected, this integrity check fails.","triggerScenarios":"Calling client.readFile(remotePath) when the file is modified, truncated, or deleted between the 'wc -c' size probe and the chunked dd reads. Also triggered if dd or base64 in the sandbox produces fewer bytes than expected (e.g., dd hitting a read error mid-file, or the sandbox killing the process).","commonSituations":"Concurrent file writes or truncation during a read (TOCTOU race). The sandbox has resource limits that kill long-running dd/base64 pipelines. The file is on a network filesystem with intermittent failures. A large file read where one chunk fails silently. The file is a log being actively rotated/truncated.","solutions":["Retry the read if the failure is transient (file being written to concurrently).","Ensure no other process is modifying, truncating, or deleting the file during the read operation."],"exampleFix":"// before: reading a file that is being actively written\nawait client.readFile('/workspace/app/runtime.log'); // truncated during read\n\n// after: snapshot the file first, then read the snapshot\nawait client.run(`cp ${shellQuote(remotePath)} ${shellQuote(remotePath + '.snapshot')}`);\nconst data = await client.readFile(remotePath + '.snapshot');\nawait client.remove(remotePath + '.snapshot');","handlingStrategy":"retry","validationCode":"async function isRemoteFileStable(client: SandboxManagedRuntimeClient, remotePath: string): Promise<boolean> {\n  try {\n    const size1 = await client.run(`wc -c < ${shellQuote(remotePath)}`);\n    const size2 = await client.run(`wc -c < ${shellQuote(remotePath)}`);\n    return size1.stdout.trim() === size2.stdout.trim();\n  } catch {\n    return false;\n  }\n}\n\n// Call before client.readFile for files that might be actively written:\nif (!(await isRemoteFileStable(client, remotePath))) {\n  // Snapshot the file first\n  await client.run(`cp ${shellQuote(remotePath)} ${shellQuote(remotePath + '.snapshot')}`);\n}","typeGuard":null,"tryCatchPattern":"try {\n  const data = await client.readFile(remotePath);\n} catch (error) {\n  if (error instanceof Error && error.message.includes('truncated')) {\n    // File was modified during read; snapshot it first and retry\n    await client.run(`cp ${shellQuote(remotePath)} ${shellQuote(remotePath + '.snapshot')}`);\n    const data = await client.readFile(remotePath + '.snapshot');\n    await client.remove(remotePath + '.snapshot');\n    return data;\n  }\n  throw error;\n}","preventionTips":["Ensure no other process writes to, truncates, or deletes the file during a readFile operation.","Retry readFile on transient truncation failures—if the file is static, the second read should succeed.","On provider-backed sandboxes with strict resource limits, read smaller files or increase the sandbox's memory/process limits.","Consider using client.run with a single 'cat | base64' for small files to avoid the multi-chunk dd pipeline."],"tags":["runtime","sandbox","file-read","truncation","adapter-utils","race-condition"],"backgroundTag":null,"analyzedSha":"67001ec6eb96ae601aa27bc91d9b2415d665334a","analyzedAt":"2026-08-12T12:05:45.408Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}