{"record":{"id":"d98a74977e9bcc79","repo":"Budibase/budibase","slug":"unable-to-retrieve-csv-invalid-stream","errorCode":null,"errorMessage":"Unable to retrieve CSV - invalid stream","messagePattern":"Unable to retrieve CSV - invalid stream","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/server/src/integrations/s3.ts","lineNumber":256,"sourceCode":"      Bucket: query.bucket,\n      Delimiter: query.delimiter,\n      Marker: query.marker,\n      MaxKeys: query.maxKeys,\n      Prefix: query.prefix,\n    })\n    return response.Contents\n  }\n\n  async readCsv(query: { bucket: string; key: string }) {\n    const response = await this.client.getObject({\n      Bucket: query.bucket,\n      Key: query.key,\n    })\n\n    const fileStream = response.Body?.transformToWebStream()\n\n    if (!fileStream || !(fileStream instanceof stream.Readable)) {\n      throw new Error(\"Unable to retrieve CSV - invalid stream\")\n    }\n\n    let csvError = false\n    return new Promise((resolve, reject) => {\n      fileStream.on(\"error\", (err: Error) => {\n        reject(err)\n      })\n      const response = csv()\n        .fromStream(fileStream)\n        .on(\"error\", () => {\n          csvError = true\n        })\n      fileStream.on(\"end\", () => {\n        resolve(response)\n      })\n    }).catch(err => {\n      if (csvError) {\n        throw new Error(\"Could not read CSV\")","sourceCodeStart":238,"sourceCodeEnd":274,"githubUrl":"https://github.com/Budibase/budibase/blob/a81a902e9a8fe55b467d106765f6638f12e35c49/packages/server/src/integrations/s3.ts#L238-L274","documentation":"readCsv in the S3 integration validates that the GetObject response body can be transformed into a Node.js Readable stream before parsing CSV. If response.Body is undefined or the transformToWebStream() result is not a stream.Readable, there is nothing to pipe the CSV from, so the integration throws this error instead of failing obscurely downstream.","triggerScenarios":"Calling the S3 read CSV query where GetObject returns no Body (object missing/deleted, no read permission yielding a response without body in some SDK paths) or transformToWebStream() returns a non-Readable value (SDK version mismatch, e.g. @aws-sdk/client-s3 v3 with wrong stream wiring).","commonSituations":"Specifying a wrong bucket/key so the object doesn't exist; IAM credentials lacking s3:GetObject; upgrading or mismatching AWS SDK versions where response.Body type changed; reading from a non-S3-compatible endpoint returning unexpected body types.","solutions":["Verify the object exists at the configured bucket/key (aws s3 ls s3://<bucket>/<key>) and that the key is correct in the query.","Check the S3 integration credentials/permissions have s3:GetObject on the object.","Ensure a supported @aws-sdk/client-s3 version is installed where response.Body implements transformToWebStream().","If using an S3-compatible store (MinIO etc.), confirm the endpoint returns standard GetObject responses."],"exampleFix":"// before: wrong key, object missing\nconst query = { bucket: \"my-bucket\", key: \"exports/data.csv\" }\n// after: verify key exists and credentials have GetObject\naws s3 ls s3://my-bucket/exports/data.csv  # then rerun the query with the confirmed key","handlingStrategy":"validation","validationCode":"import { HeadObjectCommand, S3Client } from \"@aws-sdk/client-s3\"\nconst s3 = new S3Client({})\nawait s3.send(new HeadObjectCommand({ Bucket: query.bucket, Key: query.key })) // throws if missing/no permission","typeGuard":"function hasReadableBody(res: { Body?: unknown }): res is { Body: { transformToWebStream(): ReadableStream } } {\n  return !!res.Body && typeof (res.Body as any).transformToWebStream === \"function\"\n}","tryCatchPattern":"try {\n  const csv = await s3Integration.readCsv(query)\n} catch (err) {\n  if (err instanceof Error && err.message === \"Unable to retrieve CSV - invalid stream\") {\n    // check bucket/key/permissions, then retry or surface config error\n  } else { throw err }\n}","preventionTips":["Validate bucket/key against HeadObject before reading","Grant s3:GetObject to the integration credentials","Keep @aws-sdk/client-s3 on a supported, tested version","Test S3-compatible endpoints (MinIO) for standard GetObject responses"],"tags":["aws","s3","csv","stream"],"backgroundTag":"s3-invalid-response-stream","analyzedSha":"a81a902e9a8fe55b467d106765f6638f12e35c49","analyzedAt":"2026-08-29T01:03:10.972Z","schemaVersion":2},"datasetVersion":"2026-08-29T02:17:18.158Z"}