FlowiseAI/Flowise · error · Error

Failed to download file ${key} from S3 bucket ${bucketName}:

Error message

Failed to download file ${key} from S3 bucket ${bucketName}: ${e.message}

What it means

Thrown per-object inside S3Directory's Promise.all when downloading a single key fails. Wraps the underlying S3 SDK / filesystem error (network, NoSuchKey, permissions, stream error, or 'Response body is not a readable stream') and interpolates its message. Because it throws inside Promise.all, the first failing key aborts the whole batch.

Source

Thrown at packages/components/nodes/documentloaders/S3Directory/S3Directory.ts:219

                        const objectData = await new Promise<Buffer>((resolve, reject) => {
                            const chunks: Buffer[] = []

                            if (response.Body instanceof Readable) {
                                response.Body.on('data', (chunk: Buffer) => chunks.push(chunk))
                                response.Body.on('end', () => resolve(Buffer.concat(chunks)))
                                response.Body.on('error', reject)
                            } else {
                                reject(new Error('Response body is not a readable stream.'))
                            }
                        })

                        // create the directory if it doesnt already exist
                        fsDefault.mkdirSync(path.dirname(filePath), { recursive: true })

                        // write the file to the directory
                        fsDefault.writeFileSync(filePath, objectData)
                    } catch (e: any) {
                        throw new Error(`Failed to download file ${key} from S3 bucket ${bucketName}: ${e.message}`)
                    }
                })
            )

            const loader = new DirectoryLoader(
                tempDir,
                {
                    '.json': (path) => new JSONLoader(path),
                    '.txt': (path) => new TextLoader(path),
                    '.csv': (path) => new CSVLoader(path),
                    '.xls': (path) => new LoadOfSheet(path),
                    '.xlsx': (path) => new LoadOfSheet(path),
                    '.xlsm': (path) => new LoadOfSheet(path),
                    '.xlsb': (path) => new LoadOfSheet(path),
                    '.docx': (path) => new DocxLoader(path),
                    '.ppt': (path) => new PowerpointLoader(path),
                    '.pptx': (path) => new PowerpointLoader(path),
                    '.pdf': (path) =>

View on GitHub (pinned to abe4a8601a)

Solutions

1. Confirm the IAM credential has s3:GetObject on the bucket/prefix.
2. Check the key still exists (List then Get can race for mutable buckets) — re-list or skip missing keys.
3. Verify tempDir is writable and has free space.
4. Make the per-key failure non-fatal by collecting errors instead of throwing inside Promise.all (so one bad key does not abort the batch).
5. Ensure the AWS SDK v3 version matches the streaming shape expected (Readable check).

Example fix

// before - one bad key aborts the whole batch
await Promise.all(keys.map(async (key) => {
  try { /* download */ } catch (e) {
    throw new Error(`Failed to download file ${key} from S3 bucket ${bucketName}: ${e.message}`)
  }
}))
// after - collect failures, continue, surface them at the end
const failures: { key: string; message: string }[] = []
await Promise.all(keys.map(async (key) => {
  try { /* download */ }
  catch (e: any) { failures.push({ key, message: e.message }) }
}))
if (failures.length) options.logger.warn(`S3 partial failure: ${JSON.stringify(failures)}`)

// Verify IAM can list+get before the batch run
async function verifyS3Access(s3Client, bucket) {
  await s3Client.send(new (require('@aws-sdk/client-s3').ListObjectsV2Command)({ Bucket: bucket, MaxKeys: 1 }))
}

const { Readable } = require('stream')
function isReadable(body) { return body instanceof Readable }

// Make per-key failure non-fatal; collect and continue
const failures = []
await Promise.all(keys.map(async (key) => {
  try { /* download key */ }
  catch (e) { failures.push({ key, message: e.message }) }
}))
if (failures.length) throw new Error(`S3 partial download failure: ${JSON.stringify(failures)}`)

When it happens

Trigger: GetObject returns NoSuchKey (key deleted between List and Get); access denied (bucket policy/IAM denies s3:GetObject); socket/stream 'error' event fires mid-download; response.Body is not a Readable (SDK v3 streaming shape change); disk full or permission error on fsDefault.writeFileSync.

Common situations: Bucket policy tightened mid-run; very large object timing out the stream; IAM role missing s3:GetObject; tempDir filesystem read-only or out of space; AWS SDK v3 version where Body is a Blob/Readable-Native mismatch.

Related errors

• Failed to download file ${keyName} from S3 bucket ${bucketNa
• Failed to load data from bucket ${bucketName}: ${e.message}
• Failed to load S3 document: ${error.message}
• Failed to load file ${filePath} using unstructured loader.
• Unsupported binary file type: ${mimeType}

AI-assisted analysis of FlowiseAI/Flowise@abe4a8601a (2026-08-12). Data as JSON: /api/errors/6026a1e7bd981323. Report an issue: GitHub.