{"record":{"id":"13bb7e51567807cd","repo":"apify/crawlee","slug":"loading-requests-with-sourcesfunction-failed-caus","errorCode":null,"errorMessage":"Loading requests with sourcesFunction failed.\nCause: ${err.message}","messagePattern":"Loading requests with sourcesFunction failed\\.\nCause: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/core/src/storages/request_list.ts","lineNumber":448,"sourceCode":"\n        // Drop the original array full of empty indexes.\n        this.#sources = [];\n\n        if (this.#sourcesFunction) {\n            try {\n                const sourcesFromFunction = await this.#sourcesFunction();\n                const sourcesFromFunctionCount = sourcesFromFunction.length;\n                for (let i = 0; i < sourcesFromFunctionCount; i++) {\n                    const source = sourcesFromFunction[i];\n                    // oxlint-disable-next-line typescript/no-array-delete -- intentional, drop the slot so V8 can collect the object\n                    delete sourcesFromFunction[i];\n                    this.addRequest(source);\n                }\n\n                sourcesFromFunction.length = 0;\n            } catch (e) {\n                const err = e as Error;\n                throw new Error(`Loading requests with sourcesFunction failed.\\nCause: ${err.message}`);\n            }\n        }\n    }\n\n    /**\n     * @inheritDoc\n     */\n    async persistState(): Promise<void> {\n        if (!this.#persistStateKey) {\n            throw new Error('Cannot persist state. options.persistStateKey is not set.');\n        }\n        if (this.isStatePersisted) return;\n        try {\n            this.#store ??= await KeyValueStore.open();\n            await this.#store.setValue(this.#persistStateKey, this.getState());\n            this.isStatePersisted = true;\n        } catch (e) {\n            const err = e as Error;","sourceCodeStart":430,"sourceCodeEnd":466,"githubUrl":"https://github.com/apify/crawlee/blob/dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c/packages/core/src/storages/request_list.ts#L430-L466","documentation":"When sourcesFunction is invoked during initialization and it throws (network failure, bad fetch, parse error), the library cancels initialization and rethrows the cause wrapped in this prefixed error so you can tell which phase failed.","triggerScenarios":"sourcesFunction returns a promise that rejects — e.g. fetching URLs from an API that returns 500, DNS failure, invalid response shape that throws while mapping, or a synchronous throw inside the function body.","commonSituations":"Loading URL lists from a Google Sheets export, internal API, or sitemap whose endpoint is down or rate-limiting; auth token expired for the sources API; deploying to an environment without network access to the source.","solutions":["Fix the underlying error reported after 'Cause:' (check network/endpoint/auth)","Make sourcesFunction defensive: retry fetching, validate the response before returning","Provide static sources as a fallback when the dynamic source is unavailable"],"exampleFix":"// before\nsourcesFunction: async () => (await fetch(url)).json(),\n// after\nsourcesFunction: async () => {\n    for (let i = 0; i < 3; i++) {\n        try {\n            const res = await fetch(url);\n            if (!res.ok) throw new Error(`HTTP ${res.status}`);\n            return res.json();\n        } catch (e) {\n            if (i === 2) throw e;\n            await new Promise((r) => setTimeout(r, 2000));\n        }\n    }\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  const list = await RequestList.open('list', { sourcesFunction: loadUrls });\n} catch (e) {\n  if (e.message.startsWith('Loading requests with sourcesFunction failed')) {\n    console.error('sourcesFunction failed:', e.message.split('Cause:')[1]);\n    // retry with backoff or fall back to static sources\n  } else throw e;\n}","preventionTips":["Wrap the fetch logic inside sourcesFunction with its own retries and timeouts","Validate the fetched payload shape before returning it as sources","Provide a static sources fallback for resilience"],"tags":["request-list","network","sources-function"],"backgroundTag":"sources-function-failed","analyzedSha":"dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c","analyzedAt":"2026-08-30T22:22:28.328Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}