{"record":{"id":"7ba3690191fae00c","repo":"apify/crawlee","slug":"exclusivestartkey-exclusivestartkey-was-not-f","errorCode":null,"errorMessage":"exclusiveStartKey \"${exclusiveStartKey}\" was not found in the key-value store. This is likely a bug — the key may have been deleted between paginated listKeys calls.","messagePattern":"exclusiveStartKey \"(.+?)\" was not found in the key-value store\\. This is likely a bug — the key may have been deleted between paginated listKeys calls\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"warning","filePath":"packages/core/src/memory-storage/resource-clients/key-value-store.ts","lineNumber":131,"sourceCode":"\n        for (const record of this.#keyValueEntries.values()) {\n            const size = Buffer.byteLength(record.value);\n            items.push({\n                key: record.key,\n                size,\n                contentType: record.contentType ?? 'application/octet-stream',\n            });\n        }\n\n        // Lexically sort to emulate API.\n        items.sort((a, b) => a.key.localeCompare(b.key));\n\n        let filteredItems = items.filter((item) => !prefix || item.key.startsWith(prefix));\n\n        if (exclusiveStartKey) {\n            const keyPos = filteredItems.findIndex((item) => item.key === exclusiveStartKey);\n            if (keyPos === -1) {\n                throw new Error(\n                    `exclusiveStartKey \"${exclusiveStartKey}\" was not found in the key-value store. ` +\n                        `This is likely a bug — the key may have been deleted between paginated listKeys calls.`,\n                );\n            }\n            filteredItems = filteredItems.slice(keyPos + 1);\n        }\n\n        const isTruncated = limit !== undefined && filteredItems.length > limit;\n        const pageItems = isTruncated ? filteredItems.slice(0, limit) : filteredItems;\n        const nextExclusiveStartKey = isTruncated ? pageItems[pageItems.length - 1].key : undefined;\n\n        this.updateTimestamps(false);\n\n        return {\n            items: pageItems,\n            count: pageItems.length,\n            limit: limit ?? pageItems.length,\n            exclusiveStartKey,","sourceCodeStart":113,"sourceCodeEnd":149,"githubUrl":"https://github.com/apify/crawlee/blob/dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c/packages/core/src/memory-storage/resource-clients/key-value-store.ts#L113-L149","documentation":"The memory-storage key-value store's listKeys() supports pagination via exclusiveStartKey. If that key is no longer found among filtered items, the library assumes it was deleted between paginated calls and throws, since continuing pagination would be incorrect.","triggerScenarios":"Paginating listKeys() where the record at exclusiveStartKey was deleted (or renamed) between calls, or where a prefix filter excludes the previously returned start key.","commonSituations":"Long-running crawlers deleting keys while another process lists pages; using a start key obtained with a different prefix; concurrent writers trimming the store during migration.","solutions":["Restart listing from the beginning (no exclusiveStartKey) after catching the error","Freeze deletes during pagination or retry the whole listing snapshot","Ensure the exclusiveStartKey and prefix are taken from the same listing call","Serialize writers so keys are not removed mid-pagination"],"exampleFix":"// before\nconst page2 = await store.listKeys({ prefix, exclusiveStartKey: page1.nextExclusiveStartKey });\n// after\nlet page2; try { page2 = await store.listKeys({ prefix, exclusiveStartKey: page1.nextExclusiveStartKey }); } catch { page2 = await store.listKeys({ prefix }); }","handlingStrategy":"retry","validationCode":"const first = await store.listKeys({ prefix });\nif (first.items.length === 0) return []; // nothing to paginate","typeGuard":null,"tryCatchPattern":"try { return await store.listKeys({ prefix, exclusiveStartKey }); } catch (e) { if (String(e).includes('was not found in the key-value store')) return restartListingFromBeginning(); throw e; }","preventionTips":["Avoid deleting keys while paginating","Always pair exclusiveStartKey with the same prefix used to obtain it","Restart listing from scratch when pagination breaks"],"tags":["key-value-store","pagination","memory-storage"],"backgroundTag":"pagination-key-missing","analyzedSha":"dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c","analyzedAt":"2026-08-30T22:22:28.328Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}