apify/crawlee · error · Error

The state object is invalid: nextIndex must be a non-negativ

Error message

The state object is invalid: nextIndex must be a non-negative number.

What it means

When a persisted state is restored, nextIndex (the pointer into the request list) must be a valid non-negative number. A persisted state with a missing, non-numeric, or negative nextIndex is considered corrupt, and restoreState() throws during initialize().

Source

Thrown at packages/core/src/storages/request_list.ts:504

     * are automatically persisted at RequestList initialization (if the persistRequestsKey is set),
     * but there's no reason to persist it again afterwards, because RequestList is immutable.
     */
    private async persistRequests(): Promise<void> {
        const serializedRequests = await serializeArray(this.requests);
        this.#store ??= await KeyValueStore.open();
        await this.#store.setValue(this.#persistRequestsKey!, serializedRequests, { contentType: CONTENT_TYPE_BINARY });
        this.areRequestsPersisted = true;
    }

    /**
     * Restores RequestList state from a state object.
     */
    private restoreState(state?: RequestListState): void {
        // If there's no state it means we've not persisted any (yet).
        if (!state) return;
        // Restore previous state.
        if (typeof state.nextIndex !== 'number' || state.nextIndex < 0) {
            throw new Error('The state object is invalid: nextIndex must be a non-negative number.');
        }
        if (state.nextIndex > this.requests.length) {
            throw new Error('The state object is not consistent with RequestList, too few requests loaded.');
        }
        if (
            state.nextIndex < this.requests.length &&
            this.requests[state.nextIndex].uniqueKey !== state.nextUniqueKey
        ) {
            throw new Error(
                'The state object is not consistent with RequestList the order of URLs seems to have changed.',
            );
        }

        const deleteFromInProgress: string[] = [];
        state.inProgress.forEach((uniqueKey) => {
            const index = this.#uniqueKeyToIndex[uniqueKey];
            if (typeof index !== 'number') {
                throw new Error(

View on GitHub (pinned to dbe57fb09c)

Solutions

1. Delete the corrupted persisted state record (the CRAWLEE_<persistStateKey> entry in the default KeyValueStore) so a fresh state is created
2. Verify the state was written by the same crawlee version; migrate or clear on version upgrade
3. Wrap initialization to detect this error and fall back to starting without persisted state

Example fix

// before
// corrupted state { nextIndex: -1 } restored from KV store
// after
const store = await KeyValueStore.open();
await store.setValue('CRAWLEE_list-state', null); // drop corrupted state, list restarts fresh

function isValidState(state) {
  return state == null || (typeof state.nextIndex === 'number' && Number.isInteger(state.nextIndex) && state.nextIndex >= 0);
}
const stored = await store.getValue('CRAWLEE_list-state');
if (!isValidState(stored)) await store.setValue('CRAWLEE_list-state', null); // drop corrupt state before open

const isValidRequestListState = (s) => s == null || (typeof s?.nextIndex === 'number' && s.nextIndex >= 0);

try {
  const list = await RequestList.open('list', { sources, persistStateKey: 'state' });
} catch (e) {
  if (e.message.includes('nextIndex must be a non-negative number')) {
    const store = await KeyValueStore.open();
    await store.setValue('CRAWLEE_state', null); // reset and retry once
    return RequestList.open('list', { sources, persistStateKey: 'state' });
  }
  throw e;
}

When it happens

Trigger: The KeyValueStore record under the persistStateKey contains malformed data — manually edited, written by a different/incompatible version, truncated, or null/absent fields after deserialization.

Common situations: Manual edits of the key-value store between runs; restoring state persisted by an older crawlee version with a different state schema; corrupted storage after a crashed write.

Related errors

• The state object is not consistent with RequestList, too few
• StateValidationError(persistStateKey, result.issues)
• At least one of "sources" or "sourcesFunction" must be provi
• The state object is not consistent with RequestList the orde
• The state object is not consistent with RequestList. Unknown

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/ec3041004e9436f0. Report an issue: GitHub.