What causes " Out of memory in Indx256::IndexString! " in dotnet/runtime?

Reduce the number of distinct labels/symbols in the input (split methods/assemblies).

Out of memory in Indx256::IndexString!

Error message

Out of memory in Indx256::IndexString!

What it means

Printed by Indx256::IndexStringOneTable in asmtemplates.h when 'new Indx256[INDX256_TABLE_SIZE] {}' returns NULL. The function also fires an _ASSERTE (debug break in chk builds) and returns NULL, propagating NULL up IndexString. Indx256 is a 256-way trie used to index named labels/symbols in ilasm.

Source

Thrown at src/coreclr/ilasm/asmtemplates.h:234

    void ClearAll(bool DeleteObj)
    {
        if(DeleteObj) delete item;
        item = NULL;
        ClearOneTable(tableLow, DeleteObj);
        ClearOneTable(tableHigh, DeleteObj);
    };

private:
    T** IndexStringOneTable(Indx256*& table, BYTE value, BYTE* next, T* pObj)
    {
        // Ensure that child table exists.
        if(table == NULL)
        {
            table = new Indx256[INDX256_TABLE_SIZE] {};
            if(table == NULL)
            {
                _ASSERTE(!"Out of memory in Indx256::IndexString!");
                fprintf(stderr,"\nOut of memory in Indx256::IndexString!\n");
                return NULL;
            }
        }

        // Find the child node for the current BYTE at continue at the next BYTE.
        return table[value].IndexString(next,pObj);
    }

    T* FindStringOneTable(Indx256* table, BYTE value, BYTE* next)
    {
        if(table == NULL)
        {
            // If there are no child nodes, then there is nowhere to
            // look for this key.
            return NULL;
        }

        return table[value].FindString(next);

View on GitHub (pinned to 290d5ab72c)

Solutions

Reduce the number of distinct labels/symbols in the input (split methods/assemblies).
Use 64-bit ilasm with higher memory.
If patching, replace the dense Indx256 with a hashtable (the header comment suggests this), which would drastically cut memory.
Raise process memory limits.

Defensive patterns

Strategy: validation

Validate before calling

# the trie is memory-heavy; limit distinct label count by splitting methods
LABELS=$(grep -cE '^[[:space:]]*IL_[0-9a-f]+$' big.il)
[ "$LABELS" -lt 1000000 ] || echo "~$LABELS distinct labels - Indx256 may exhaust memory"

Prevention

Reduce distinct label/symbol counts by splitting large methods or assemblies.
Use 64-bit ilasm with high memory limits.
If patching ilasm, consider replacing Indx256 with a hashtable per the header comment.

When it happens

Trigger: Adding a new unique key (label/symbol name) to the trie requires allocating a fresh 128-element child table; that allocation fails. Happens with extremely large numbers of distinct names (e.g. ildasm-style IL_<hex> labels from huge methods).

Common situations: Assembling IL generated from very large methods or many types; the trie is documented as memory-intensive; OOM under memory pressure or 32-bit.

Related errors

AI-assisted analysis of dotnet/runtime@290d5ab72c (2026-08-06). Data as JSON: /api/errors/854672a4d2bdb68c. Report an issue: GitHub.