pola-rs/polars · error

could not allocate memory for CPUID check

Error message

could not allocate memory for CPUID check

What it means

At import time on Windows, polars allocates a small executable page via VirtualAlloc (MEM_COMMIT|MEM_RESERVE, PAGE_EXECUTE_READWRITE) to hold a CPUID stub that detects CPU features (sse4.2, avx2, ...) before choosing/validating the Rust runtime variant. This MemoryError means VirtualAlloc returned NULL: the OS refused to reserve/commit the page, almost always because commit charge (RAM + pagefile) is exhausted or a per-process/job commit limit was reached.

Source

Thrown at py-polars/src/polars/_cpu_check.py:176

            opc = _POSIX_64_OPC if _IS_64BIT else _CDECL_32_OPC

        size = len(opc)
        code = (ctypes.c_ubyte * size)(*opc)

        if _IS_WINDOWS:
            self.win.VirtualAlloc.restype = c_void_p
            self.win.VirtualAlloc.argtypes = [
                ctypes.c_void_p,
                ctypes.c_size_t,
                ctypes.c_ulong,
                ctypes.c_ulong,
            ]
            self.addr = self.win.VirtualAlloc(
                None, size, _MEM_COMMIT | _MEM_RESERVE, _PAGE_EXECUTE_READWRITE
            )
            if not self.addr:
                msg = "could not allocate memory for CPUID check"
                raise MemoryError(msg)
            ctypes.memmove(self.addr, code, size)
        else:
            import mmap  # Only import if necessary.

            # On some platforms PROT_WRITE + PROT_EXEC is forbidden, so we first
            # only write and then mprotect into PROT_EXEC.
            libc = _open_posix_libc()
            mprotect = libc.mprotect
            mprotect.argtypes = (ctypes.c_void_p, ctypes.c_size_t, ctypes.c_int)
            mprotect.restype = ctypes.c_int

            self.mmap = mmap.mmap(
                -1,
                size,
                mmap.MAP_PRIVATE | mmap.MAP_ANONYMOUS,
                mmap.PROT_READ | mmap.PROT_WRITE,
            )
            self.addr = ctypes.addressof(ctypes.c_void_p.from_buffer(self.mmap))

View on GitHub (pinned to df599052da)

Solutions

1. Free memory or restart the process before importing polars
2. Increase the Windows pagefile (or set it to system-managed) so commit charge is available
3. Raise the commit limit of the job object/container the process runs under
4. As a workaround, set `POLARS_SKIP_CPU_CHECK=1` before importing polars — the CPUID probe (and thus the allocation) is skipped entirely, at the risk of running an incompatible runtime

Example fix

# before
import polars as pl  # MemoryError: could not allocate memory for CPUID check

# after
import os
os.environ["POLARS_SKIP_CPU_CHECK"] = "1"  # skip the CPUID allocation
import polars as pl

import ctypes

class MEMORYSTATUSEX(ctypes.Structure):
    _fields_ = [("dwLength", ctypes.c_ulong), ("dwMemoryLoad", ctypes.c_ulong),
                ("ullTotalPhys", ctypes.c_uint64), ("ullAvailPhys", ctypes.c_uint64),
                ("ullTotalPageFile", ctypes.c_uint64), ("ullAvailPageFile", ctypes.c_uint64),
                ("ullTotalVirtual", ctypes.c_uint64), ("ullAvailVirtual", ctypes.c_uint64),
                ("ullAvailExtendedVirtual", ctypes.c_uint64)]

st = MEMORYSTATUSEX(); st.dwLength = ctypes.sizeof(MEMORYSTATUSEX)
ctypes.windll.kernel32.GlobalMemoryStatusEx(ctypes.byref(st))
if st.ullAvailPageFile < 64 * 1024 * 1024:  # < 64 MB commit available
    import os; os.environ["POLARS_SKIP_CPU_CHECK"] = "1"

import sys
try:
    import polars as pl
except MemoryError:
    # CPUID probe could not allocate its page; retry once without the probe
    import os
    os.environ["POLARS_SKIP_CPU_CHECK"] = "1"
    for m in [m for m in sys.modules if m.startswith("polars")]:
        del sys.modules[m]
    import polars as pl  # noqa

When it happens

Trigger: `import polars` on Windows while the system is out of committable memory: pagefile disabled or capped, a memory-hungry parent process, or a job-object commit limit (CI runners, containers on Windows). The allocation happens in CPUID.__init__ during check_cpu_flags, before the Rust runtime is selected.

Common situations: Windows CI agents with small commit limits; machines with pagefile disabled for 'performance'; long-running processes that leaked memory before first importing polars; batch jobs launched after several heavy processes filled the commit limit.

Related errors

• could not execute mprotect for CPUID check
• unknown feature flag: {f!r}
• Polars Rust module for '{_force}' ({sys.modules[__name__].__
• Invalid value for `POLARS_FORCE_PKG` variable: '{_force}'
• Invalid value for `POLARS_PREFER_PKG` variable: '{_prefer}'

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/b78518813d2516d4. Report an issue: GitHub.