ruby-concurrency/concurrent-ruby · critical · OutOfMemoryError

Required array size too large

Error message

Required array size too large

What it means

CHMView.toArray() is the shared implementation behind keySet().toArray(), values().toArray() and entrySet().toArray(). It sizes the result from map.mappingCount() and throws OutOfMemoryError("Required array size too large") up front when the count exceeds MAX_ARRAY_SIZE (Integer.MAX_VALUE - 8), because no Java array can hold more references. It is a fail-fast size guard, not a report of actual heap exhaustion.

Source

Thrown at ext/concurrent-ruby/com/concurrent_ruby/ext/jsr166e/ConcurrentHashMapV8.java:3522

         * @return the map backing this view
         */
        public ConcurrentHashMapV8<K,V> getMap() { return map; }

        public final int size()                 { return map.size(); }
        public final boolean isEmpty()          { return map.isEmpty(); }
        public final void clear()               { map.clear(); }

        // implementations below rely on concrete classes supplying these
        abstract public Iterator<?> iterator();
        abstract public boolean contains(Object o);
        abstract public boolean remove(Object o);

        private static final String oomeMsg = "Required array size too large";

        public final Object[] toArray() {
            long sz = map.mappingCount();
            if (sz > (long)(MAX_ARRAY_SIZE))
                throw new OutOfMemoryError(oomeMsg);
            int n = (int)sz;
            Object[] r = new Object[n];
            int i = 0;
            Iterator<?> it = iterator();
            while (it.hasNext()) {
                if (i == n) {
                    if (n >= MAX_ARRAY_SIZE)
                        throw new OutOfMemoryError(oomeMsg);
                    if (n >= MAX_ARRAY_SIZE - (MAX_ARRAY_SIZE >>> 1) - 1)
                        n = MAX_ARRAY_SIZE;
                    else
                        n += (n >>> 1) + 1;
                    r = Arrays.copyOf(r, n);
                }
                r[i++] = it.next();
            }
            return (i == n) ? r : Arrays.copyOf(r, i);
        }

View on GitHub (pinned to 0b88d5ff75)

Solutions

1. Check map.mappingCount() <= Integer.MAX_VALUE - 8 before calling toArray(); if over, do not materialize
2. Stream instead of snapshot: view.forEach(k -> ...) processes elements without a backing array
3. Batch with the iterator into fixed-size chunks when arrays are genuinely needed
4. If you truly need more than 2^31 elements, shard across multiple maps — no single Java array can hold them

Example fix

// before
Object[] keys = map.keySet().toArray();  // OutOfMemoryError: Required array size too large

// after
if (map.mappingCount() <= Integer.MAX_VALUE - 8L) {
    Object[] keys = map.keySet().toArray();
} else {
    map.keySet().forEach(k -> consume(k)); // never materialize
}

static final long MAX_ARRAY = Integer.MAX_VALUE - 8L;
if (map.mappingCount() > MAX_ARRAY) {
    map.keySet().forEach(k -> consume(k)); // stream, no array
} else {
    Object[] snapshot = map.keySet().toArray();
}

When it happens

Trigger: map.keySet().toArray(), map.values().toArray(), map.entrySet().toArray(), or constructors that copy collections such as new ArrayList<>(map.keySet()) when map.mappingCount() exceeds Integer.MAX_VALUE - 8.

Common situations: Multi-billion-entry caches or counters on very large 64-bit heaps; passing views to helper APIs that materialize arrays; load and stress tests with huge key spaces.

Related errors

• Required array size too large
• Could not initialize intrinsics
• Could not initialize intrinsics
• unbuffered channels cannot have a capacity
• capacity must be at least 1 for this buffer type

AI-assisted analysis of ruby-concurrency/concurrent-ruby@0b88d5ff75 (2026-08-21). Data as JSON: /api/errors/5b4e5b6ea84c1183. Report an issue: GitHub.