ruby-concurrency/concurrent-ruby · critical · OutOfMemoryError

Required array size too large

Error message

Required array size too large

What it means

Thrown by the JSR166e ConcurrentHashMapV8 backport that concurrent-ruby bundles under ext/ for JRuby. The map's collection views (keySet/values/entrySet) implement Object[] toArray() by reading map.mappingCount() and refusing to allocate a result array larger than MAX_ARRAY_SIZE (Integer.MAX_VALUE - 8), since JVM arrays cannot exceed that. The pre-check at this line fires when the live mapping count already exceeds the ceiling before any element is copied.

Source

Thrown at ext/concurrent-ruby/com/concurrent_ruby/ext/jsr166e/nounsafe/ConcurrentHashMapV8.java:3514

         * @return the map backing this view
         */
        public ConcurrentHashMapV8<K,V> getMap() { return map; }

        public final int size()                 { return map.size(); }
        public final boolean isEmpty()          { return map.isEmpty(); }
        public final void clear()               { map.clear(); }

        // implementations below rely on concrete classes supplying these
        abstract public Iterator<?> iterator();
        abstract public boolean contains(Object o);
        abstract public boolean remove(Object o);

        private static final String oomeMsg = "Required array size too large";

        public final Object[] toArray() {
            long sz = map.mappingCount();
            if (sz > (long)(MAX_ARRAY_SIZE))
                throw new OutOfMemoryError(oomeMsg);
            int n = (int)sz;
            Object[] r = new Object[n];
            int i = 0;
            Iterator<?> it = iterator();
            while (it.hasNext()) {
                if (i == n) {
                    if (n >= MAX_ARRAY_SIZE)
                        throw new OutOfMemoryError(oomeMsg);
                    if (n >= MAX_ARRAY_SIZE - (MAX_ARRAY_SIZE >>> 1) - 1)
                        n = MAX_ARRAY_SIZE;
                    else
                        n += (n >>> 1) + 1;
                    r = Arrays.copyOf(r, n);
                }
                r[i++] = it.next();
            }
            return (i == n) ? r : Arrays.copyOf(r, i);
        }

View on GitHub (pinned to 0b88d5ff75)

Solutions

1. Do not materialize the view: iterate it lazily via its Java iterator (view.iterator) instead of view.to_a.
2. Fix the unbounded growth: add eviction/expiry or switch to a bounded cache so the map stays far below Integer.MAX_VALUE.
3. If a snapshot is truly needed, copy in bounded batches (iterate and append into chunks) or move data to an external store.
4. Profile what inserts into the map - a 2-billion-entry map is almost always a defect (missing dedup, duplicate keys, or a tight insert loop).

Example fix

# before
keys = java_map.keySet.to_a   # OutOfMemoryError: Required array size too large

# after
it = java_map.keySet.iterator
while it.hasNext
  process(it.next)
end

begin
  arr = view.to_a
rescue Java::JavaLang::OutOfMemoryError => e
  raise unless e.message == 'Required array size too large'
  it = view.iterator                 # fall back to lazy iteration
  while it.hasNext
    handle(it.next)
  end
end

When it happens

Trigger: On JRuby, calling .to_a / toArray() on a keySet, values, or entrySet view of a Java-backed ConcurrentHashMapV8 (for example the Java extension used by Concurrent::Map or a jruby-specific cache class) while the map holds more than Integer.MAX_VALUE - 8 (2,147,483,639) mappings.

Common situations: In practice this signals runaway map growth, not a transient condition: an unbounded cache or counter map in a long-lived process crossing ~2.1 billion entries, a producer loop inserting without eviction, or a bulk-import bug duplicating keys. Normal applications never approach this threshold.

Related errors

• Required array size too large
• Could not initialize intrinsics
• Could not initialize intrinsics
• no block given
• #{@fallback_policy} is not a valid fallback policy

AI-assisted analysis of ruby-concurrency/concurrent-ruby@0b88d5ff75 (2026-08-21). Data as JSON: /api/errors/940ee5b4db692732. Report an issue: GitHub.