microsoft/FASTER · error · FasterException

Byte array provided has invalid length

Error message

Byte array provided has invalid length

What it means

When iterating, the caller supplied a getMemory allocator delegate whose returned buffer is shorter than the record length being read. The iterator (FasterLogIterator.cs:302, reachable from GetNext/GetAsyncEnumerable) suspends the epoch and throws because the record would be truncated.

Solutions

1. Make the getMemory delegate honor the requested length: allocate at least entryLength bytes or grow/retrip the buffer.
2. Use ArrayPool with exact rentals: ArrayPool<byte>.Shared.Rent(entryLength) and handle the oversize array, or verify Length >= entryLength.
3. Remove fixed-size pre-allocated buffers from the allocator path.
4. Fall back to the default heap allocation by not passing a custom allocator.

Example fix

// before
memory => fixedBuffer // Length may be < entryLength

// after
memory => ArrayPool<byte>.Shared.Rent(entryLength) // always >= entryLength

byte[] GetMemory(int length) => length <= maxSupportedRecordSize ? ArrayPool<byte>.Shared.Rent(length) : throw new ArgumentException($"Record length {length} exceeds supported size");

try { iter.GetNext(out var entry, out var addr); } catch (FasterException ex) when (ex.Message == "Byte array provided has invalid length") { /* fix allocator delegate */ }

When it happens

Trigger: Passing an allocator Func<int, byte[]>/IMemoryOwner factory that returns pooled or pre-sized buffers smaller than the record length; iterating records written with a larger payload schema than the current allocator expects.

Common situations: Using a fixed-size buffer pool without honoring the requested length argument; schema/version change where new larger records are read by old reader code; pooling bugs returning zero-length arrays.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Cannot use named iterators with read-only FasterLog
• Cannot use scanUncommitted with read-only FasterLog
• Cannot use scanUncommitted without setting…
• Entry does not fit on page
• Invalid checksum found during scan, skipping

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/89947791f7ffe6dc. Report an issue: GitHub.

Appendix: source

Thrown at cs/src/core/FasterLog/FasterLogIterator.cs:302

                    FasterLogRecoveryInfo info = new();
                    info.Initialize(new BinaryReader(new UnmanagedMemoryStream((byte*) (headerSize + physicalAddress), entryLength)));
                    if (info.CommitNum != long.MaxValue) continue;
                    
                    // Otherwise, no more entries
                    entry = default;
                    entryLength = default;
                    epoch.Suspend();
                    return false;
                }
                
                if (getMemory != null)
                {
                    // Use user delegate to allocate memory
                    entry = getMemory(entryLength);
                    if (entry.Length < entryLength)
                    {
                        epoch.Suspend();
                        throw new FasterException("Byte array provided has invalid length");
                    }
                }
                else
                {
                    // We allocate a byte array from heap
                    entry = new byte[entryLength];
                }

                fixed (byte* bp = entry)
                    Buffer.MemoryCopy((void*) (headerSize + physicalAddress), bp, entryLength, entryLength);
                
                epoch.Suspend();
                return true;
            }
        }

        /// <summary>
        /// GetNext supporting memory pools

View on GitHub (pinned to 321d872eab)