dotnet/machinelearning · error · EndOfStreamException

Unable to read beyond the end of the stream.

Error message

Unable to read beyond the end of the stream.

What it means

ReadAtLeastCore reads repeatedly until the requested minimum bytes are read; if the stream returns 0 bytes (end of stream) and throwOnEndOfStream is true, it throws EndOfStreamException('Unable to read beyond the end of the stream.'), matching Stream.ReadExactly semantics.

Solutions

  1. Verify the stream length/remaining bytes >= requested count before calling ReadExactly, or use the ReadAtLeast variant with throwOnEndOfStream:false
  2. Reset the stream position (stream.Position = 0) if the data was already read
  3. Check data integrity: the source may be truncated or the format header lies about sizes

Example fix

// before
StreamUtils.ReadExactly(stream, buffer, 0, expectedSize); // throws on short stream
// after
if (stream.Length - stream.Position < expectedSize)
    throw new InvalidDataException($"Expected {expectedSize} bytes, only {stream.Length - stream.Position} available");
StreamUtils.ReadExactly(stream, buffer, 0, expectedSize);
Defensive patterns

Strategy: try-catch

Validate before calling

bool canReadExactly = stream.CanSeek == false || (stream.Length - stream.Position) >= expectedBytes;

Try / catch

try { StreamUtils.ReadExactly(stream, buffer, 0, expectedBytes); }
catch (EndOfStreamException ex) when (ex.Message.Contains("Unable to read beyond the end"))
{ // handle truncated input: abort, resync format parser, or re-download
    throw new InvalidDataException("Stream ended before expected bytes were read", ex); }

Prevention

When it happens

Trigger: Calling ReadExactly for N bytes when the underlying stream contains fewer than N bytes remaining (truncated file, closed network stream, already fully consumed stream).

Common situations: Parsing binary formats whose length header disagrees with actual payload size; truncated/corrupt downloads; reading past the end when the stream position was left at EOF by earlier code; network streams closed mid-transfer.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/1137287d647cecf6. Report an issue: GitHub.

Appendix: source

Thrown at src/Microsoft.ML.Data/Utilities/StreamUtils.cs:225

            }
        }

        // No argument checking is done here. It is up to the caller.
        private static int ReadAtLeastCore(Stream stream, byte[] buffer, int offset, int minimumBytes, bool throwOnEndOfStream)
        {
            Debug.Assert(minimumBytes <= buffer.Length);
            int count = minimumBytes;
            int totalRead = 0;
            while (totalRead < minimumBytes)
            {
                int read = stream.Read(buffer, offset, count);
                offset += read;
                count -= read;
                if (read == 0)
                {
                    if (throwOnEndOfStream)
                    {
                        throw new EndOfStreamException("Unable to read beyond the end of the stream.");
                    }

                    return totalRead;
                }

                totalRead += read;
            }

            return totalRead;
        }

        /// <summary>
        /// Reads <paramref name="count"/> number of bytes from the current stream and advances the position within the stream.
        /// </summary>
        /// <param name="stream"></param>
        /// <param name="buffer">
        /// An array of bytes. When this method returns, the buffer contains the specified byte array with the values
        /// between <paramref name="offset"/> and (<paramref name="offset"/> + <paramref name="count"/> - 1) replaced

View on GitHub (pinned to 7b76e69cf9)