{"record":{"id":"94928184a27f3397","repo":"dotnet/machinelearning","slug":"file-path-too-big-to-open","errorCode":null,"errorMessage":"File {path} too big to open.","messagePattern":"File (.+?) too big to open\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.ImageAnalytics/ImageLoader.cs","lineNumber":306,"sourceCode":"                            var editor = VBufferEditor.Create(ref dst, 0);\n                            dst = editor.Commit();\n                        }\n\n                    };\n\n                return del;\n            }\n\n            private static bool TryLoadDataIntoBuffer(string path, ref VBuffer<byte> imgData)\n            {\n                int count = -1;\n                int bytesread = -1;\n                // bufferSize == 1 used to avoid unnecessary buffer in FileStream\n                using (FileStream fs = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 1))\n                {\n                    long fileLength = fs.Length;\n                    if (fileLength > int.MaxValue)\n                        throw new IOException($\"File {path} too big to open.\");\n                    else if (fileLength == 0)\n                    {\n                        byte[] imageBuffer;\n                        // Some file systems (e.g. procfs on Linux) return 0 for length even when there's content.\n                        // Thus we need to assume 0 doesn't mean empty.\n                        imageBuffer = File.ReadAllBytes(path);\n                        count = imageBuffer.Length;\n                        imgData = new VBuffer<byte>(count, imageBuffer);\n                        return (count > 0);\n                    }\n\n                    count = (int)fileLength;\n                    var editor = VBufferEditor.Create(ref imgData, count);\n                    bytesread = ReadToEnd(fs, editor.Values);\n                    imgData = editor.Commit();\n                    return (count > 0);\n                }\n","sourceCodeStart":288,"sourceCodeEnd":324,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.ImageAnalytics/ImageLoader.cs#L288-L324","documentation":"When loading an image file into the buffer, ImageLoader opens the file with a FileStream and checks its length. Files larger than int.MaxValue bytes (~2 GB) cannot be represented in the int-sized byte buffer, so an IOException is thrown. Note a zero-length reported size is tolerated by re-reading with File.ReadAllBytes because some filesystems (e.g. Linux procfs) report 0 for non-empty files.","triggerScenarios":"Calling an ImageLoader/ImageLoadingTransformer load path (via TryLoadDataIntoBuffer, reached when the loader processes a row) on an image file whose size exceeds 2,147,483,647 bytes.","commonSituations":"Pointing an image loader at raw video frames, medical/scientific imagery, TIFF stacks, or accidentally at non-image large binaries (database dumps, archives) sitting in the watched folder.","solutions":["Remove or downsample the oversized file so each image is well under 2 GB","Pre-filter input rows/files by new FileInfo(path).Length <= int.MaxValue before feeding the loader","If huge images are genuinely required, read them via a library/streaming path supporting long-length data instead of ML.NET ImageLoader"],"exampleFix":"// before\nloader.LoadImage(row.Path); // throws if > 2GB\n// after\nvar fi = new FileInfo(row.Path);\nif (fi.Length > int.MaxValue)\n    throw new InvalidOperationException($\"Image {row.Path} exceeds 2GB limit; resize or split it.\");\nloader.LoadImage(row.Path);","handlingStrategy":"validation","validationCode":"var fi = new FileInfo(path);\nif (fi.Length > int.MaxValue)\n    throw new InvalidOperationException($\"{path} exceeds 2GB image limit.\");","typeGuard":"bool IsLoadableImageFile(string path) => File.Exists(path) && new FileInfo(path).Length <= int.MaxValue;","tryCatchPattern":"try\n{\n    LoadImage(path);\n}\ncatch (IOException ex) when (ex.Message.Contains(\"too big to open\"))\n{\n    logger.LogWarning(\"Skipping oversized image {Path}\", path);\n}","preventionTips":["Pre-screen input directories for file sizes before pipeline runs","Downsample large scientific/medical imagery upstream","Exclude non-image large files from input folders"],"tags":["csharp","images","filesystem","file-size"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}