{"record":{"id":"c4f20b50d9709456","repo":"microsoft/FASTER","slug":"unable-to-read-object-page-total-size-greater-than-2gb","errorCode":null,"errorMessage":"Unable to read object page, total size greater than 2GB: ","messagePattern":"Unable to read object page, total size greater than 2GB: ","errorType":"exception","errorClass":"FasterException","httpStatus":null,"severity":"error","filePath":"cs/src/core/Allocator/GenericAllocator.cs","lineNumber":702,"sourceCode":"            // If we have processed entire page, return\n            if (result.untilPtr >= result.maxPtr)\n            {\n                result.Free();\n\n                // Call the \"real\" page read callback\n                result.callback(errorCode, numBytes, context);\n                return;\n            }\n\n            // We will now be able to process all records until (but not including) untilPtr\n            GetObjectInfo(result.freeBuffer1.GetValidPointer(), ref result.untilPtr, result.maxPtr, ObjectBlockSize, out long startptr, out long alignedLength);\n\n            // Object log fragment should be aligned by construction\n            Debug.Assert(startptr % sectorSize == 0);\n            Debug.Assert(alignedLength % sectorSize == 0);\n\n            if (alignedLength > int.MaxValue)\n                throw new FasterException(\"Unable to read object page, total size greater than 2GB: \" + alignedLength);\n\n            var objBuffer = bufferPool.Get((int)alignedLength);\n            result.freeBuffer2 = objBuffer;\n\n            // Request objects from objlog\n            result.objlogDevice.ReadAsync(\n                (int)((result.page - result.offset) >> (LogSegmentSizeBits - LogPageSizeBits)),\n                (ulong)startptr,\n                (IntPtr)objBuffer.aligned_pointer, (uint)alignedLength, AsyncReadPageWithObjectsCallback<TContext>, result);\n        }\n\n        /// <summary>\n        /// Invoked by users to obtain a record from disk. It uses sector aligned memory to read \n        /// the record efficiently into memory.\n        /// </summary>\n        /// <param name=\"fromLogical\"></param>\n        /// <param name=\"numBytes\"></param>\n        /// <param name=\"callback\"></param>","sourceCodeStart":684,"sourceCodeEnd":720,"githubUrl":"https://github.com/microsoft/FASTER/blob/321d872eabda6a0345c8bd76419f89723ed864ae/cs/src/core/Allocator/GenericAllocator.cs#L684-L720","documentation":"Thrown during object-page reads in GenericAllocator when the aligned length of the object-log page/fragment to read exceeds int.MaxValue (~2GB). The in-memory object read buffer and the device ReadAsync API are sized with 32-bit ints, so a larger single read cannot be represented. FASTER throws instead of truncating the read.","triggerScenarios":"Reading back an object log whose page/segment size (LogPageSizeBits-derived alignedLength) exceeds 2GB — typically an oversized object log page size or extremely large accumulated object fragments in one page on a restart/recovery scan.","commonSituations":"Configuring very large page sizes for the object log; upgrading page size bits after logs were written with objects spanning huge regions; recovery of an object log written by a differently tuned instance.","solutions":["Reduce the object log page size (lower LogSettings.PageSizeBits / LogPageSizeBits) so each read stays under 2GB.","Keep individual serialized objects small and ensure objects are distributed across pages rather than concentrated.","If possible, re-create the store with default page sizes and re-populate data from the source of truth.","Verify sector alignment assumptions — the alignedLength check runs only after alignment; oversized raw pages are the root cause."],"exampleFix":"// before: 31-bit page size -> >2GB reads\nvar settings = new LogSettings { PageSizeBits = 31 };\n\n// after: keep pages well under 2GB\nvar settings = new LogSettings { PageSizeBits = 22 }; // 4MB pages","handlingStrategy":"validation","validationCode":"// Ensure configured page sizes cannot yield >2GB object reads\nlong alignedMax = 1L << settings.PageSizeBits;\nif (alignedMax > int.MaxValue)\n    throw new InvalidOperationException($\"PageSizeBits too large: 2^{settings.PageSizeBits} exceeds int.MaxValue\");","typeGuard":"static bool PageSizeWithinIntLimit(int pageSizeBits) => (1L << pageSizeBits) <= int.MaxValue;","tryCatchPattern":"try\n{\n    var page = ReadObjectPage(address, alignedLength);\n}\ncatch (FasterException ex) when (ex.Message.Contains(\"greater than 2GB\"))\n{\n    // Rebuild the store with smaller page sizes or migrate data to a new instance\n    log.LogError(\"Object log page exceeds 2GB; re-create store with smaller PageSizeBits\");\n    throw;\n}","preventionTips":["Keep LogSettings.PageSizeBits (especially for the object log) well below 31 (defaults around 22-24 are safe).","Avoid unbounded single objects that inflate a page's serialized footprint.","Do not change page size bits between runs against the same on-disk logs; migrating data instead.","Add a startup check that all configured device/page sizes stay within int.MaxValue."],"tags":["faster","object-log","recovery","size-limit"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"321d872eabda6a0345c8bd76419f89723ed864ae","analyzedAt":"2026-09-15T22:18:00.693Z","contentChangedAt":"2026-09-15T22:18:00.693Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}