{"record":{"id":"81249d2978dd191d","repo":"LykosAI/StabilityMatrix","slug":"json-length-exceeds-the-maximum-allowed-size","errorCode":null,"errorMessage":"JSON length exceeds the maximum allowed size.","messagePattern":"JSON length exceeds the maximum allowed size\\.","errorType":"validation","errorClass":"InvalidDataException","httpStatus":null,"severity":"error","filePath":"StabilityMatrix.Core/Models/SafetensorMetadata.cs","lineNumber":44,"sourceCode":"\n    /// <summary>\n    /// Tries to parse the metadata from a SafeTensor file.\n    /// </summary>\n    /// <param name=\"safetensorStream\">Stream to the SafeTensor file.</param>\n    /// <returns>The parsed metadata. Can be <see langword=\"null\"/> if the file does not contain metadata.</returns>\n    public static async Task<SafetensorMetadata?> ParseAsync(Stream safetensorStream)\n    {\n        // 8 bytes unsigned little-endian 64-bit integer\n        // 1 byte start of JSON object '{'\n        Memory<byte> buffer = new byte[9];\n        await safetensorStream.ReadExactlyAsync(buffer).ConfigureAwait(false);\n        var span = buffer.Span;\n\n        const ulong MAX_ALLOWED_JSON_LENGTH = 100 * 1024 * 1024; // 100 MB\n        var jsonLength = BinaryPrimitives.ReadUInt64LittleEndian(span);\n        if (jsonLength > MAX_ALLOWED_JSON_LENGTH)\n        {\n            throw new InvalidDataException(\"JSON length exceeds the maximum allowed size.\");\n        }\n        if (span[8] != '{')\n        {\n            throw new InvalidDataException(\"JSON does not start with '{'.\");\n        }\n\n        // Unfornately Utf8JsonReader does not support reading from a stream directly.\n        // Usually the size of the entire JSON object is less than 500KB,\n        // using a pooled buffer should reduce the number of large allocations.\n        var jsonBytes = ArrayPool<byte>.Shared.Rent((int)jsonLength);\n        try\n        {\n            // Important: the length of the rented buffer can be larger than jsonLength\n            // and there can be additional junk data at the end.\n\n            // we already read {, so start from index 1\n            jsonBytes[0] = (byte)'{';\n            await safetensorStream","sourceCodeStart":26,"sourceCodeEnd":62,"githubUrl":"https://github.com/LykosAI/StabilityMatrix/blob/af93d6ef57c01cd890d7e0ad0a9ea8c9fcda3002/StabilityMatrix.Core/Models/SafetensorMetadata.cs#L26-L62","documentation":"SafetensorMetadata.ParseAsync reads an 8-byte little-endian header and throws InvalidDataException when the declared JSON length exceeds 100 MB. This guards against corrupt or malicious safetensors files causing huge allocations.","triggerScenarios":"Loading a .safetensors file whose first 8 bytes encode a JSON size greater than 100 MB — corrupt file, wrong offset, or a non-safetensors file renamed to .safetensors.","commonSituations":"Truncated or corrupted downloads, files with missing/shifted headers, malicious crafted files, reading the wrong region of a stream.","solutions":["Re-download or re-export the safetensors file; the header is corrupt.","Verify the file is actually safetensors format (starts with plausible 8-byte length followed by '{').","Check that the stream position is 0 before parsing.","Validate file size and header before calling ParseAsync."],"exampleFix":"// before\nawait SafetensorMetadata.ParseAsync(stream); // throws on corrupt header\n// after\nif (stream.Length < 8) throw new InvalidDataException(\"File too small for safetensors header.\");\nvar header = new byte[8];\nawait stream.ReadExactlyAsync(header);\nvar len = BinaryPrimitives.ReadUInt64LittleEndian(header);\nif (len > 100UL * 1024 * 1024) throw new InvalidDataException($\"Unsafe header length {len}.\");\nstream.Position = 0;\nawait SafetensorMetadata.ParseAsync(stream);","handlingStrategy":"validation","validationCode":"const ulong MaxJsonLength = 100UL * 1024 * 1024;\nusing var view = await MemoryMappedFile.CreateFromFile(path).CreateViewStreamAsync(0, 8);\nvar buf = new byte[8];\nawait view.ReadExactlyAsync(buf);\nvar len = System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian(buf);\nif (len > MaxJsonLength) throw new InvalidDataException($\"Header length {len} is unsafe.\");","typeGuard":"bool HasPlausibleSafetensorsHeader(ReadOnlySpan<byte> first9Bytes) =>\n    first9Bytes.Length >= 9 &&\n    System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian(first9Bytes) <= 100UL * 1024 * 1024 &&\n    first9Bytes[8] == (byte)'{';","tryCatchPattern":"try { var meta = await SafetensorMetadata.ParseAsync(stream); }\ncatch (InvalidDataException ex) { _logger.LogWarning(ex, \"Rejecting file: {Message}\", ex.Message); return null; }","preventionTips":["Validate safetensors files after download with a hash check","Never assume a .safetensors extension means valid format","Check file size is at least 10 bytes before parsing","Keep stream position at 0 before calling ParseAsync"],"tags":["safetensors","file-validation","limit"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"af93d6ef57c01cd890d7e0ad0a9ea8c9fcda3002","analyzedAt":"2026-09-12T19:02:43.389Z","contentChangedAt":"2026-09-12T19:02:43.389Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}