{"record":{"id":"54251e55fa7c5c89","repo":"microsoft/garnet","slug":"config-file-size-numbytestowrite-is-larger-than","errorCode":null,"errorMessage":"Config file size {numBytesToWrite} is larger than the maximum allowed size {MaxConfigFileSizeAligned}","messagePattern":"Config file size (.+?) is larger than the maximum allowed size (.+?)","errorType":"validation","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"libs/common/StreamProvider.cs","lineNumber":182,"sourceCode":"        }\n\n        protected override IDevice GetDevice(string path)\n        {\n            var fileInfo = new FileInfo(path);\n\n            // Get the container info, if it does not exist it will be created\n            var settingsDeviceFactory = azureStorageNamedDeviceFactoryCreator.Create($\"{fileInfo.Directory?.Name}\");\n            var settingsDevice = settingsDeviceFactory.Get(new FileDescriptor(\"\", fileInfo.Name));\n            settingsDevice.Initialize(MaxConfigFileSizeAligned, epoch: null, omitSegmentIdFromFilename: false);\n            return settingsDevice;\n        }\n\n        protected override long GetBytesToWrite(byte[] bytes, IDevice device)\n        {\n            long numBytesToWrite = bytes.Length;\n            numBytesToWrite = ((numBytesToWrite + (device.SectorSize - 1)) & ~(device.SectorSize - 1));\n            if (numBytesToWrite > MaxConfigFileSizeAligned)\n                throw new Exception($\"Config file size {numBytesToWrite} is larger than the maximum allowed size {MaxConfigFileSizeAligned}\");\n            return numBytesToWrite;\n        }\n    }\n\n    /// <summary>\n    /// StreamProvider for reading / writing files locally\n    /// </summary>\n    internal class LocalFileStreamProvider : StreamProviderBase\n    {\n        private readonly bool readOnly;\n        private readonly LocalStorageNamedDeviceFactoryCreator localDeviceFactoryCreator;\n\n        public LocalFileStreamProvider(bool readOnly = false)\n        {\n            this.readOnly = readOnly;\n            this.localDeviceFactoryCreator = new LocalStorageNamedDeviceFactoryCreator(disableFileBuffering: false, deviceType: DeviceType.FileStream, readOnly: readOnly);\n        }\n","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/microsoft/garnet/blob/951b0fc6838721f89d102c2bbe1b914e8d39d700/libs/common/StreamProvider.cs#L164-L200","documentation":"Generic Exception thrown by AzureStreamProvider.GetBytesToWrite when the serialized config file payload, after sector-size alignment rounding, exceeds MaxConfigFileSizeAligned (262144 bytes = 256 KB). This protects the Azure Storage device from writing a config blob larger than the fixed-capacity segment it was initialized with. The check is exclusive to the Azure storage path; local file providers have no such limit.","triggerScenarios":"Writing a garnet.conf (or other config) via an AzureStreamProvider whose serialized byte count, rounded up to the Azure device sector size, surpasses 256 KB. This could happen with an extremely large Options object, huge embedded data inside the config, or a serialization bug producing runaway output.","commonSituations":"A config file that grew beyond 256 KB due to many entries (e.g., a large ACL list, many custom command definitions, or embedded binary data); migrating a config that was fine on local disk but breaches the Azure cap; a serialization regression that emits excessive whitespace or duplicate keys.","solutions":["Reduce the size of the config file below 256 KB by removing unnecessary entries, externalizing large data, or splitting configs.","If the config legitimately contains large data (e.g., key material, seed data), move that data out of the config file into a separate storage path.","Check for a serialization bug or duplicate entries inflating the file.","If using Azure storage for config is optional, switch to FileLocationType.Local which has no 256 KB cap."],"exampleFix":"// before: oversized config\nvar options = LoadVeryLargeOptions();\nConfigStoreFactory.SaveConfig(options, FileLocationType.AzureStorage, connectionString);\n\n// after: trim config, store large data separately\nvar trimmedOptions = LoadCoreOptionsOnly();\nConfigStoreFactory.SaveConfig(trimmedOptions, FileLocationType.AzureStorage, connectionString);","handlingStrategy":"validation","validationCode":"const long MaxConfigBytes = 262144; // MaxConfigFileSizeAligned\nvoid ValidateConfigSize(byte[] bytes, int sectorSize)\n{\n    long aligned = (bytes.LongLength + (sectorSize - 1)) & ~(sectorSize - 1);\n    if (aligned > MaxConfigBytes)\n        throw new InvalidOperationException($\"Config payload ({aligned} bytes) exceeds the 256 KB Azure limit.\");\n}","typeGuard":null,"tryCatchPattern":"try\n{\n    provider.Write(configBytes);\n}\ncatch (Exception ex) when (ex.Message.Contains(\"larger than the maximum allowed size\"))\n{\n    logger.LogError(\"Config file too large for Azure storage (max 256 KB). Trim or externalize data.\");\n    throw;\n}","preventionTips":["Keep config files well under 256 KB; externalize large data (seeds, ACLs) to separate storage.","When targeting Azure storage, validate the serialized config size before writing.","Prefer local file storage for large configs where the 256 KB cap does not apply."],"tags":["config","azure-storage","size-limit","streamprovider","sector-alignment"],"backgroundTag":null,"analyzedSha":"951b0fc6838721f89d102c2bbe1b914e8d39d700","analyzedAt":"2026-08-13T19:01:32.939Z","schemaVersion":2},"datasetVersion":"2026-08-13T19:17:28.613Z"}