microsoft/garnet · error · Exception

Config file size {numBytesToWrite} is larger than the maximu

Error message

Config file size {numBytesToWrite} is larger than the maximum allowed size {MaxConfigFileSizeAligned}

What it means

Generic Exception thrown by AzureStreamProvider.GetBytesToWrite when the serialized config file payload, after sector-size alignment rounding, exceeds MaxConfigFileSizeAligned (262144 bytes = 256 KB). This protects the Azure Storage device from writing a config blob larger than the fixed-capacity segment it was initialized with. The check is exclusive to the Azure storage path; local file providers have no such limit.

Source

Thrown at libs/common/StreamProvider.cs:182

        }

        protected override IDevice GetDevice(string path)
        {
            var fileInfo = new FileInfo(path);

            // Get the container info, if it does not exist it will be created
            var settingsDeviceFactory = azureStorageNamedDeviceFactoryCreator.Create($"{fileInfo.Directory?.Name}");
            var settingsDevice = settingsDeviceFactory.Get(new FileDescriptor("", fileInfo.Name));
            settingsDevice.Initialize(MaxConfigFileSizeAligned, epoch: null, omitSegmentIdFromFilename: false);
            return settingsDevice;
        }

        protected override long GetBytesToWrite(byte[] bytes, IDevice device)
        {
            long numBytesToWrite = bytes.Length;
            numBytesToWrite = ((numBytesToWrite + (device.SectorSize - 1)) & ~(device.SectorSize - 1));
            if (numBytesToWrite > MaxConfigFileSizeAligned)
                throw new Exception($"Config file size {numBytesToWrite} is larger than the maximum allowed size {MaxConfigFileSizeAligned}");
            return numBytesToWrite;
        }
    }

    /// <summary>
    /// StreamProvider for reading / writing files locally
    /// </summary>
    internal class LocalFileStreamProvider : StreamProviderBase
    {
        private readonly bool readOnly;
        private readonly LocalStorageNamedDeviceFactoryCreator localDeviceFactoryCreator;

        public LocalFileStreamProvider(bool readOnly = false)
        {
            this.readOnly = readOnly;
            this.localDeviceFactoryCreator = new LocalStorageNamedDeviceFactoryCreator(disableFileBuffering: false, deviceType: DeviceType.FileStream, readOnly: readOnly);
        }

View on GitHub (pinned to 951b0fc683)

Solutions

1. Reduce the size of the config file below 256 KB by removing unnecessary entries, externalizing large data, or splitting configs.
2. If the config legitimately contains large data (e.g., key material, seed data), move that data out of the config file into a separate storage path.
3. Check for a serialization bug or duplicate entries inflating the file.
4. If using Azure storage for config is optional, switch to FileLocationType.Local which has no 256 KB cap.

Example fix

// before: oversized config
var options = LoadVeryLargeOptions();
ConfigStoreFactory.SaveConfig(options, FileLocationType.AzureStorage, connectionString);

// after: trim config, store large data separately
var trimmedOptions = LoadCoreOptionsOnly();
ConfigStoreFactory.SaveConfig(trimmedOptions, FileLocationType.AzureStorage, connectionString);

const long MaxConfigBytes = 262144; // MaxConfigFileSizeAligned
void ValidateConfigSize(byte[] bytes, int sectorSize)
{
    long aligned = (bytes.LongLength + (sectorSize - 1)) & ~(sectorSize - 1);
    if (aligned > MaxConfigBytes)
        throw new InvalidOperationException($"Config payload ({aligned} bytes) exceeds the 256 KB Azure limit.");
}

try
{
    provider.Write(configBytes);
}
catch (Exception ex) when (ex.Message.Contains("larger than the maximum allowed size"))
{
    logger.LogError("Config file too large for Azure storage (max 256 KB). Trim or externalize data.");
    throw;
}

When it happens

Trigger: Writing a garnet.conf (or other config) via an AzureStreamProvider whose serialized byte count, rounded up to the Azure device sector size, surpasses 256 KB. This could happen with an extremely large Options object, huge embedded data inside the config, or a serialization bug producing runaway output.

Common situations: A config file that grew beyond 256 KB due to many entries (e.g., a large ACL list, many custom command definitions, or embedded binary data); migrating a config that was fine on local disk but breaches the Azure cap; a serialization regression that emits excessive whitespace or duplicate keys.

Related errors

• Azure Storage connection string is required to read/write to
• Assembly is required to read from embedded resource
• Cannot use AzureStorage device without supplying storage-str
• Cannot use AzureStorage device with both storage-string and
• Gossip sample fraction should be in range [0,100]

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/54251e55fa7c5c89. Report an issue: GitHub.