microsoft/garnet · error · TsavoriteException

Allocator with sector size {sectorSize} cannot flush to devi

Error message

Allocator with sector size {sectorSize} cannot flush to device with sector size {device.SectorSize}

What it means

Thrown by AllocatorBase.VerifyCompatibleSectorSize(IDevice) before flushing pages to a storage device. The allocator performs sector-aligned I/O using the sector size it captured from its primary LogDevice at construction; if it is asked to flush to a different device whose SectorSize does not evenly divide that captured value, alignment would be violated and on-disk data could be corrupted, so the write is rejected up front. The check is `sectorSize % device.SectorSize != 0`, where `sectorSize` is the allocator's own (inherited from the original LogDevice).

Source

Thrown at libs/storage/Tsavorite/cs/src/core/Allocator/AllocatorBase.cs:489

            if (isEpochOwned)
                epoch.Dispose();
            bufferPool.Free();

            flushEvent.Dispose();
            notifyFlushedUntilAddressTcs?.TrySetCanceled();
            notifyFlushedUntilAddressTcs = null;

            onReadOnlyObserver?.OnCompleted();
            onEvictionObserver?.OnCompleted();
        }

        #endregion abstract and virtual methods

        private protected void VerifyCompatibleSectorSize(IDevice device)
        {
            if (sectorSize % device.SectorSize != 0)
                throw new TsavoriteException($"Allocator with sector size {sectorSize} cannot flush to device with sector size {device.SectorSize}");
        }

        /// <summary>
        /// This writes data from a page (or pages) for allocators that support only inline data.
        /// </summary>
        /// <param name="alignedSourceAddress">The source address, aligned to start of allocator page</param>
        /// <param name="alignedDestinationAddress">The destination address, aligned to start of allocator page</param>
        /// <param name="numBytesToWrite">Number of bytes to be written, based on allocator page range</param>
        /// <param name="callback">The callback for the operation</param>
        /// <param name="asyncResult">The callback state information, including information for the flush operation</param>
        /// <param name="device">The device to write to</param>
        [MethodImpl(MethodImplOptions.NoInlining)]
        internal void WriteInlinePageAsync<TContext>(IntPtr alignedSourceAddress, ulong alignedDestinationAddress, uint numBytesToWrite,
                DeviceIOCompletionCallback callback, PageAsyncFlushResult<TContext> asyncResult, IDevice device)
        {
            if (asyncResult.partial)
            {
                // Write only required bytes within the page

View on GitHub (pinned to 951b0fc683)

Solutions

  1. Make the target device's SectorSize a divisor of the allocator's sectorSize. Easiest: ensure the primary LogDevice and any device the same allocator flushes to report the same SectorSize.
  2. If you control the custom IDevice, set its SectorSize to a value that divides the main device's (e.g. 512 when the main device is 512 or 4096), or match it exactly.
  3. Recreate the Tsavorite store so the allocator captures the desired sectorSize from a device consistent with every device it will flush to.
  4. For checkpoint-to-disk flows, point checkpoints at a device constructed with the same sector size as the log device (Devices.CreateLogDevice uses the same sector-size logic for both).

Example fix

// before: main log on 512-byte device, checkpoint to a 4096-byte device
var logDev = Devices.CreateLogDevice("log", sectorSize: 512);
var ckptDev = new MyCustomDevice(sectorSize: 4096);
// -> VerifyCompatibleSectorSize throws during flush

// after: align sector sizes
var logDev = Devices.CreateLogDevice("log", sectorSize: 512);
var ckptDev = new MyCustomDevice(sectorSize: 512); // divides/matches allocator sectorSize
Defensive patterns

Strategy: validation

Validate before calling

// Before flushing the allocator to a secondary device, confirm sector compatibility.
// 'allocatorSectorSize' is the sector size the allocator captured from its primary LogDevice
// (expose it via your wrapper, or read device.SectorSize of the original log device).
static bool CanFlushToDevice(int allocatorSectorSize, IDevice target)
    => target is not null && allocatorSectorSize % target.SectorSize == 0;

// usage
if (!CanFlushToDevice(logDevice.SectorSize, checkpointDevice))
    throw new InvalidOperationException(
        $"Cannot flush: allocator sector {logDevice.SectorSize} not divisible by device sector {checkpointDevice.SectorSize}");

Try / catch

try
{
    await store.FlushAsync();   // or checkpoint to the secondary device
}
catch (TsavoriteException ex) when (ex.Message.Contains("cannot flush to device with sector size"))
{
    // Sector-size mismatch: reconcile device SectorSize values, then retry on a rebuilt store.
    logger.Error("Sector-size mismatch on flush; rebuild store with matching device sector sizes: {Msg}", ex.Message);
    throw;
}

Prevention

When it happens

Trigger: A flush or checkpoint path routes data to an IDevice whose SectorSize is not a divisor of the allocator's sectorSize. For example the main hybrid log was opened on a device with sectorSize 512, but the operation flushes to a device (or a custom IDevice wrapper, or a copy-to device) reporting SectorSize 4096; 512 % 4096 != 0, so the guard fires.

Common situations: Mixing backing stores in one store (local SSD for the log, Azure/network device for checkpoints or a secondary copy); wrapping a real device in a custom IDevice whose SectorSize getter returns an unrelated power of two; upgrading a device layer that changed its reported SectorSize (e.g. a 512-native device now reporting 4096 logical sectors) without re-creating the allocator.

Related errors

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/f5bb881102c3ec79. Report an issue: GitHub.