What causes "Invalid number of chunks: " in microsoft/FASTER?

Use an index size where size / kSizeofChunk is a power of 2 (typically sizes of kSizeofChunk * 2^n); log the computed numChunks to verify.

microsoft/FASTER · error · FasterException

Invalid number of chunks:

Error message

Invalid number of chunks:

What it means

During a checkpoint-driven resize, SplitIndex.SplitBuckets divides the new table size into chunks (Constants.kSizeofChunk each); the resulting chunk count must be a power of two so buckets map to chunks via masking. If numChunks is not a power of two, FASTER throws FasterException because the split would be unbalanced/incorrect. This is an invariant over the configured table size and chunk constant.

Solutions

Use an index size where size / kSizeofChunk is a power of 2 (typically sizes of kSizeofChunk * 2^n); log the computed numChunks to verify.
Round the configured index size up to the nearest valid chunked size before constructing the store.
If using a modified Constants.kSizeofChunk, ensure it keeps size/numChunks a power of two, or file a FASTER bug with the size.

Example fix

// before
var store = new FasterKV<long, long>(indexSize: 1 << 20 + 1); // arbitrary size
await store.TakeFullCheckpointAsync(token);
// after
long indexSize = Utility.NextPowerOf2(1 << 21); // keep size/kSizeofChunk a power of 2
var store = new FasterKV<long, long>(indexSize);
await store.TakeFullCheckpointAsync(token);

Defensive patterns

Strategy: validation

Validate before calling

long indexSize = /* configured */;
long numChunks = Math.Max(1, indexSize / Constants.kSizeofChunk);
if (!Utility.IsPowerOfTwo(numChunks))
    indexSize = (long)Constants.kSizeofChunk * Utility.NextPowerOf2(numChunks);
var store = new FasterKV<long, long>(indexSize);

Type guard

bool SupportsCheckpointSplit(long indexSize) => Utility.IsPowerOfTwo(Math.Max(1, indexSize / Constants.kSizeofChunk));

Try / catch

try { await store.TakeFullCheckpointAsync(token); }
catch (FasterException ex) when (ex.Message.StartsWith("Invalid number of chunks"))
{ logger.LogCritical(ex, "Index size yields non-power-of-2 chunk count"); throw; }

Prevention

Choose index sizes that are kSizeofChunk * 2^n in configuration templates.
Add a startup validation that checks numChunks is a power of two before the first checkpoint.
Avoid modifying Constants.kSizeofChunk in custom builds without re-validating sizes.

When it happens

Trigger: Calling Checkpoint/FullCheckpoint/StartCheckpoint (which triggers SplitBuckets) with an index size such that (newSize / kSizeofChunk) is not a power of two — e.g. odd or non-power-of-2 chunk counts produced by unusual index sizes interacting with the chunk constant.

Common situations: Configuring FasterKV with an index size that passes constructor checks but yields a non-power-of-2 chunk count at resize/checkpoint time; mismatched Constants.kSizeofChunk in custom builds.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/438059155e2e5ca4. Report an issue: GitHub.

Appendix: source

Thrown at cs/src/core/Index/FASTER/Implementation/SplitIndex.cs:24

namespace FASTER.core
{
    public unsafe partial class FasterKV<Key, Value> : FasterBase, IFasterKV<Key, Value>
    {
        private void SplitBuckets(long hash)
        {
            long masked_bucket_index = hash & state[1 - resizeInfo.version].size_mask;
            int offset = (int)(masked_bucket_index >> Constants.kSizeofChunkBits);
            SplitBuckets(offset);
        }

        private void SplitBuckets(int offset)
        {
            int numChunks = (int)(state[1 - resizeInfo.version].size / Constants.kSizeofChunk);
            if (numChunks == 0) numChunks = 1; // at least one chunk

            if (!Utility.IsPowerOfTwo(numChunks))
            {
                throw new FasterException("Invalid number of chunks: " + numChunks);
            }
            for (int i = offset; i < offset + numChunks; i++)
            {
                if (0 == Interlocked.CompareExchange(ref splitStatus[i & (numChunks - 1)], 1, 0))
                {
                    long chunkSize = state[1 - resizeInfo.version].size / numChunks;
                    long ptr = chunkSize * (i & (numChunks - 1));

                    HashBucket* src_start = state[1 - resizeInfo.version].tableAligned + ptr;
                    HashBucket* dest_start0 = state[resizeInfo.version].tableAligned + ptr;
                    HashBucket* dest_start1 = state[resizeInfo.version].tableAligned + state[1 - resizeInfo.version].size + ptr;

                    SplitChunk(src_start, dest_start0, dest_start1, chunkSize);

                    // split for chunk is done
                    splitStatus[i & (numChunks - 1)] = 2;

                    if (Interlocked.Decrement(ref numPendingChunksToBeSplit) == 0)

View on GitHub (pinned to 321d872eab)