weaviate/weaviate · warning

merge keys: %w

Error message

merge keys: %w

What it means

While merging keys of two inverted-index segments, writeKeys periodically checks ctx.Err() (every AbortCheckEveryN iterations) so a cancelled or timed-out context stops the merge promptly. On cancellation it returns "merge keys: %w" wrapping context.Canceled or context.DeadlineExceeded. This is a cooperative-abort mechanism, not data corruption.

Source

Thrown at adapters/repos/db/lsmkv/compactor_inverted.go:328

		return 0, err
	}
	c.offset += len(encoded) + 8 + 8 + 8
	return len(encoded) + 8 + 8 + 8, nil
}

func (c *compactorInverted) writeKeys(ctx context.Context) ([]segmentindex.KeyRedux, error) {
	key1, value1, _ := c.c1.first()
	key2, value2, _ := c.c2.first()

	// the (dummy) header was already written, this is our initial offset

	kis := make([]segmentindex.KeyRedux, 0, c.c1.segment.index.KeyCount()+c.c2.segment.index.KeyCount())
	sim := newSortedMapMerger()

	for i := 0; ; i++ {
		if i%compactor.AbortCheckEveryN == 0 {
			if err := ctx.Err(); err != nil {
				return nil, fmt.Errorf("merge keys: %w", err)
			}
		}

		if key1 == nil && key2 == nil {
			break
		}
		if bytes.Equal(key1, key2) {

			value1Clean, _ := c.cleanupValues(value1)

			sim.reset([][]MapPair{value1Clean, value2})
			mergedPairs, err := sim.
				doKeepTombstonesReusable()
			if err != nil {
				return nil, err
			}

			if len(mergedPairs) == 0 {

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Expected on graceful shutdown — no action needed; compaction resumes later
2. If unintended, increase the timeout governing the compaction context
3. Reduce merge sizes / tune compaction schedule to avoid long-running merges
4. Ensure the context isn't cancelled prematurely by the caller (e.g. request-scoped ctx used for background work)

Example fix

// before: request-scoped context used for background compaction
compactor.Do(ctx) // ctx dies with the HTTP request
// after
cctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
defer cancel()
compactor.Do(cctx)

// verify the context outlives the merge before starting
if ctx.Err() != nil {
    return fmt.Errorf("context already done before merge: %w", ctx.Err())
}
deadline, ok := ctx.Deadline()
if ok && time.Until(deadline) < estimatedMergeTime {
    // extend deadline or use a background context
}

if err := compactor.Do(ctx); err != nil {
    if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
        // benign on shutdown; otherwise retry with a longer-lived context
        return retryWithBackgroundContext(job)
    }
    return err
}

When it happens

Trigger: The context passed into the compaction/merge is cancelled or its deadline expires while writeKeys is iterating merged keys — e.g. shutdown, shard drop, or an upper-layer timeout on a long merge of large segments.

Common situations: Server shutdown or shard release during a large compaction; overly tight timeouts on background operations; very large inverted segments making merges exceed deadlines.

Related errors

• flush buffered: %w
• write compactorMap segment checksum: %w
• close new segment file %q: %w
• replace compacted segments on disk: %w
• replace compacted segments (blocking): %w

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/66b7f024465dbf8c. Report an issue: GitHub.