apache/pulsar · error · IllegalArgumentException

Segment not found

Error message

Segment not found

What it means

SegmentLayout.mergeSegments(long,long,long,int) looks up both segment IDs in the immutable layout snapshot (allSegments map) before merging. If either ID is absent from this snapshot it throws IllegalArgumentException("Segment not found"). It is a programmatic precondition check: the layout is an immutable in-memory view of a scalable topic's segment DAG, so a missing ID means the caller is operating on stale metadata or a fabricated ID.

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/service/scalable/SegmentLayout.java:220

     * @param nowMs      wall-clock millis used as the parents' seal time and the
     *                   merged child's create time
     * @return a new SegmentLayout with the merge applied
     */
    public SegmentLayout mergeSegments(long segmentId1, long segmentId2, long nowMs) {
        return mergeSegments(segmentId1, segmentId2, nowMs, EntryBucketSplits.MAX_BUCKETS);
    }

    /**
     * As {@link #mergeSegments(long, long, long)}, clamping the merged segment's entry-bucket
     * count to {@code maxBucketsPerSegment} (the configured per-segment ceiling): the merged
     * segment recovers the parents' buckets, but never past the hard ceiling.
     */
    public SegmentLayout mergeSegments(long segmentId1, long segmentId2, long nowMs,
                                       int maxBucketsPerSegment) {
        SegmentInfo seg1 = allSegments.get(segmentId1);
        SegmentInfo seg2 = allSegments.get(segmentId2);
        if (seg1 == null || seg2 == null) {
            throw new IllegalArgumentException("Segment not found");
        }
        if (!seg1.isActive() || !seg2.isActive()) {
            throw new IllegalArgumentException("Both segments must be active");
        }
        if (!seg1.hashRange().isAdjacentTo(seg2.hashRange())) {
            throw new IllegalArgumentException("Segments are not adjacent: "
                    + seg1.hashRange() + " and " + seg2.hashRange());
        }

        long newEpoch = epoch + 1;
        long mergedId = nextSegmentId;
        HashRange mergedRange = seg1.hashRange().merge(seg2.hashRange());

        // PIP-486: a merge is the inverse of a split — the merged segment recovers both parents' buckets
        // (N1 + N2), so the topic's total entry-bucket count stays ≈ the budget as segments coalesce.
        List<Integer> mergedEntryBucketSplits = EntryBucketSplits.equalWidth(
                Math.min(seg1.bucketCount() + seg2.bucketCount(), maxBucketsPerSegment));
        SegmentInfo sealed1 = seg1.sealed(newEpoch, nowMs, List.of(mergedId));

View on GitHub (pinned to 820761864e)

Solutions

1. Reload the current layout (SegmentLayout.fromMetadata(metadata)) immediately before merging so IDs are current
2. Verify both IDs with layout.getAllSegments().containsKey(id) before calling mergeSegments
3. Use getActiveSegments() / activeSegments.values() to pick merge candidates instead of stale cached IDs
4. After a CAS retry on the metadata, rebuild the layout and re-validate the segment pair before retrying the merge

Example fix

// before
layout.mergeSegments(cachedSegIdA, cachedSegIdB, nowMs);
// after
SegmentLayout fresh = SegmentLayout.fromMetadata(latestMetadata);
if (fresh.getAllSegments().containsKey(cachedSegIdA) && fresh.getAllSegments().containsKey(cachedSegIdB)) {
    fresh.mergeSegments(cachedSegIdA, cachedSegIdB, nowMs);
}

boolean canMerge(SegmentLayout layout, long id1, long id2) {
    var segs = layout.getAllSegments();
    return segs.containsKey(id1) && segs.containsKey(id2)
        && segs.get(id1).isActive() && segs.get(id2).isActive()
        && segs.get(id1).hashRange().isAdjacentTo(segs.get(id2).hashRange());
}

boolean segmentExists(SegmentLayout layout, long id) {
    return id > 0 && layout.getAllSegments().containsKey(id);
}

try {
    newLayout = layout.mergeSegments(id1, id2, nowMs);
} catch (IllegalArgumentException e) {
    if (e.getMessage().startsWith("Segment not found")) {
        layout = SegmentLayout.fromMetadata(refreshMetadata()); // stale snapshot; retry on fresh layout
    } else {
        throw e;
    }
}

When it happens

Trigger: Calling mergeSegments(segmentId1, segmentId2, nowMs[, maxBucketsPerSegment]) with an ID that is not in the layout — typically an ID that was already pruned, an ID from a previous (older-epoch) SegmentLayout snapshot, or a hardcoded/guessed ID.

Common situations: Reusing segment IDs captured from an earlier layout after another rebalance/merge/split changed the DAG; applying an auto-scale policy against a refreshed layout from ScalableTopicMetadata where the segments were concurrently modified; unit tests constructing partial SegmentLayout maps and merging IDs not present.

Related errors

• Both segments must be active
• Segments are not adjacent: ${hashRange1} and ${hashRange2}
• configuredService should not be an instance of SystemTopicBa
• Segment not found: ${segmentId}
• Cannot rebucket non-active segment: ${segmentId}

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/b17e6cb85bd98fd4. Report an issue: GitHub.