apache/skywalking · error · IllegalArgumentException

Rank {rank} doesn't exist in the previous ranks {ranks}

Error message

Rank {rank} doesn't exist in the previous ranks {ranks}

What it means

AvgHistogramPercentileFunction is the storage-backend-side aggregator for MAL `avg_histogram_percentile` metrics. On the first accept() in a time window it locks in the rank set (e.g. 50, 75, 90, 99); every later accept() for the same metric must pass the exact same ranks. This exception fires when a subsequent input carries a rank that was not in that first-locked set, i.e. the metric definition changed mid-window.

Source

Thrown at oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/meter/function/avg/AvgHistogramPercentileFunction.java:140

                    "Incompatible BucketedValues [" + value + "] for current PercentileFunction[" + dataset + "]");
            }
        }

        for (final int rank : value.getRanks()) {
            if (rank <= 0) {
                throw new IllegalArgumentException("Illegal rank value " + rank + ", must be positive");
            }
        }

        if (ranks.size() > 0) {
            if (ranks.size() != value.getRanks().length) {
                throw new IllegalArgumentException(
                    "Incompatible ranks size = [" + value.getRanks().length + "] for current PercentileFunction[" + ranks
                        .size() + "]");
            } else {
                for (final int rank : value.getRanks()) {
                    if (!ranks.include(rank)) {
                        throw new IllegalArgumentException(
                            "Rank " + rank + " doesn't exist in the previous ranks " + ranks);
                    }
                }
            }
        } else {
            for (final int rank : value.getRanks()) {
                ranks.add(rank);
            }
        }

        this.entityId = entity.id();

        String template = "%s";
        if (CollectionUtils.isNotEmpty(value.getBucketedValues().getLabels())) {
            template  = value.getBucketedValues().getLabels() + ":%s";
        }
        final long[] values = value.getBucketedValues().getValues();
        for (int i = 0; i < values.length; i++) {

View on GitHub (pinned to 102af09b4a)

Solutions

1. Make every producer of this metric (all MAL rules, all OAP nodes in the cluster) use the identical rank list, e.g. percentile(50, 75, 90, 99) -> avg_histogram_percentile
2. If you intentionally changed ranks, wait for the current minute window to roll over (or restart all OAP nodes together) so the next window locks the new set
3. Grep all MAL/LAL-derived meter rules for the metric name to find the rule with divergent ranks

Example fix

# meter-analyzer-config/service.yaml — before (node A)
percentile(50, 75, 90) -> avg_histogram_percentile
# after (aligned everywhere)
percentile(50, 75, 90, 99) -> avg_histogram_percentile

// before feeding a percentile metric, ensure the rank set matches the locked one
IntList locked = function.getRanks(); // ranks locked this window
int[] incoming = argument.getRanks();
if (locked.size() > 0 && (locked.size() != incoming.length || Arrays.stream(incoming).anyMatch(r -> !locked.include(r)))) {
    log.warn("Skipping incompatible rank set {} for metric {}; expected {}", incoming, metricName, locked);
    return; // do not call accept()
}
function.accept(entity, argument);

catch (IllegalArgumentException e) only around the accept() call, log the metric name and both rank sets, drop the sample, and alert — never let one bad sample kill the meter pipeline thread.

When it happens

Trigger: Two MAL rules (or two OAP nodes, or a changed agent/meter config) emit the same avg_histogram_percentile metric name within the same minute window with different percentile rank arguments; the second input whose rank is absent from the locked IntList triggers the throw.

Common situations: Editing percentile ranks in meter-analyzer-config/*.yaml or otel-rules and rolling OAP instances one by one (old and new configs coexist); duplicating a metric name across MAL files with different ranks; a sending agent updated its rank config while the window is still open.

Related errors

• Incompatible ranks size = [{value.getRanks().length}] for cu
• Rank {rank} doesn't exist in the previous ranks {ranks}
• Incompatible BucketedValues [{value}] for current Percentile
• Illegal rank value {rank}, must be positive
• Illegal rank value {rank}, must be positive

AI-assisted analysis of apache/skywalking@102af09b4a (2026-08-14). Data as JSON: /api/errors/fe25da4c3e576ee5. Report an issue: GitHub.