apache/skywalking · error · EngineCompileException

{causeMessage} — {rootCauseMessage}

Error message

{causeMessage} — {rootCauseMessage}

What it means

On the FILTER_ONLY fast path of MalRuleEngine.compile, an ApplyException from applier.apply is handled by first removing exactly the partially-registered metrics (applier.remove(ae.getPartiallyRegistered(), ...)), then rethrowing as EngineCompileException whose message joins the apply-failure message with its root cause ('{cause} — {rootCause}'). FILTER_ONLY edits swap only filters, so this error means a filter-only hot update of a MAL file failed partway through registration; the old appliedMal entry is untouched and keeps serving.

Source

Thrown at oap-server/server-admin/runtime-rule/src/main/java/org/apache/skywalking/oap/server/receiver/runtimerule/engine/mal/MalRuleEngine.java:407

            throw new IllegalStateException("MeterSystem unavailable for MAL compile of "
                + sourceName);
        }
        final MalFileApplier.Applied oldApplied = appliedFor(ctx.getRules(), key);

        // FILTER_ONLY fast path: no shape-break drop, no DDL move, no alarm reset, no
        // classloader retire. Just produce the freshly-compiled Applied and let commit do
        // the in-memory swap. The classifier already ran — engines don't re-classify here.
        if (classification == Classification.FILTER_ONLY) {
            final MalFileApplier.Applied fresh;
            try {
                fresh = applier.apply(
                    file.getContent(), sourceName, newHash, ctx.getStorageOpt(), kind);
            } catch (final MalFileApplier.ApplyException ae) {
                // Engine-internal partial rollback: undo whatever this attempt managed to
                // register before the throw. Old appliedMal[key] is untouched — it's still
                // serving — so removing the partial set is the only mutation needed.
                applier.remove(ae.getPartiallyRegistered(), ctx.getStorageOpt());
                throw new EngineCompileException(ae);
            }
            return new CompiledMalDSL(file.getCatalog(), file.getName(), newHash, classification,
                file.getContent(), oldApplied, fresh, /* delta */ null, Collections.emptySet());
        }

        // STRUCTURAL / NEW: re-derive the precise delta (added / removed / shape-break) from
        // the prior content. The scheduler's classify() call handed us the verdict but not
        // the delta sets — recomputing here keeps the SPI lean and ensures the delta the
        // engine acts on is internally consistent with the content it's compiling.
        final AppliedRuleScript priorScript = ctx.getRules().get(key);
        final String priorContent = priorScript == null ? null : priorScript.getContent();
        final DSLDelta delta = DeltaClassifier.classifyMal(priorContent, file.getContent());

        // Shape-break metrics MUST be dropped before applier.apply re-registers them at the
        // new shape — MeterSystem.create rejects re-register at a different (function, scope)
        // with an IllegalArgumentException. This IS the destructive shape-break contract:
        // the REST handler's allowStorageChange guardrail has already gated it, and the
        // design accepts that a verify-failure after this point loses shape-break data.

View on GitHub (pinned to 102af09b4a)

Solutions

1. Read the two-part message: the tail after '—' is the root cause (often the same partial-registration failure as error 128) — address that first
2. Retry the FILTER_ONLY update once the cluster and MeterSystem are quiescent; the engine already cleaned the partial set and the old rule still serves
3. Check for concurrent applies of the same file across OAP nodes and serialize them (apply via the elected main/REST orchestrator)
4. If a name collision is the root cause, resolve ownership of the colliding metric before re-applying

// Keep FILTER_ONLY edits truly filter-only: confirm no metric-name/shape changes before choosing the fast path
final DSLDelta d = DeltaClassifier.classifyMal(priorContent, newContent);
if (!d.getAdded().isEmpty() || !d.getRemoved().isEmpty() || !d.getShapeBreak().isEmpty())
    reject("not a FILTER_ONLY edit — submit as STRUCTURAL update");

catch (EngineCompileException e) on the FILTER_ONLY path: the engine already removed exactly the partially-registered set and the old rule keeps serving — parse the '— '-joined root cause, fix it (usually a concurrent registration/collision), then re-submit the update. Safe to retry after cluster quiescence.

When it happens

Trigger: Submitting a runtime-rule update classified as Classification.FILTER_ONLY (only filter config changed) where applier.apply throws MalFileApplier.ApplyException partway — e.g. a concurrent peer re-registered the same metric names, or MeterSystem rejected a registration mid-file. The engine unwinds the landed subset, then rethrows this exception.

Common situations: Filter-only edit submitted while a peer node concurrently re-registers the same metrics; MeterSystem transiently rejecting a re-register during high load; Prior bundle partially uninstalled by an earlier failed operation, leaving collision residue

Related errors

• MAL register failed for {sourceName} (partial)
• MAL compile failed for {sourceName}
• teardown_deferred
• YAML parse failure for {sourceName}
• Load meter analyzer configs failed

AI-assisted analysis of apache/skywalking@102af09b4a (2026-08-14). Data as JSON: /api/errors/4dcdf105630582e7. Report an issue: GitHub.