apache/skywalking · warning · StorageException

interrupted while waiting for no-init backend resources for

Error message

interrupted while waiting for no-init backend resources for model {modelName}

What it means

In the legacy non-init poll path, ModelInstaller loops every 3 seconds waiting for the init OAP to create the model's backend resources (deferDDLToInitNode is set, so this node must not create them itself). If the thread is interrupted during Thread.sleep(3000L), it restores the interrupt flag and throws StorageException wrapping the InterruptedException — meaning OAP shutdown (or thread cancellation) raced the wait loop.

Source

Thrown at oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelInstaller.java:141

                        throw e;
                    }
                    // A transient backend error during the probe (e.g. a BanyanDB cluster data node
                    // still Init-ing, "client connection is closing") is NOT a reason to abort boot:
                    // the init OAP will create the resource and the next probe succeeds. Treat it like
                    // "not present yet" and retry in-loop, rather than letting it escape and crash-loop
                    // the pod — which would only re-enter this same loop after a full restart.
                    allExist = false;
                    log.warn("install info: existence probe for model: [{}] threw a transient backend "
                        + "error. OAP is running in 'no-init' mode, retry 3s later.", model.getName(), e);
                }
                if (allExist) {
                    break;
                }
                try {
                    Thread.sleep(3000L);
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                    throw new StorageException(
                        "interrupted while waiting for no-init backend resources for model " + model.getName(), e);
                }
            }
            return;
        }

        InstallInfo info = isExists(model, opt);
        if (info.isAllExist()) {
            return;
        }
        if (!flags.isCreateMissing()) {
            // Inspect-but-don't-create: caller wants existence reported as outcome but
            // explicitly forbids DDL. Today no canonical mode hits this branch, but the
            // flag combination is valid (e.g. dry-run reporting) and falling through to
            // createTable would silently violate the contract.
            opt.recordOutcome("table", model.getName(),
                StorageManipulationOpt.Outcome.MISSING,
                "missing on backend; createMissing flag is off — skipping DDL");

View on GitHub (pinned to 102af09b4a)

Solutions

1. Treat this as a symptom, not the bug: find why the loop was still waiting — usually the init OAP has not created the resource (check its logs) or storage is unreachable.
2. If it was an intentional shutdown, no action is needed; the node exits cleanly with the interrupt preserved.
3. Increase init OAP capacity or pre-create schema to shorten the wait window; ensure liveness/Readiness probe timeouts tolerate the install phase.
4. Verify the resource names in the preceding 'waiting create or update...' log lines exist in the backend once init completes, then restart the node.

// Server-side pattern (already implemented): restore interrupt and rethrow
} catch (InterruptedException ie) { Thread.currentThread().interrupt(); throw new StorageException("interrupted while waiting...", ie); }
// Operator: treat this exception during shutdown as benign; during run as 'init OAP never delivered schema'.

When it happens

Trigger: OAP shutdown or worker-thread interruption while a non-init node is still in the 3s-poll loop waiting for resources that the init OAP has not yet created.

Common situations: Killing/restarting a pod that is stuck waiting because the init OAP is down or slow; long waits during large first-time schema installs where k8s liveness probes or OOM kills interrupt the installer thread; CI test harnesses tearing down the OAP mid-boot.

Related errors

• local-cache-verify boot: backend resources for model '{model
• Metric TTL should be at least 2 days, current value is {}
• Record TTL should be at least 2 days, current value is {}
• Create {stream.builder().getSimpleName()} none stream record
• Create {stream.getBuilder().getSimpleName()} metrics DAO fai

AI-assisted analysis of apache/skywalking@102af09b4a (2026-08-14). Data as JSON: /api/errors/f307303b9c4c8bcf. Report an issue: GitHub.