pinpoint-apm/pinpoint · warning

reconnectScheduledExecutor shutdown failed

Error message

reconnectScheduledExecutor shutdown failed

What it means

In GrpcModuleLifeCycle.shutdown(), the reconnect scheduler (reconnectScheduledExecutorService, which retries gRPC channel connections to the collector) is stopped with a 3-second shutdownAndAwaitTermination. If scheduled reconnect tasks don't finish/terminate in time, "reconnectScheduledExecutor shutdown failed" is logged. Non-fatal: shutdown proceeds, but reconnect worker threads may linger briefly.

Source

Thrown at agent-module/profiler/src/main/java/com/navercorp/pinpoint/profiler/context/module/GrpcModuleLifeCycle.java:121

    }

    @Override
    public void shutdown() {
        logger.info("shutdown()");
        IOUtils.closeQuietly(spanDataSender, (ex) -> logger.warn("spanDataSender close fail", ex));
        IOUtils.closeQuietly(statDataSender, (ex) -> logger.warn("statDataSender close fail", ex));
        IOUtils.closeQuietly(agentDataSender, (ex) -> logger.warn("agentDataSender close fail", ex));
        IOUtils.closeQuietly(metadataDataSender, (ex) -> logger.warn("metadataDataSender close fail", ex));

        if (dnsExecutorService != null) {
            if (!MoreExecutors.shutdownAndAwaitTermination(dnsExecutorService, Duration.ofSeconds(3))) {
                logger.warn("dnsExecutor shutdown failed");
            }
        }
        if (reconnectScheduledExecutorService != null) {
            if (!MoreExecutors.shutdownAndAwaitTermination(reconnectScheduledExecutorService, Duration.ofSeconds(3))) {
                logger.warn("reconnectScheduledExecutor shutdown failed");
            }
        }
        IOUtils.closeQuietly(reporter, (ex) -> logger.warn("reporter close fail", ex));
    }

    @Override
    public String toString() {
        return "GrpcModuleLifeCycle{" +
                ", agentDataSender=" + agentDataSender +
                ", metadataDataSender=" + metadataDataSender +
                ", spanDataSender=" + spanDataSender +
                ", statDataSender=" + statDataSender +
                ", dnsExecutorService=" + dnsExecutorService +
                ", reconnectScheduledExecutorService=" + reconnectScheduledExecutorService +
                '}';
    }
}

View on GitHub (pinned to 744c3d3075)

Solutions

  1. Restore connectivity to the Pinpoint collector so reconnect tasks idle down, then shut down.
  2. Ignore the warning at JVM exit if it appears sporadically — threads are daemon-bound and die with the process.
  3. Check collector address/port config to prevent perpetual reconnect loops.
  4. Upgrade Pinpoint for improved executor shutdown handling in GrpcModuleLifeCycle.

Example fix

// before: perpetual reconnect loop blocks clean shutdown
profiler.collector.span.ip=unreachable-host
// after
profiler.collector.span.ip=collector.pinpoint.local
Defensive patterns

Strategy: fallback

Validate before calling

// Verify collector reachability so reconnect tasks are idle before shutdown
boolean reachable = new Socket(host, port).connect(new InetSocketAddress(host, port), 2000);

Try / catch

try {
    lifeCycle.shutdown();
} catch (Exception e) {
    logger.warn("reconnect scheduler did not stop in time", e);
}

Prevention

When it happens

Trigger: shutdown() called while reconnect tasks are queued/executing — typically when the collector is unreachable and the retry scheduler is actively re-attempting connections every few seconds.

Common situations: Stopping the agent during a collector outage; flaky network causing aggressive reconnect scheduling; abrupt container termination racing the scheduler; tests that shut down the module without ever establishing a connection.

Related errors

AI-assisted analysis of pinpoint-apm/pinpoint@744c3d3075 (2026-09-07). Data as JSON: /api/errors/5e6f9348ec53b223. Report an issue: GitHub.