{"record":{"id":"3b83da703709a176","repo":"apache/beam","slug":"error-starting-background-log-thread","errorCode":null,"errorMessage":"Error starting background log thread ","messagePattern":"Error starting background log thread ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/harness/src/main/java/org/apache/beam/fn/harness/logging/BeamFnLoggingClient.java","lineNumber":175,"sourceCode":"                // Logging which occurs in this thread will attempt to publish log entries into the\n                // above handler which should never block if the queue is full otherwise\n                // this thread will get stuck.\n                streamWriter.drainQueueToStream(bufferedLogEntries);\n              } finally {\n                restoreLoggers();\n                // Now that loggers are restored, do a final flush of any buffered logs\n                // in case they help with understanding above failures.\n                flushFinalLogs();\n              }\n              return COMPLETED;\n            },\n            executorService);\n    try {\n      // Wait for the thread to be running and log handlers installed or an error with the thread\n      // that is supposed to be consuming logs.\n      CompletableFuture.anyOf(this.bufferedLogConsumer, started).get();\n    } catch (ExecutionException e) {\n      throw new RuntimeException(\"Error starting background log thread \" + e.getCause());\n    } catch (InterruptedException e) {\n      Thread.currentThread().interrupt();\n      throw new RuntimeException(e);\n    }\n  }\n\n  @RequiresNonNull(\"logRecordHandler\")\n  @RequiresNonNull(\"configuredLoggers\")\n  private void installLogging(\n      @UnderInitialization BeamFnLoggingClient this, SdkHarnessOptions options) {\n    // Reset the global log manager, get the root logger and remove the default log handlers.\n    LogManager logManager = LogManager.getLogManager();\n    logManager.reset();\n    Logger rootLogger = logManager.getLogger(ROOT_LOGGER_NAME);\n    for (Handler handler : rootLogger.getHandlers()) {\n      rootLogger.removeHandler(handler);\n    }\n    // configure loggers from default sdk harness log level and log level overrides","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/logging/BeamFnLoggingClient.java#L157-L193","documentation":"BeamFnLoggingClient starts a background thread that consumes buffered log messages and forwards them to the logging service over gRPC. If that thread fails to start or dies during startup, the CompletableFuture completes exceptionally and the constructor rethrows it as RuntimeException('Error starting background log thread ...').","triggerScenarios":"The background log-consuming thread throws before/while installing log handlers (e.g. gRPC channel setup failure, uncaught exception in the consumer runnable), causing ExecutionException on the anyOf(...).get() future.","commonSituations":"Logging service endpoint unreachable or misconfigured (bad env var like worker logging service address); thread factory/executor unable to create threads (resource exhaustion, thread limits); serialization failures of log records at startup.","solutions":["Read the wrapped cause (e.getCause()) in the message to find the root failure (gRPC channel, thread start, etc.).","Verify the logging ApiServiceDescriptor endpoint is correct and reachable from the SDK worker.","Check executor/thread availability (ulimit -u, container thread limits) if the cause is a thread-start failure.","Upgrade/match runner and harness versions to rule out protocol incompatibilities, then retry the worker."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// pre-check endpoint reachability: assert !loggingEndpoint.isEmpty();","typeGuard":null,"tryCatchPattern":"try { new BeamFnLoggingClient(...); } catch (RuntimeException e) { LOG.error(\"Log thread startup failed\", e.getCause()); /* reconfigure endpoint or fail worker */ throw e; }","preventionTips":["Ensure the logging endpoint env var is set and reachable","Monitor thread/resource limits in worker containers","Log and inspect e.getCause() for the root failure","Keep runner/harness versions aligned"],"tags":["apache-beam","java-harness","logging","grpc"],"backgroundTag":"thread-start-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}