What causes "Failed to stream log for task %s" in apache/druid?

Check Overlord logs for the full exception stack trace

Failed to stream log for task %s

Error message

Failed to stream log for task %s

What it means

The Overlord's task log streaming endpoint (GET /druid/indexer/v1/task/{id}/log) hit an unexpected exception while locating or streaming the task log. It logs the exception and returns a bare HTTP 500 with no body.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/overlord/http/OverlordResource.java:800

      @PathParam("taskid") final String taskid,
      @QueryParam("offset") @DefaultValue("0") final long offset
  )
  {
    try {
      final Optional<InputStream> stream = taskLogStreamer.streamTaskLog(taskid, offset);
      if (stream.isPresent()) {
        return Response.ok(stream.get()).build();
      } else {
        return Response.status(Response.Status.NOT_FOUND)
                       .entity(
                           "No log was found for this task. "
                           + "No logs found for this task. Ensure that the task is running and logging is configured correctly."
                       )
                       .build();
      }
    }
    catch (Exception e) {
      log.warn(e, "Failed to stream log for task %s", taskid);
      return Response.status(Response.Status.INTERNAL_SERVER_ERROR).build();
    }
  }

  @GET
  @Path("/task/{taskid}/reports")
  @Produces(MediaType.APPLICATION_JSON)
  @ResourceFilters(TaskResourceFilter.class)
  public Response doGetReports(
      @PathParam("taskid") final String taskid
  )
  {
    try {
      final Optional<InputStream> stream = taskLogStreamer.streamTaskReports(taskid);
      if (stream.isPresent()) {
        return Response.ok(stream.get()).build();
      } else {
        return Response.status(Response.Status.NOT_FOUND)

View on GitHub (pinned to 9b90983fd2)

Solutions

Check Overlord logs for the full exception stack trace
Confirm the task exists and its log files exist on the worker (druid.indexer.logs.directory)
Retry the request; if logs were purged, re-run the task to regenerate them
If running on remote workers, ensure the worker is reachable and serving log endpoints

Defensive patterns

Strategy: try-catch

Validate before calling

// Confirm the task exists before streaming logs
// GET /druid/indexer/v1/task/{id}/status -> 200

Try / catch

try { streamLog(taskId); } catch (WebApplicationException e) { if (e.getResponse().getStatus() == 500) { /* fall back to reading worker log dir directly */ } }

Prevention

Fetch logs promptly before log retention purges them
Keep druid.indexer.logs.directory on a healthy, writable volume
Check task status before requesting logs

When it happens

Trigger: GET /druid/indexer/v1/task/{taskid}/log when the TaskRunner's streamTaskLog throws, e.g. the log file disappears mid-stream or the underlying worker lookup fails.

Common situations: Task logs already cleaned up by log retention; middle manager unreachable or task log file rotated while streaming; misconfigured log directory in the worker.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/4d0e8de68e6e30a9. Report an issue: GitHub.