{"record":{"id":"c906e4c8ed6faae7","repo":"apache/druid","slug":"failed-to-stream-logs-from-s-c906e4","errorCode":null,"errorMessage":"Failed to stream logs from: %s","messagePattern":"Failed to stream logs from: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":500,"severity":"error","filePath":"extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleTaskLogs.java","lineNumber":166,"sourceCode":"      try {\n        final long start;\n\n        if (offset > 0 && offset < length) {\n          start = offset;\n        } else if (offset < 0 && (-1 * offset) < length) {\n          start = length + offset;\n        } else {\n          start = 0;\n        }\n\n        return Optional.of(new GoogleByteSource(storage, config.getBucket(), taskKey).openStream(start));\n      }\n      catch (Exception e) {\n        throw new IOException(e);\n      }\n    }\n    catch (IOException e) {\n      throw new IOE(e, \"Failed to stream logs from: %s\", taskKey);\n    }\n  }\n\n  private String getTaskLogKey(String taskid)\n  {\n    return config.getPrefix() + \"/\" + taskid.replace(':', '_');\n  }\n\n  private String getTaskReportKey(String taskid)\n  {\n    return config.getPrefix() + \"/\" + taskid.replace(':', '_') + \".report.json\";\n  }\n\n  private String getTaskStatusKey(String taskid)\n  {\n    return config.getPrefix() + \"/\" + taskid.replace(':', '_') + \".status.json\";\n  }\n","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleTaskLogs.java#L148-L184","documentation":"GoogleTaskLogs.streamTaskFile opens a GCS object for streaming task logs/reports/status. Any failure to open or read the object is wrapped into an IOException with this message and the GCS taskKey. It indicates the requested log could not be streamed, most often because it does not exist.","triggerScenarios":"Calling streamTaskLog/streamTaskReports/streamTaskStatus for a task whose file is not in GCS (task still running, log never pushed, wrong prefix), or when opening the GCS read channel fails (permissions, transient client errors).","commonSituations":"Overlord console streaming logs of a still-running task before push; wrong prefix configured in GoogleTaskLogsConfig; deleted logs past retention; service account lost read access.","solutions":["Confirm the object exists at <prefix>/<taskId with ':' replaced by '_'> in the configured bucket.","If the task is still running, expect this failure and serve the local file instead (callers normally fall back).","Check GoogleTaskLogsConfig bucket/prefix against where the task actually pushed.","Grant the reading node's credentials storage.objects.get on the bucket.","Inspect the wrapped cause for StorageException codes (404 vs 403 vs 5xx) to pick retry vs fix."],"exampleFix":"// before\nOptional<InputStream> in = taskLogs.streamTaskLog(taskId, 0);\n// after\nOptional<InputStream> in = Optional.empty();\ntry {\n  in = taskLogs.streamTaskLog(taskId, 0);\n} catch (IOException e) {\n  log.warn(\"No task log in GCS for %s, falling back to local file\", taskId);\n}","handlingStrategy":"fallback","validationCode":"String key = config.getPrefix() + \"/\" + taskid.replace(':', '_');\nboolean exists = storage.get(config.getBucket(), key) != null;","typeGuard":null,"tryCatchPattern":"try { return Optional.of(stream.open()); }\ncatch (IOException e) {\n  log.warn(\"Task log not streamable from GCS: %s\", taskKey);\n  return Optional.empty(); // serve local file instead\n}","preventionTips":["Expect missing logs for running tasks; use local-file fallback first.","Replace ':' with '_' exactly as getTaskLogKey does when checking keys.","Align GoogleTaskLogsConfig prefix/bucket with the task runner's push config.","Inspect cause codes: 404 -> not yet pushed; 403 -> permissions."],"tags":["gcs","streaming","task-logs","io"],"backgroundTag":"file-read-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}