{"record":{"id":"bb138bdc9b58ddc9","repo":"apache/druid","slug":"recoverable-exception-bb138b","errorCode":null,"errorMessage":"Recoverable exception","messagePattern":"Recoverable exception","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"warning","filePath":"extensions-core/azure-extensions/src/main/java/org/apache/druid/storage/azure/AzureByteSource.java","lineNumber":66,"sourceCode":"    this.azureStorage = azureStorage;\n    this.containerName = containerName;\n    this.blobPath = blobPath;\n  }\n\n  @Override\n  public InputStream openStream() throws IOException\n  {\n    return openStream(0L);\n  }\n\n  public InputStream openStream(long offset) throws IOException\n  {\n    try {\n      return azureStorage.getBlockBlobInputStream(offset, containerName, blobPath);\n    }\n    catch (BlobStorageException e) {\n      if (AzureUtils.AZURE_RETRY.apply(e)) {\n        throw new IOException(\"Recoverable exception\", e);\n      }\n      log.error(\"Exception when opening stream to azure resource, containerName: %s, blobPath: %s, Error: %s\",\n               containerName, blobPath, e.getMessage()\n      );\n      throw new RuntimeException(e);\n    }\n  }\n}\n","sourceCodeStart":48,"sourceCodeEnd":75,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/azure-extensions/src/main/java/org/apache/druid/storage/azure/AzureByteSource.java#L48-L75","documentation":"AzureByteSource.openStream wraps BlobStorageException from Azure storage: if the error is classified as retryable by AZURE_RETRY, it is rethrown as an IOException('Recoverable exception') so callers can retry; non-retryable errors are logged and rethrown as RuntimeException.","triggerScenarios":"openStream() calls azureStorage.getBlockBlobInputStream(offset, containerName, blobPath) and Azure returns a transient BlobStorageException — HTTP 500/503 from storage, throttling (429), server busy, or intermittent connection resets.","commonSituations":"Azure storage account under throttling during heavy ingest; transient storage-service incidents; network blips between Druid and Azure blob endpoint during deep-storage reads (e.g., segment loading).","solutions":["Retry the operation — the error is explicitly marked recoverable; Druid task retry machinery usually handles it","Enable/configure retry policies with backoff on the Azure storage client","Check Azure Storage metrics/alerts for throttling (429) or availability incidents","Scale down concurrent readers or increase storage account capacity if consistently throttled"],"exampleFix":"// caller pattern\ntry {\n  InputStream in = byteSource.openStream();\n} catch (IOException e) {\n  if (e.getMessage().contains(\"Recoverable exception\")) {\n    // retry with backoff\n    in = retryer.call(() -> byteSource.openStream());\n  } else {\n    throw e;\n  }\n}","handlingStrategy":"retry","validationCode":"// preflight blob existence before openStream\nCloudBlockBlob blob = container.getBlockBlobReference(blobPath);\nif (!blob.exists()) throw new FileNotFoundException(blobPath);","typeGuard":null,"tryCatchPattern":"Retryer<InputStream> retryer = RetryerBuilder.<InputStream>newBuilder()\n    .retryIfException(e -> e instanceof IOException\n        && \"Recoverable exception\".equals(e.getMessage()))\n    .withWaitStrategy(WaitStrategies.exponentialWait(1, TimeUnit.SECONDS))\n    .withStopStrategy(StopStrategies.stopAfterAttempt(5))\n    .build();\nInputStream in = retryer.call(() -> byteSource.openStream());","preventionTips":["Configure Azure SDK retry with exponential backoff","Monitor storage account throttling (429) and ServerBusy metrics","Spread deep-storage reads to avoid thundering-herd on startup","Check Azure Service Health during ingestion failures"],"tags":["azure","blob-storage","transient-failure","retry"],"backgroundTag":"request-timeout","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}