{"record":{"id":"7e0873ed8ba25e28","repo":"apache/hadoop","slug":"error-while-tracking-duration-of-an-abfsrestoperat","errorCode":null,"errorMessage":"Error while tracking Duration of an AbfsRestOperation call","messagePattern":"Error while tracking Duration of an AbfsRestOperation call","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java","lineNumber":308,"sourceCode":"      throws AzureBlobFileSystemException {\n    // Since this might be a sub-sequential or parallel rest operation\n    // triggered by a single file system call, using a new tracing context.\n    lastUsedTracingContext = createNewTracingContext(tracingContext);\n    try {\n      if (abfsCounters != null) {\n        abfsCounters.getLastExecutionTime().set(now());\n      }\n      if (client.getAbfsMetricsManager() != null) {\n        client.getAbfsMetricsManager()\n            .timerOrchestrator(TimerFunctionality.RESUME, null);\n      }\n      IOStatisticsBinding.trackDurationOfInvocation(abfsCounters,\n          AbfsStatistic.getStatNameFromHttpCall(method),\n          () -> completeExecute(lastUsedTracingContext));\n    } catch (AzureBlobFileSystemException aze) {\n      throw aze;\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Error while tracking Duration of an \"\n          + \"AbfsRestOperation call\", e);\n    }\n  }\n\n  /**\n   * Executes the REST operation with retry, by issuing one or more\n   * HTTP operations.\n   * @param tracingContext TracingContext instance to track correlation IDs\n   */\n  void completeExecute(TracingContext tracingContext)\n      throws AzureBlobFileSystemException {\n    // see if we have latency reports from the previous requests\n    String latencyHeader = getClientLatency();\n    if (latencyHeader != null && !latencyHeader.isEmpty()) {\n      AbfsHttpHeader httpHeader =\n              new AbfsHttpHeader(HttpHeaderConfigurations.X_MS_ABFS_CLIENT_LATENCY, latencyHeader);\n      requestHeaders.add(httpHeader);\n    }","sourceCodeStart":290,"sourceCodeEnd":326,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java#L290-L326","documentation":"AbfsRestOperation.execute runs the HTTP operation inside IOStatisticsBinding.trackDurationOfInvocation; completeExecute only declares AzureBlobFileSystemException, so any other IOException escaping that lambda is wrapped as UncheckedIOException(\"Error while tracking Duration of an AbfsRestOperation call\", e). AzureBlobFileSystemExceptions are rethrown untouched — this wrapper appears only for unexpected IOExceptions on the per-call path (request construction/signing/audit or interruption), and the original IOException is always attached as the cause.","triggerScenarios":"An IOException that is not translated by the ABFS retry/error handling escaping completeExecute: thread interrupted during the REST call, SSL/TLS handshake failures raised outside retry handling, malformed request-building failures, or bugs in the statistics/tracking path itself.","commonSituations":"Thread pools shutting down and interrupting in-flight ABFS calls; restrictive TLS configurations on the JVM; upgrading hadoop-azure and hitting a regression in the REST/statistics path; the underlying IO error being masked by the wrapper so the real cause goes unread.","solutions":["Unwrap the cause: catch UncheckedIOException and inspect getCause() — the real IOException names the actual problem","Fix the underlying I/O issue it names (TLS config, proxy, interruption, request size)","Avoid interrupting threads that are executing ABFS operations; shut pools down gracefully","If the cause is empty or the failure reproduces deterministically, collect debug logs and check for / file a hadoop-azure JIRA; upgrade to a patched release"],"exampleFix":"// before\ntry {\n  fs.getFileStatus(path);\n} catch (UncheckedIOException e) {\n  throw e;   // cause (the real IOException) never inspected\n}\n\n// after\ntry {\n  fs.getFileStatus(path);\n} catch (UncheckedIOException e) {\n  IOException cause = (IOException) e.getCause();\n  log.error(\"ABFS REST call failed: {}\", cause.getMessage(), cause);\n  // handle the underlying condition (retry on transient IO, fix TLS, etc.)\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  fs.getFileStatus(path);\n} catch (UncheckedIOException e) {\n  IOException cause = (IOException) e.getCause();   // the real failure\n  log.error(\"ABFS REST call failed: {}\", cause, cause);\n  // branch on the actual cause: TLS/proxy config, interruption, transient IO\n}","preventionTips":["Always unwrap UncheckedIOException.getCause() — the wrapper message is generic","Shut down thread pools gracefully; do not interrupt in-flight ABFS REST calls","Verify JVM TLS/proxy settings for storage endpoints; keep hadoop-azure patched"],"tags":["azure-blob","abfs","rest","unchecked-exception","io-statistics","wrapper","hadoop"],"backgroundTag":"unchecked-io-exception","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}