{"record":{"id":"ab3cac4bc2ea70d1","repo":"apache/hadoop","slug":"the-useerrorstream-error-input-stream-i","errorCode":null,"errorMessage":"The ${useErrorStream ? \"error\" : \"input\"} stream is null.","messagePattern":"The (.+?) stream is null\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java","lineNumber":491,"sourceCode":"          result);\n    }\n    workingDir = absolutePath;\n  }\n\n  private Path makeAbsolute(Path f) {\n    return f.isAbsolute()? f: new Path(workingDir, f);\n  }\n\n  @VisibleForTesting\n  public static Map<?, ?> jsonParse(final HttpURLConnection c,\n      final boolean useErrorStream) throws IOException {\n    if (c.getContentLength() == 0) {\n      return null;\n    }\n    final InputStream in = useErrorStream ?\n        c.getErrorStream() : c.getInputStream();\n    if (in == null) {\n      throw new IOException(\"The \" + (useErrorStream? \"error\": \"input\") +\n          \" stream is null.\");\n    }\n    try {\n      final String contentType = c.getContentType();\n      if (contentType != null) {\n        final MediaType parsed = MediaType.valueOf(contentType);\n        if (!MediaType.APPLICATION_JSON_TYPE.isCompatible(parsed)) {\n          throw new IOException(\"Content-Type \\\"\" + contentType\n              + \"\\\" is incompatible with \\\"\" + MediaType.APPLICATION_JSON\n              + \"\\\" (parsed=\\\"\" + parsed + \"\\\")\");\n        }\n      }\n      return JsonSerialization.mapReader().readValue(in);\n    } finally {\n      in.close();\n    }\n  }\n","sourceCodeStart":473,"sourceCodeEnd":509,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java#L473-L509","documentation":"WebHdfsFileSystem.jsonParse chooses either the HTTP error stream or input stream and refuses to parse a null stream. A zero content length returns null earlier, so this IOException means the connection advertised a nonzero or unknown content length but HttpURLConnection could not provide the corresponding body. The WebHDFS response therefore cannot be decoded.","triggerScenarios":"A WebHDFS operation reaches jsonParse and c.getContentLength() is not 0, but getInputStream() or getErrorStream() returns null. This occurs when a server or proxy closes the connection without a body, sends malformed HTTP, or supplies headers inconsistent with the actual response.","commonSituations":"A reverse proxy, firewall, or API gateway drops a WebHDFS response body; a NameNode or HttpFS process fails while writing the response; keep-alive connection reuse hands the client a stale connection; an overloaded server resets the connection.","solutions":["Reproduce the request with curl -i against the exact WebHDFS URL and inspect status, Content-Length, and body.","Check NameNode, HttpFS, and reverse-proxy logs for the same request; this is a transport/response defect rather than bad application input.","Remove or fix proxies that strip or truncate response bodies and disable unsafe response buffering/rewriting for WebHDFS paths.","Retry once on a new connection to rule out a stale keep-alive socket, then report the endpoint failure if it repeats."],"exampleFix":"// before\nHttpURLConnection conn = (HttpURLConnection) url.openConnection();\nMap<?, ?> json = WebHdfsFileSystem.jsonParse(conn, false);\n\n// after: diagnose at the HTTP boundary before JSON parsing\nHttpURLConnection conn = (HttpURLConnection) url.openConnection();\nint code = conn.getResponseCode();\nInputStream body = code >= 400 ? conn.getErrorStream() : conn.getInputStream();\nif (body == null) {\n  throw new IOException(\"WebHDFS endpoint returned no body for HTTP \" + code);\n}\nMap<?, ?> json = WebHdfsFileSystem.jsonParse(conn, false);","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  return fs.getFileStatus(path);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().endsWith(\"stream is null.\")) {\n    LOG.error(\"WebHDFS endpoint returned headers without a body for {}\", path, e);\n    // One retry can recover a stale keep-alive connection; persistent failures are endpoint/proxy defects.\n    return retryOnceOnNewFileSystem(fs.getUri(), fs.getConf(), path);\n  }\n  throw e;\n}","preventionTips":["Use curl -i to verify that every WebHDFS request returns a body consistent with its Content-Length.","Do not deploy proxies that strip or buffer response bodies incorrectly.","Close and recreate FileSystem/connections after transport errors rather than reusing a suspect connection."],"tags":["java","hadoop","webhdfs","http","json","response-body"],"backgroundTag":"empty-response-body","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}