apache/hadoop · error · IOException

Content-Type "{0}" is incompatible with "application/json" (

Error message

Content-Type "{0}" is incompatible with "application/json" (parsed="{1}")

What it means

HttpFSUtils.jsonParse() refuses to parse a response unless its Content-Type is compatible with application/json (checked via JAX-RS MediaType.isCompatible). A non-JSON content type (text/html, text/plain, application/octet-stream...) triggers this IOException listing the actual type. It fires before any parsing, so it is a reliable signal that something other than the HttpFS JSON API answered the request.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSUtils.java:138

  /**
   * Convenience method that JSON Parses the <code>InputStream</code> of a
   * <code>HttpURLConnection</code>.
   *
   * @param conn the <code>HttpURLConnection</code>.
   *
   * @return the parsed JSON object.
   *
   * @throws IOException thrown if the <code>InputStream</code> could not be
   * JSON parsed.
   */
  public static Object jsonParse(HttpURLConnection conn) throws IOException {
    try {
      String contentType = conn.getContentType();
      if (contentType != null) {
        final MediaType parsed = MediaType.valueOf(contentType);
        if (!MediaType.APPLICATION_JSON_TYPE.isCompatible(parsed)) {
          throw new IOException("Content-Type \"" + contentType
              + "\" is incompatible with \"" + MediaType.APPLICATION_JSON
              + "\" (parsed=\"" + parsed + "\")");
        }
      }
      JSONParser parser = new JSONParser();
      return parser.parse(
          new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8));
    } catch (ParseException ex) {
      throw new IOException("JSON parser error, " + ex.getMessage(), ex);
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

curl -i the failing WebHDFS URL and read the actual Content-Type and body to identify who answered
Fix or bypass the intermediary: exempt /webhdfs/v1 from the HTML-producing auth/error pages, or authenticate properly so the JSON API responds
Point the webhdfs:// URI at the real HttpFS/WebHDFS host:port
Confirm httpfs-site.xml authentication type (simple vs kerberos) matches what the client presents, so no interstitial page is returned

Defensive patterns

Strategy: try-catch

Try / catch

try {
  FileStatus st = fs.getFileStatus(path);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("is incompatible with")) {
    // a non-JSON page (auth form, error page) answered instead of the HttpFS API
    LOG.error("Non-JSON response from {} - check auth/proxy in front of HttpFS", fs.getUri());
  }
  throw e;
}

Prevention

curl -i the WebHDFS endpoint after any proxy/auth-layer change and assert Content-Type: application/json
Keep authentication mode (simple/kerberos) consistent between client config and httpfs-site.xml
Alert on text/html responses from /webhdfs/v1 URLs as a proxy-misconfiguration signal

When it happens

Trigger: Any metadata operation (GETFILESTATUS, LISTSTATUS, GETXATTRS...) where a gateway returns an HTML login/error page, an auth challenge body, or where the request lands on a non-HttpFS servlet that answers with text/plain or a binary stream.

Common situations: Kerberos/SSO proxy in front of HttpFS returning an HTML form with status 200; nginx serving a static maintenance page; wrong port hitting a different webapp; MIME-mangling middleware between client and server.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/47e7cfb13bfc96f2. Report an issue: GitHub.