jhy/jsoup · error · UnsupportedMimeTypeException

Unhandled content type. Must be a text or XML media type

Error message

Unhandled content type. Must be a text or XML media type

What it means

Jsoup only parses text/* and XML content types by default. If the server returns another MIME type (e.g. application/octet-stream, images, zip) and ignoreContentType is false, jsoup throws UnsupportedMimeTypeException rather than attempting to parse binary data. This prevents garbage parses and accidental large binary downloads.

Solutions

1. Parse only text/XML URLs; skip or download-with-other-tools binary content
2. Call ignoreContentType(true) if you want the body regardless of MIME type
3. Catch UnsupportedMimeTypeException and filter those URLs in your crawler
4. For JSON endpoints use a JSON client, not jsoup parsing

Example fix

// before
Document doc = Jsoup.connect(pdfUrl).get(); // throws
// after
Connection.Response res = Jsoup.connect(pdfUrl)
    .ignoreContentType(true)
    .execute();
// or catch it
try { Jsoup.connect(url).get(); }
catch (UnsupportedMimeTypeException e) { /* skip binary URL */ }

try { doc = Jsoup.connect(url).get(); } catch (UnsupportedMimeTypeException e) { /* skip binary resource or download with other tool */ }

When it happens

Trigger: Jsoup.connect(url).get() against a URL serving a PDF, image, JSON, or binary file without setting ignoreContentType(true) or a matching parser.

Common situations: Crawlers following links into attachments (PDFs, XLSX); endpoints returning application/json when scraping (modern servers label JSON content types that older jsoup versions may not treat as xml); CDNs serving generic octet-stream.

Related errors

• You must execute the request before getting a response.
• URL not set. Make sure to call #url(...) before executing…
• Cannot follow redirect with a streamed request body…
• HTTP error fetching URL
• Too many redirects occurred trying to load URL

AI-assisted analysis of jhy/jsoup@9851ac5d9c (2026-09-08). Data as JSON: /api/errors/ad5c9974b2877f2f. Report an issue: GitHub.

Appendix: source

Thrown at src/main/java/org/jsoup/helper/HttpConnection.java:968

                    }
                    req.url(redir);

                    return execute(req, res);
                }
                if ((res.statusCode < 200 || res.statusCode >= 400) && !req.ignoreHttpErrors())
                        throw new HttpStatusException("HTTP error fetching URL", res.statusCode, req.url().toString());

                // check that we can handle the returned content type; if not, abort before fetching it
                String contentType = res.contentType();
                boolean isText = contentType != null && contentType.regionMatches(true, 0, "text/", 0, 5);
                boolean isXml = contentType != null && xmlContentTypeRxp.matcher(contentType).matches();

                if (contentType != null
                        && !req.ignoreContentType()
                        && !isText
                        && !isXml
                        )
                    throw new UnsupportedMimeTypeException("Unhandled content type. Must be a text or XML media type",
                            contentType, req.url().toString());

                // switch to the XML parser if content type is xml and not parser not explicitly set
                if (isXml) {
                    if (!req.parserDefined) req.parser(Parser.xmlParser());
                }

                res.charset = DataUtil.getCharsetFromContentType(res.contentType); // may be null, readInputStream deals with it
                if (res.contentLength != 0 && req.method() != HEAD) { // -1 means unknown, chunked. sun throws an IO exception on 500 response with no content when trying to read body
                    InputStream stream = executor.responseBody();
                    if (res.hasHeaderWithValue(CONTENT_ENCODING, "gzip"))
                        stream = new GZIPInputStream(stream);
                    else if (res.hasHeaderWithValue(CONTENT_ENCODING, "deflate"))
                        stream = new InflaterInputStream(stream, new Inflater(true));
                    
                    res.bodyStream = ControllableInputStream.wrap(
                        stream, DefaultBufferSize, req.maxBodySize())
                        .timeout(startTime, req.timeout());

View on GitHub (pinned to 9851ac5d9c)