{"record":{"id":"ad5c9974b2877f2f","repo":"jhy/jsoup","slug":"unhandled-content-type-must-be-a-text-or-xml-medi","errorCode":null,"errorMessage":"Unhandled content type. Must be a text or XML media type","messagePattern":"Unhandled content type\\. Must be a text or XML media type","errorType":"http","errorClass":"UnsupportedMimeTypeException","httpStatus":null,"severity":"error","filePath":"src/main/java/org/jsoup/helper/HttpConnection.java","lineNumber":968,"sourceCode":"                    }\n                    req.url(redir);\n\n                    return execute(req, res);\n                }\n                if ((res.statusCode < 200 || res.statusCode >= 400) && !req.ignoreHttpErrors())\n                        throw new HttpStatusException(\"HTTP error fetching URL\", res.statusCode, req.url().toString());\n\n                // check that we can handle the returned content type; if not, abort before fetching it\n                String contentType = res.contentType();\n                boolean isText = contentType != null && contentType.regionMatches(true, 0, \"text/\", 0, 5);\n                boolean isXml = contentType != null && xmlContentTypeRxp.matcher(contentType).matches();\n\n                if (contentType != null\n                        && !req.ignoreContentType()\n                        && !isText\n                        && !isXml\n                        )\n                    throw new UnsupportedMimeTypeException(\"Unhandled content type. Must be a text or XML media type\",\n                            contentType, req.url().toString());\n\n                // switch to the XML parser if content type is xml and not parser not explicitly set\n                if (isXml) {\n                    if (!req.parserDefined) req.parser(Parser.xmlParser());\n                }\n\n                res.charset = DataUtil.getCharsetFromContentType(res.contentType); // may be null, readInputStream deals with it\n                if (res.contentLength != 0 && req.method() != HEAD) { // -1 means unknown, chunked. sun throws an IO exception on 500 response with no content when trying to read body\n                    InputStream stream = executor.responseBody();\n                    if (res.hasHeaderWithValue(CONTENT_ENCODING, \"gzip\"))\n                        stream = new GZIPInputStream(stream);\n                    else if (res.hasHeaderWithValue(CONTENT_ENCODING, \"deflate\"))\n                        stream = new InflaterInputStream(stream, new Inflater(true));\n                    \n                    res.bodyStream = ControllableInputStream.wrap(\n                        stream, DefaultBufferSize, req.maxBodySize())\n                        .timeout(startTime, req.timeout());","sourceCodeStart":950,"sourceCodeEnd":986,"githubUrl":"https://github.com/jhy/jsoup/blob/9851ac5d9c576c6888910b5a51a2362bbc978959/src/main/java/org/jsoup/helper/HttpConnection.java#L950-L986","documentation":"Jsoup only parses text/* and XML content types by default. If the server returns another MIME type (e.g. application/octet-stream, images, zip) and ignoreContentType is false, jsoup throws UnsupportedMimeTypeException rather than attempting to parse binary data. This prevents garbage parses and accidental large binary downloads.","triggerScenarios":"Jsoup.connect(url).get() against a URL serving a PDF, image, JSON, or binary file without setting ignoreContentType(true) or a matching parser.","commonSituations":"Crawlers following links into attachments (PDFs, XLSX); endpoints returning application/json when scraping (modern servers label JSON content types that older jsoup versions may not treat as xml); CDNs serving generic octet-stream.","solutions":["Parse only text/XML URLs; skip or download-with-other-tools binary content","Call ignoreContentType(true) if you want the body regardless of MIME type","Catch UnsupportedMimeTypeException and filter those URLs in your crawler","For JSON endpoints use a JSON client, not jsoup parsing"],"exampleFix":"// before\nDocument doc = Jsoup.connect(pdfUrl).get(); // throws\n// after\nConnection.Response res = Jsoup.connect(pdfUrl)\n    .ignoreContentType(true)\n    .execute();\n// or catch it\ntry { Jsoup.connect(url).get(); }\ncatch (UnsupportedMimeTypeException e) { /* skip binary URL */ }","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { doc = Jsoup.connect(url).get(); } catch (UnsupportedMimeTypeException e) { /* skip binary resource or download with other tool */ }","preventionTips":["HEAD-request or inspect Content-Type before GET when crawling","Use ignoreContentType(true) only for endpoints you actually handle","Route JSON/binary resources to appropriate clients instead of jsoup parsing"],"tags":["java","jsoup","http","content-type"],"backgroundTag":"unexpected-http-status","analyzedSha":"9851ac5d9c576c6888910b5a51a2362bbc978959","analyzedAt":"2026-09-08T15:22:04.931Z","contentChangedAt":"2026-09-08T15:22:04.931Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}