code4craft/webmagic · error · IllegalArgumentException

Illegal HTTP Method

Error message

Illegal HTTP Method 

What it means

HttpUriRequestConverter.selectRequestMethod() maps a Request's method string to an Apache HttpClient RequestBuilder; if the method is not one of GET/POST/PUT/DELETE/HEAD/PATCH/OPTIONS/TRACE it throws IllegalArgumentException('Illegal HTTP Method ' + method).

Solutions

1. Use HttpConstant.Method constants (Request.Method GET/POST/...) instead of raw strings
2. Trim/uppercase the method string before creating the Request
3. Only use methods in the supported set: GET, POST, PUT, DELETE, HEAD, PATCH, OPTIONS, TRACE

Example fix

// before
Request request = new Request("http://x", "gets");
// after
Request request = new Request("http://x", HttpConstant.Method.GET);

java.util.Set<String> ok = Set.of("GET","POST","PUT","DELETE","HEAD","PATCH","OPTIONS","TRACE"); if (!ok.contains(method.trim().toUpperCase())) throw ...;

try { spider.addRequest(request); } catch (IllegalArgumentException e) { /* fix method name */ }

When it happens

Trigger: Building a Request with Request.create(method, ...) where method is misspelled ('gets', 'post ') or an unsupported verb (e.g. 'CONNECT'); passing a null/empty method string.

Common situations: Typing the HTTP verb by hand; copying a method name from another framework; dynamically deriving the method from an upstream value that includes unexpected verbs.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• threadNum should be more than one!
• emptySleepTime should be more than zero!
• task or site can not be null
• illegal encoding
• language and script must not be null!

AI-assisted analysis of code4craft/webmagic@67816a19d6 (2026-09-08). Data as JSON: /api/errors/6c4df5d74ca87f6f. Report an issue: GitHub.

Appendix: source

Thrown at webmagic-core/src/main/java/us/codecraft/webmagic/downloader/HttpUriRequestConverter.java:107

    }

    private RequestBuilder selectRequestMethod(Request request) {
        String method = request.getMethod();
        if (method == null || method.equalsIgnoreCase(HttpConstant.Method.GET)) {
            //default get
            return RequestBuilder.get();
        } else if (method.equalsIgnoreCase(HttpConstant.Method.POST)) {
            return addFormParams(RequestBuilder.post(),request);
        } else if (method.equalsIgnoreCase(HttpConstant.Method.HEAD)) {
            return RequestBuilder.head();
        } else if (method.equalsIgnoreCase(HttpConstant.Method.PUT)) {
            return addFormParams(RequestBuilder.put(), request);
        } else if (method.equalsIgnoreCase(HttpConstant.Method.DELETE)) {
            return RequestBuilder.delete();
        } else if (method.equalsIgnoreCase(HttpConstant.Method.TRACE)) {
            return RequestBuilder.trace();
        }
        throw new IllegalArgumentException("Illegal HTTP Method " + method);
    }

    private RequestBuilder addFormParams(RequestBuilder requestBuilder, Request request) {
        if (request.getRequestBody() != null) {
            ByteArrayEntity entity = new ByteArrayEntity(request.getRequestBody().getBody());
            entity.setContentType(request.getRequestBody().getContentType());
            requestBuilder.setEntity(entity);
        }
        return requestBuilder;
    }

}

View on GitHub (pinned to 67816a19d6)