gocolly/colly · info

ErrAbortedAfterHeaders

ErrAbortedAfterHeaders

Error message

Aborted after receiving response headers

What it means

ErrAbortedAfterHeaders is returned by Do when an OnResponseHeaders callback aborts the transfer. After Colly has received the HTTP response status and headers but before reading the body, the user callback may return/trigger an abort; the request is then cancelled and this sentinel error is produced. It is an intentional, user-driven cancellation — not a network or server fault.

Source

Thrown at colly.go:244

	// ErrMaxDepth is the error type for exceeding max depth
	ErrMaxDepth = errors.New("Max depth limit reached")
	// ErrForbiddenURL is the error thrown if visiting
	// a URL which is not allowed by URLFilters
	ErrForbiddenURL = errors.New("ForbiddenURL")

	// ErrNoURLFiltersMatch is the error thrown if visiting
	// a URL which is not allowed by URLFilters
	ErrNoURLFiltersMatch = errors.New("No URLFilters match")
	// ErrRobotsTxtBlocked is the error type for robots.txt errors
	ErrRobotsTxtBlocked = errors.New("URL blocked by robots.txt")
	// ErrNoCookieJar is the error type for missing cookie jar
	ErrNoCookieJar = errors.New("Cookie jar is not available")
	// ErrNoPattern is the error type for LimitRules without patterns
	ErrNoPattern = errors.New("No pattern defined in LimitRule")
	// ErrEmptyProxyURL is the error type for empty Proxy URL list
	ErrEmptyProxyURL = errors.New("Proxy URL list is empty")
	// ErrAbortedAfterHeaders is the error returned when OnResponseHeaders aborts the transfer.
	ErrAbortedAfterHeaders = errors.New("Aborted after receiving response headers")
	// ErrAbortedBeforeRequest is the error returned when OnResponseHeaders aborts the transfer.
	ErrAbortedBeforeRequest = errors.New("Aborted before Do Request")
	// ErrQueueFull is the error returned when the queue is full
	ErrQueueFull = errors.New("Queue MaxSize reached")
	// ErrMaxRequests is the error returned when exceeding max requests
	ErrMaxRequests = errors.New("Max Requests limit reached")
	// ErrRetryBodyUnseekable is the error when retry with not seekable body
	ErrRetryBodyUnseekable = errors.New("Retry Body Unseekable")
)

var envMap = map[string]func(*Collector, string){
	"ALLOWED_DOMAINS": func(c *Collector, val string) {
		c.AllowedDomains = strings.Split(val, ",")
	},
	"CACHE_DIR": func(c *Collector, val string) {
		c.CacheDir = val
	},
	"DETECT_CHARSET": func(c *Collector, val string) {

View on GitHub (pinned to 17d1d6ca92)

Solutions

1. Treat this error as expected control flow: match on it and skip/retry-free handling rather than logging as failure
2. If the abort was unintended, remove the c.Abort() call or narrow its condition in the OnResponseHeaders callback
3. Check response headers in the callback before aborting to make the condition precise (e.g. only abort on Content-Type you truly want to skip)
4. If you need the body anyway, do your filtering in OnResponse/OnHTML instead of aborting after headers

Example fix

// before
c.OnResponseHeaders(func(r *colly.ResponseHeaders) {
    if r.Headers.Get("Content-Type") != "text/html" {
        c.Abort()
    }
})
// err == ErrAbortedAfterHeaders is surfaced for every non-HTML URL
// after
c.OnResponseHeaders(func(r *colly.ResponseHeaders) {
    ct := r.Headers.Get("Content-Type")
    if strings.HasPrefix(ct, "image/") {
        c.Abort() // only abort what you truly want to skip
    }
})
// at call site:
if errors.Is(err, colly.ErrAbortedAfterHeaders) {
    return nil // expected skip
}

func isAbortedAfterHeaders(err error) bool {
    return errors.Is(err, colly.ErrAbortedAfterHeaders)
}

if err := c.Visit(u); err != nil {
    if errors.Is(err, colly.ErrAbortedAfterHeaders) {
        return nil // intentional skip from OnResponseHeaders
    }
    return err
}

When it happens

Trigger: Calling c.Abort() (or returning an abort) inside c.OnResponseHeaders(fn) after inspecting Content-Type, Content-Length, status code, etc.; conditional abort logic (e.g. skip binary/large responses) that fires on the visited URL; returning this sentinel deliberately from custom middleware built on OnResponseHeaders.

Common situations: Skipping downloads of non-HTML or oversized responses by aborting after headers; aborting redirects or auth-challenged responses early to save bandwidth; pipelines where an unexpected Content-Type should terminate the request quietly.

Related errors

• ErrForbiddenDomain
• ErrMissingURL
• ErrMaxDepth
• ErrForbiddenURL
• ErrNoURLFiltersMatch

AI-assisted analysis of gocolly/colly@17d1d6ca92 (2026-08-30). Data as JSON: /api/errors/7732648cca56ed3b. Report an issue: GitHub.