gocolly/colly · error

ErrEmptyProxyURL

ErrEmptyProxyURL

Error message

Proxy URL list is empty

What it means

ErrEmptyProxyURL is returned by NewRoundRobinProxySwitcher when the supplied proxy URL list is empty. The proxy switcher needs at least one proxy to rotate through; with an empty slice there is nothing to switch to, so construction fails immediately.

Source

Thrown at colly.go:242

	// ErrMissingURL is the error type for missing URL errors
	ErrMissingURL = errors.New("Missing URL")
	// ErrMaxDepth is the error type for exceeding max depth
	ErrMaxDepth = errors.New("Max depth limit reached")
	// ErrForbiddenURL is the error thrown if visiting
	// a URL which is not allowed by URLFilters
	ErrForbiddenURL = errors.New("ForbiddenURL")

	// ErrNoURLFiltersMatch is the error thrown if visiting
	// a URL which is not allowed by URLFilters
	ErrNoURLFiltersMatch = errors.New("No URLFilters match")
	// ErrRobotsTxtBlocked is the error type for robots.txt errors
	ErrRobotsTxtBlocked = errors.New("URL blocked by robots.txt")
	// ErrNoCookieJar is the error type for missing cookie jar
	ErrNoCookieJar = errors.New("Cookie jar is not available")
	// ErrNoPattern is the error type for LimitRules without patterns
	ErrNoPattern = errors.New("No pattern defined in LimitRule")
	// ErrEmptyProxyURL is the error type for empty Proxy URL list
	ErrEmptyProxyURL = errors.New("Proxy URL list is empty")
	// ErrAbortedAfterHeaders is the error returned when OnResponseHeaders aborts the transfer.
	ErrAbortedAfterHeaders = errors.New("Aborted after receiving response headers")
	// ErrAbortedBeforeRequest is the error returned when OnResponseHeaders aborts the transfer.
	ErrAbortedBeforeRequest = errors.New("Aborted before Do Request")
	// ErrQueueFull is the error returned when the queue is full
	ErrQueueFull = errors.New("Queue MaxSize reached")
	// ErrMaxRequests is the error returned when exceeding max requests
	ErrMaxRequests = errors.New("Max Requests limit reached")
	// ErrRetryBodyUnseekable is the error when retry with not seekable body
	ErrRetryBodyUnseekable = errors.New("Retry Body Unseekable")
)

var envMap = map[string]func(*Collector, string){
	"ALLOWED_DOMAINS": func(c *Collector, val string) {
		c.AllowedDomains = strings.Split(val, ",")
	},
	"CACHE_DIR": func(c *Collector, val string) {
		c.CacheDir = val

View on GitHub (pinned to 17d1d6ca92)

Solutions

1. Ensure the proxy slice passed to NewRoundRobinProxySwitcher has at least one URL
2. Validate the proxy list at startup and fail fast with a clear message before constructing the switcher
3. If proxies are optional, only build/attach the switcher when len(proxies) > 0
4. Check the source of the list (env var, file, API) actually returned entries

Example fix

// before
proxies := strings.Split(os.Getenv("PROXY_LIST"), ",") // may be [""] or empty
switcher, _ := colly.NewRoundRobinProxySwitcher(proxies) // ErrEmptyProxyURL
// after
proxies := strings.Split(os.Getenv("PROXY_LIST"), ",")
if len(proxies) > 0 && proxies[0] != "" {
    switcher, err := colly.NewRoundRobinProxySwitcher(proxies)
    c.SetProxyFunc(switcher)
}

proxies := strings.Split(os.Getenv("PROXY_LIST"), ",")
var valid []string
for _, p := range proxies {
    if u, err := url.Parse(strings.TrimSpace(p)); err == nil && u.Scheme != "" {
        valid = append(valid, p)
    }
}
if len(valid) == 0 {
    return fmt.Errorf("no proxy URLs configured")
}
switcher, err := colly.NewRoundRobinProxySwitcher(valid)

func hasProxies(list []string) bool {
    return len(list) > 0 && list[0] != ""
}

switcher, err := colly.NewRoundRobinProxySwitcher(proxies)
if err != nil {
    if errors.Is(err, colly.ErrEmptyProxyURL) {
        log.Println("proxy list empty, running without proxy")
        return nil
    }
    return err
}

When it happens

Trigger: Calling colly.NewRoundRobinProxySwitcher([]string{}) or with a slice that is nil; building the proxy list from config/env/file parsing that yielded no entries (empty env var, missing file, all lines filtered out); passing a slice after filtering removed all invalid entries.

Common situations: Reading PROXY_LIST env var that is unset, producing an empty split; a proxies.txt file that is empty or has only comments; dynamic proxy pools that were pruned to zero healthy proxies before the switcher was created.

Related errors

• ErrNoPattern
• ErrForbiddenDomain
• ErrMissingURL
• ErrMaxDepth
• ErrForbiddenURL

AI-assisted analysis of gocolly/colly@17d1d6ca92 (2026-08-30). Data as JSON: /api/errors/21b965f96484b8fd. Report an issue: GitHub.