{"record":{"id":"c53cc6944c528f08","repo":"gocolly/colly","slug":"errnourlfiltersmatch","errorCode":"ErrNoURLFiltersMatch","errorMessage":"No URLFilters match","messagePattern":"No URLFilters match","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"colly.go","lineNumber":234,"sourceCode":"\n// The prefix for environment variables of Colly settings\nconst envVariablePrefix = \"COLLY_\"\n\nvar (\n\t// ErrForbiddenDomain is the error thrown if visiting\n\t// a domain which is not allowed in AllowedDomains\n\tErrForbiddenDomain = errors.New(\"Forbidden domain\")\n\t// ErrMissingURL is the error type for missing URL errors\n\tErrMissingURL = errors.New(\"Missing URL\")\n\t// ErrMaxDepth is the error type for exceeding max depth\n\tErrMaxDepth = errors.New(\"Max depth limit reached\")\n\t// ErrForbiddenURL is the error thrown if visiting\n\t// a URL which is not allowed by URLFilters\n\tErrForbiddenURL = errors.New(\"ForbiddenURL\")\n\n\t// ErrNoURLFiltersMatch is the error thrown if visiting\n\t// a URL which is not allowed by URLFilters\n\tErrNoURLFiltersMatch = errors.New(\"No URLFilters match\")\n\t// ErrRobotsTxtBlocked is the error type for robots.txt errors\n\tErrRobotsTxtBlocked = errors.New(\"URL blocked by robots.txt\")\n\t// ErrNoCookieJar is the error type for missing cookie jar\n\tErrNoCookieJar = errors.New(\"Cookie jar is not available\")\n\t// ErrNoPattern is the error type for LimitRules without patterns\n\tErrNoPattern = errors.New(\"No pattern defined in LimitRule\")\n\t// ErrEmptyProxyURL is the error type for empty Proxy URL list\n\tErrEmptyProxyURL = errors.New(\"Proxy URL list is empty\")\n\t// ErrAbortedAfterHeaders is the error returned when OnResponseHeaders aborts the transfer.\n\tErrAbortedAfterHeaders = errors.New(\"Aborted after receiving response headers\")\n\t// ErrAbortedBeforeRequest is the error returned when OnResponseHeaders aborts the transfer.\n\tErrAbortedBeforeRequest = errors.New(\"Aborted before Do Request\")\n\t// ErrQueueFull is the error returned when the queue is full\n\tErrQueueFull = errors.New(\"Queue MaxSize reached\")\n\t// ErrMaxRequests is the error returned when exceeding max requests\n\tErrMaxRequests = errors.New(\"Max Requests limit reached\")\n\t// ErrRetryBodyUnseekable is the error when retry with not seekable body\n\tErrRetryBodyUnseekable = errors.New(\"Retry Body Unseekable\")","sourceCodeStart":216,"sourceCodeEnd":252,"githubUrl":"https://github.com/gocolly/colly/blob/17d1d6ca92bd32a5651f34256bf7a2855c967f65/colly.go#L216-L252","documentation":"ErrNoURLFiltersMatch is returned when a URL to visit matches none of the collector's URLFilters rules. When URLFilters are configured, they act as a whitelist: a URL that satisfies no rule is rejected. This differs from ErrForbiddenURL in that no rule explicitly disallowed the URL — it simply failed to match any allow rule.","triggerScenarios":"Calling c.Visit() on a URL when c.URLFilters is populated but no compiled rule matches the URL; mixing URLFilter (single func) and URLFilters (rule list) where the rule list is the active check; rules written for a different path/scheme than the URLs actually encountered.","commonSituations":"Setting c.URLFilters with pattern rules for one site layout and crawling a differently-shaped URL; regex anchors (^/$) that fail on query strings or trailing slashes; copying filter config between projects whose URL spaces differ.","solutions":["Add a rule to c.URLFilters that matches the URL you want to visit","Test the failing URL against each filter rule/regex to find the non-match","Relax or remove URLFilters if URL-level whitelisting is not needed","Normalize the URL (scheme, query, trailing slash) before Visit so it matches the rules"],"exampleFix":"// before\nc := colly.NewCollector()\nc.URLFilters = []*regexp.Regexp{regexp.MustCompile(`^https://example\\.com/blog/\\d+$`)}\nc.Visit(\"https://example.com/blog/post/abc\") // no rule matches\n// after\nc := colly.NewCollector()\nc.URLFilters = []*regexp.Regexp{regexp.MustCompile(`^https://example\\.com/blog/[\\w-]+$`)}\nc.Visit(\"https://example.com/blog/post/abc\")","handlingStrategy":"validation","validationCode":"for _, r := range filters {\n    if r.MatchString(target) {\n        break\n    }\n    // if loop completes, no rule matches -> would trigger ErrNoURLFiltersMatch\n}","typeGuard":null,"tryCatchPattern":"if err := c.Visit(target); err != nil && errors.Is(err, colly.ErrNoURLFiltersMatch) {\n    log.Printf(\"no URLFilters rule matches %s\", target)\n    return nil\n}","preventionTips":["Test every URL shape your crawl will encounter against the rule set","Prefer permissive prefix regexes over tightly anchored ones","When updating site structure, re-audit URLFilters rules"],"tags":["colly","scraping","url-filter","regex","go"],"backgroundTag":"url-blocked-by-filter","analyzedSha":"17d1d6ca92bd32a5651f34256bf7a2855c967f65","analyzedAt":"2026-08-30T21:32:31.579Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}