{"record":{"id":"df8756720aa3cf35","repo":"gocolly/colly","slug":"errmaxdepth","errorCode":"ErrMaxDepth","errorMessage":"Max depth limit reached","messagePattern":"Max depth limit reached","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"info","filePath":"colly.go","lineNumber":227,"sourceCode":"type key int\n\n// ProxyURLKey is the context key for the request proxy address.\nconst (\n\tProxyURLKey key = iota\n\tCheckRevisitKey\n)\n\n// The prefix for environment variables of Colly settings\nconst envVariablePrefix = \"COLLY_\"\n\nvar (\n\t// ErrForbiddenDomain is the error thrown if visiting\n\t// a domain which is not allowed in AllowedDomains\n\tErrForbiddenDomain = errors.New(\"Forbidden domain\")\n\t// ErrMissingURL is the error type for missing URL errors\n\tErrMissingURL = errors.New(\"Missing URL\")\n\t// ErrMaxDepth is the error type for exceeding max depth\n\tErrMaxDepth = errors.New(\"Max depth limit reached\")\n\t// ErrForbiddenURL is the error thrown if visiting\n\t// a URL which is not allowed by URLFilters\n\tErrForbiddenURL = errors.New(\"ForbiddenURL\")\n\n\t// ErrNoURLFiltersMatch is the error thrown if visiting\n\t// a URL which is not allowed by URLFilters\n\tErrNoURLFiltersMatch = errors.New(\"No URLFilters match\")\n\t// ErrRobotsTxtBlocked is the error type for robots.txt errors\n\tErrRobotsTxtBlocked = errors.New(\"URL blocked by robots.txt\")\n\t// ErrNoCookieJar is the error type for missing cookie jar\n\tErrNoCookieJar = errors.New(\"Cookie jar is not available\")\n\t// ErrNoPattern is the error type for LimitRules without patterns\n\tErrNoPattern = errors.New(\"No pattern defined in LimitRule\")\n\t// ErrEmptyProxyURL is the error type for empty Proxy URL list\n\tErrEmptyProxyURL = errors.New(\"Proxy URL list is empty\")\n\t// ErrAbortedAfterHeaders is the error returned when OnResponseHeaders aborts the transfer.\n\tErrAbortedAfterHeaders = errors.New(\"Aborted after receiving response headers\")\n\t// ErrAbortedBeforeRequest is the error returned when OnResponseHeaders aborts the transfer.","sourceCodeStart":209,"sourceCodeEnd":245,"githubUrl":"https://github.com/gocolly/colly/blob/17d1d6ca92bd32a5651f34256bf7a2855c967f65/colly.go#L209-L245","documentation":"ErrMaxDepth is returned when a link passed to Visit() would exceed the collector's MaxDepth limit. Colly tracks the crawl depth of each request (depth 0 for direct Visit calls, incremented per followed link via OnHTML->Visit) and refuses to crawl deeper than the configured limit. This is an intentional boundary, not a network failure.","triggerScenarios":"Calling c.Visit() from inside OnHTML/OnRequest handlers when c.Depth() already equals MaxDepth; setting c.MaxDepth(1) (or similar) and recursively following links; unbounded link-following loops that quickly hit the configured depth cap.","commonSituations":"Crawlers that only want the target page plus one level of links but whose handler keeps visiting; forgetting that depth counts from the initial Visit; recursive sitemap/category crawls whose structure is deeper than the configured MaxDepth; test TestCollectorMaxDepth-style exercises.","solutions":["Increase the limit: c.MaxDepth(n) with n large enough for your crawl tree","Guard recursion: inside the handler check c.Depth() < maxDepth before calling Visit","If you only need depth-limited traversal, keep MaxDepth but accept the error as the stop signal","If unlimited crawling is intended, call c.MaxDepth(0)... note depth checks only apply when MaxDepth is set >0 — otherwise remove manual recursive Visit calls"],"exampleFix":"// before\nc.OnHTML(\"a[href]\", func(e *colly.HTMLElement) {\n    c.Visit(e.Request.AbsoluteURL(e.Attr(\"href\"))) // ErrMaxDepth past limit\n})\n// after\nc.MaxDepth(3)\nc.OnHTML(\"a[href]\", func(e *colly.HTMLElement) {\n    if c.Depth() < 3 {\n        c.Visit(e.Request.AbsoluteURL(e.Attr(\"href\")))\n    }\n})","handlingStrategy":"validation","validationCode":"const maxDepth = 3\nif c.Depth() >= maxDepth {\n    return nil // would trigger ErrMaxDepth\n}","typeGuard":null,"tryCatchPattern":"if err := c.Visit(u); err != nil && errors.Is(err, colly.ErrMaxDepth) {\n    return nil // depth boundary reached, stop silently\n}","preventionTips":["Set MaxDepth explicitly and compare c.Depth() before recursive Visits","Model your crawl tree depth before choosing MaxDepth","Prefer explicit depth checks over relying on the error as flow control"],"tags":["colly","scraping","recursion-limit","go"],"backgroundTag":"max-depth-exceeded","analyzedSha":"17d1d6ca92bd32a5651f34256bf7a2855c967f65","analyzedAt":"2026-08-30T21:32:31.579Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}