{"record":{"id":"a0d52eb308654788","repo":"owasp-amass/amass","slug":"the-context-expired","errorCode":null,"errorMessage":"the context expired","messagePattern":"the context expired","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"internal/net/http/http.go","lineNumber":212,"sourceCode":"\treq.Header.Set(\"Accept-Language\", AcceptLang)\n\tfor k, values := range r.Header {\n\t\tfor _, v := range values {\n\t\t\treq.Header.Set(k, v)\n\t\t}\n\t}\n\n\tresp, err := client.Do(req)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn RespToAmassResponse(resp), nil\n}\n\n// Crawl will spider the web page at the URL argument looking while staying within the scope provided.\nfunc Crawl(ctx context.Context, u string, scope []string, max int, callback func(*Request, *Response)) error {\n\tselect {\n\tcase <-ctx.Done():\n\t\treturn fmt.Errorf(\"the context expired\")\n\tdefault:\n\t}\n\n\tvar count int\n\tvar m sync.Mutex\n\tfilter := bf.NewDefaultStableBloomFilter(10000, 0.01)\n\tdefer filter.Reset()\n\tattrs := []string{\"action\", \"cite\", \"data\", \"formaction\",\n\t\t\"href\", \"longdesc\", \"poster\", \"src\", \"srcset\", \"xmlns\"}\n\ttags := []string{\"a\", \"area\", \"audio\", \"base\", \"blockquote\", \"button\",\n\t\t\"embed\", \"form\", \"frame\", \"frameset\", \"html\", \"iframe\", \"img\", \"input\",\n\t\t\"ins\", \"link\", \"noframes\", \"object\", \"q\", \"script\", \"source\", \"track\", \"video\"}\n\n\tg := geziyor.NewGeziyor(&geziyor.Options{\n\t\tStartURLs:             []string{u},\n\t\tRobotsTxtDisabled:     true,\n\t\tUserAgent:             UserAgent,\n\t\tLogDisabled:           true,","sourceCodeStart":194,"sourceCodeEnd":230,"githubUrl":"https://github.com/owasp-amass/amass/blob/79299dce87b0085db0f2f4ef3e9c52cccb49f514/internal/net/http/http.go#L194-L230","documentation":"Crawl spiders a web page within a given scope, but first performs a non-blocking select on the provided context. If ctx is already done — cancelled or past its deadline — it returns 'the context expired' immediately instead of starting work. This is standard Go context propagation: the caller owns the crawl's lifetime.","triggerScenarios":"Calling Crawl with a context that is already cancelled, a deadline that expired before the call, or a very short timeout (e.g. context.WithTimeout(ctx, 1*time.Millisecond)); also as in TestCrawl, where a test context expires during crawling.","commonSituations":"HTTP-handler parent contexts ending before slow crawls finish, test harnesses with tight deadlines, goroutine shutdown racing with crawl start.","solutions":["Increase the WithTimeout duration so it covers the expected crawl time.","Do not pass an already-cancelled context; build a fresh one (context.Background() plus a new timeout).","Audit upstream code for premature cancel() calls or parent contexts ending early.","Treat the error as normal cancellation: stop work and propagate rather than retry blindly."],"exampleFix":"// before\nctx, cancel := context.WithTimeout(parent, 500*time.Millisecond)\nerr := Crawl(ctx, url, scope, max, cb)  // deadline too short -> the context expired\n// after\nctx, cancel := context.WithTimeout(parent, 30*time.Second)\nerr := Crawl(ctx, url, scope, max, cb)","handlingStrategy":"retry","validationCode":"// Go — check before calling\nselect {\ncase <-ctx.Done():\n\treturn fmt.Errorf(\"cannot start crawl: %v\", ctx.Err())\ndefault:\n}","typeGuard":null,"tryCatchPattern":"// Go\ncrawlCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)\ndefer cancel()\nif err := Crawl(crawlCtx, u, scope, max, cb); err != nil {\n\tif errors.Is(err, context.DeadlineExceeded) || strings.Contains(err.Error(), \"the context expired\") {\n\t\t// retry with a longer deadline or back off\n\t}\n}","preventionTips":["Size crawl deadlines generously; crawls are long-running by nature.","Never pass a context that may already be cancelled into Crawl.","Use context.WithTimeout on a fresh Background context for standalone crawls.","Treat this error as cancellation, not a bug; avoid tight retry loops without backoff."],"tags":["context","timeout","cancellation","crawler"],"backgroundTag":"request-timeout","analyzedSha":"79299dce87b0085db0f2f4ef3e9c52cccb49f514","analyzedAt":"2026-09-06T08:22:48.198Z","contentChangedAt":"2026-09-06T08:22:48.198Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}