{"record":{"id":"4daa9391e731cd33","repo":"crowdsecurity/crowdsec","slug":"unable-to-download-data-for-parser-s-w","errorCode":null,"errorMessage":"unable to download data for parser '%s': %w","messagePattern":"unable to download data for parser '(.+?)': %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/hubtest/hubtest_item.go","lineNumber":247,"sourceCode":"\n\t// load installed hub\n\thub, err := cwhub.NewHub(t.RuntimeHubConfig, nil)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := hub.Load(); err != nil {\n\t\treturn err\n\t}\n\n\t// prevent concurrent downloads of the same file\n\tdownloadMutex.Lock()\n\tdefer downloadMutex.Unlock()\n\n\t// install data for parsers if needed\n\tfor _, item := range hub.GetInstalledByType(cwhub.PARSERS, true) {\n\t\tif _, err := hubops.DownloadDataIfNeeded(ctx, hub, item, false); err != nil {\n\t\t\treturn fmt.Errorf(\"unable to download data for parser '%s': %w\", item.Name, err)\n\t\t}\n\n\t\tlog.Debugf(\"parser '%s' installed successfully in runtime environment\", item.Name)\n\t}\n\n\t// install data for scenarios if needed\n\tfor _, item := range hub.GetInstalledByType(cwhub.SCENARIOS, true) {\n\t\tif _, err := hubops.DownloadDataIfNeeded(ctx, hub, item, false); err != nil {\n\t\t\treturn fmt.Errorf(\"unable to download data for parser '%s': %w\", item.Name, err)\n\t\t}\n\n\t\tlog.Debugf(\"scenario '%s' installed successfully in runtime environment\", item.Name)\n\t}\n\n\t// install data for postoverflows if needed\n\tfor _, item := range hub.GetInstalledByType(cwhub.POSTOVERFLOWS, true) {\n\t\tif _, err := hubops.DownloadDataIfNeeded(ctx, hub, item, false); err != nil {\n\t\t\treturn fmt.Errorf(\"unable to download data for parser '%s': %w\", item.Name, err)","sourceCodeStart":229,"sourceCodeEnd":265,"githubUrl":"https://github.com/crowdsecurity/crowdsec/blob/909b5157986a2b2c2163300fdaef5ed01289f7d2/pkg/hubtest/hubtest_item.go#L229-L265","documentation":"InstallHub installs parser data files into the test runtime via hubops.DownloadDataIfNeeded for each installed parser; any download failure is wrapped as 'unable to download data for parser <name>: %w'. The hub items are installed but their associated data files (e.g. geoip databases) could not be fetched.","triggerScenarios":"HubTestItem.Run -> InstallHub with no network access to the data URL, an invalid/expired data download URL in the hub index, or hub index/data URL misconfiguration.","commonSituations":"CI without internet access; corporate proxy/firewall blocking hub data hosts; stale .index.json referencing moved data files; TLS/certificate problems; rate limiting on the data CDN.","solutions":["Read the wrapped error: DNS/timeout/refused -> fix network or proxy settings (set HTTPS_PROXY); 404 -> update the hub index.","Run `cscli hub update` / refresh .index.json so data URLs are current.","Pre-download or vendor the data files into the runtime directory so DownloadDataIfNeeded finds them locally and skips the download.","Re-run with network access enabled in CI (e.g. allow-list hub data hosts, or use a caching proxy)."],"exampleFix":"null","handlingStrategy":"retry","validationCode":"resp, err := http.Head(dataURL)\nif err != nil || resp.StatusCode != 200 {\n    log.Printf(\"data URL unreachable, check network/proxy before running hub tests\")\n}","typeGuard":null,"tryCatchPattern":"err := item.Run(ctx)\nif err != nil && strings.Contains(err.Error(), \"unable to download data\") {\n    // transient? retry with backoff; else fix network/proxy and re-run\n    time.Sleep(2 * time.Second)\n    err = item.Run(ctx)\n}","preventionTips":["Update the hub index before tests so data URLs are current.","Configure HTTPS_PROXY and CA certificates in CI containers.","Pre-cache hub data files for offline runs."],"tags":["network","download","hub","hubtest"],"backgroundTag":"http-request-failed","analyzedSha":"909b5157986a2b2c2163300fdaef5ed01289f7d2","analyzedAt":"2026-09-06T12:27:26.012Z","contentChangedAt":"2026-09-06T12:27:26.012Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}