{"record":{"id":"1a025c9bafeb450b","repo":"crowdsecurity/crowdsec","slug":"while-downloading-data-for-s-w-1a025c","errorCode":null,"errorMessage":"while downloading data for %s: %w","messagePattern":"while downloading data for (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/hubops/download.go","lineNumber":196,"sourceCode":"\n\tif downloaded {\n\t\tplan.ReloadNeeded = true\n\t}\n\n\ti.State.Tainted = false\n\ti.State.UpToDate = true\n\n\t// read content to get the list of data files\n\treader, err := os.Open(finalPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"while opening %s: %w\", finalPath, err)\n\t}\n\n\tdefer reader.Close()\n\n\tneedReload, err := downloadDataSet(ctx, plan.hub.GetDataDir(), c.Force, reader)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"while downloading data for %s: %w\", i.FileName, err)\n\t}\n\n\tif needReload {\n\t\tplan.ReloadNeeded = true\n\t}\n\n\treturn nil\n}\n\nfunc (*DownloadCommand) OperationType() string {\n\treturn \"download\"\n}\n\nfunc (c *DownloadCommand) ItemType() string {\n\treturn c.Item.Type\n}\n\nfunc (c *DownloadCommand) Detail() string {","sourceCodeStart":178,"sourceCodeEnd":214,"githubUrl":"https://github.com/crowdsecurity/crowdsec/blob/909b5157986a2b2c2163300fdaef5ed01289f7d2/pkg/hubops/download.go#L178-L214","documentation":"Wraps any error surfaced by downloadDataSet (which itself aggregates URL validation and per-data-file download failures) while processing the data files belonging to item i. It attaches the item's FileName so the user knows which item's data could not be set up.","triggerScenarios":"Download Run on an item whose .data.yaml references data files that are unreachable, have invalid URLs, or fail to write into the hub data directory — the underlying cause is wrapped by errors 1090/1091 upstream in the same call chain.","commonSituations":"A parser/collection ships a data section pointing at a dead blocklist URL; offline install of an item requiring remote data; expired 7-day shelf-life file needing refresh but the host is offline.","solutions":["Inspect the wrapped inner error to find which data file failed and why.","Test/fix the offending data SourceURL, or remove the item if its data source is gone upstream.","Retry when network access is restored; note downloaded data files expire after 7 days and must be refreshed.","Use `cscli hub list -a` and item-specific inspect commands to see the data section state."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// verify each data entry before download\nfor _, d := range item.DataSources {\n    if u, err := url.Parse(d.SourceURL); err != nil || u.Scheme == \"\" {\n        return fmt.Errorf(\"bad data URL for %s: %s\", item.FileName, d.SourceURL)\n    }\n}","typeGuard":null,"tryCatchPattern":"err := runDownloadPlan(ctx, plan)\nif err != nil {\n    log.Fatalf(\"while downloading data: %v\", err) // inner error names the failing item/file\n}","preventionTips":["Inspect an item's data section with `cscli <type> inspect <item>` before enabling it.","Provide network access or offline mirrors for remote blocklist data.","Remember downloaded data expires after 7 days; schedule hub refresh accordingly.","Check disk space and data-dir permissions on hosts with large blocklists."],"tags":["hub","data-download","wrapped-error","network"],"backgroundTag":"http-request-failed","analyzedSha":"909b5157986a2b2c2163300fdaef5ed01289f7d2","analyzedAt":"2026-09-06T12:27:26.012Z","contentChangedAt":"2026-09-06T12:27:26.012Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}