What causes "failed to download %s: %w" in xpzouying/xiaohongshu-mcp?

Unwrap the returned joined errors (errors.As / strings.Split on the joined message) to find which URLs failed and why, then handle per cause

xpzouying/xiaohongshu-mcp · error

failed to download %s: %w

Error message

failed to download %s: %w

What it means

DownloadImages loops over imageURLs, calls DownloadImage for each, and aggregates per-URL failures via %w wrapping while continuing with the rest. This error reports that a specific URL failed, with the underlying cause (e.g. the 'download failed with status %d' error from DownloadImage) available through errors.Unwrap/As.

Source

Thrown at pkg/downloader/images.go:114

	}

	// 保存到文件
	if err := os.WriteFile(filePath, imageData, 0644); err != nil {
		return "", errors.Wrap(err, "failed to save image")
	}

	return filePath, nil
}

// DownloadImages 批量下载图片
func (d *ImageDownloader) DownloadImages(imageURLs []string) ([]string, error) {
	var localPaths []string
	var errs []error

	for _, imageURL := range imageURLs {
		localPath, err := d.DownloadImage(imageURL)
		if err != nil {
			errs = append(errs, fmt.Errorf("failed to download %s: %w", imageURL, err))
			continue
		}
		localPaths = append(localPaths, localPath)
	}

	if len(errs) > 0 {
		return localPaths, fmt.Errorf("download errors occurred: %v", errs)
	}

	return localPaths, nil
}

// isValidImageURL 检查是否为有效的图片URL
func (d *ImageDownloader) isValidImageURL(rawURL string) bool {
	// 检查是否以http/https开头
	if !strings.HasPrefix(strings.ToLower(rawURL), "http://") &&
		!strings.HasPrefix(strings.ToLower(rawURL), "https://") {
		return false

View on GitHub (pinned to 332d196854)

Solutions

Unwrap the returned joined errors (errors.As / strings.Split on the joined message) to find which URLs failed and why, then handle per cause
Re-run DownloadImages with only the failed URLs after addressing the cause (backoff for 429, fresh URLs for 404/403)
Add per-host throttling or concurrency limits to avoid rate limiting during batch downloads
Treat it as partial failure: successful localPaths are still returned/collected, so continue processing what succeeded
Pre-validate URLs (HEAD request or scheme/host check) before the batch to skip known-bad ones

Example fix

// before
localPath, err := d.DownloadImage(imageURL)
if err != nil {
	errs = append(errs, fmt.Errorf("failed to download %s: %w", imageURL, err))
	continue
}
// after — retry transient failures once before giving up
localPath, err := d.DownloadImage(imageURL)
if err != nil {
	time.Sleep(2 * time.Second)
	localPath, err = d.DownloadImage(imageURL)
	if err != nil {
		errs = append(errs, fmt.Errorf("failed to download %s: %w", imageURL, err))
		continue
	}
}

Defensive patterns

Strategy: try-catch

Validate before calling

// 批量前预检
valid := make([]string, 0, len(imageURLs))
for _, u := range imageURLs {
	if pu, err := url.Parse(u); err == nil && (pu.Scheme == "http" || pu.Scheme == "https") && pu.Host != "" {
		valid = append(valid, u)
	}
}

Try / catch

paths, err := d.DownloadImages(ctx, urls)
if err != nil {
	// 部分失败：paths 中成功的仍可用；对 err 逐条解包定位失败 URL
	for _, u := range strings.Split(err.Error(), "\n") {
		log.Printf("批量下载部分失败: %s", u)
	}
	retryFailed(extractFailedURLs(err))
}

Prevention

Treat batch downloads as partial-failure operations; check both returned paths and error
Unwrap wrapped errors to identify per-URL causes before blanket retries
Limit concurrency and add per-host rate limits for large batches
Persist failed URLs to a queue for a later retry pass

When it happens

Trigger: Any imageURL in the input slice for which d.DownloadImage returns an error (non-200 response, network failure, read failure); the error is wrapped as "failed to download <imageURL>: <cause>" and appended to errs; after the loop all wrapped errors are joined and returned.

Common situations: Batch-scraping a page where some images return 403/404 (hotlink protection, removed assets); one host rate-limits (429) mid-batch; DNS or TLS failures for a single CDN domain; expired signed URLs among otherwise valid ones.

Related errors

AI-assisted analysis of xpzouying/xiaohongshu-mcp@332d196854 (2026-09-05). Data as JSON: /api/errors/1d20142185c66275. Report an issue: GitHub.