Tencent/WeKnora · error

downloaded data is not an image (sniffed: %s)

Error message

downloaded data is not an image (sniffed: %s)

What it means

When the server responds with Content-Type application/octet-stream, downloadImage sniffs the actual type with http.DetectContentType; if the sniffed type does not start with image/, the payload is rejected with this error. This prevents non-image data masquerading as an image from entering the pipeline.

Source

Thrown at internal/infrastructure/docparser/image_resolver.go:1053

	}

	// Read body with size limit.
	limited := io.LimitReader(resp.Body, maxRemoteImageSize+1)
	body, err := io.ReadAll(limited)
	if err != nil {
		return nil, "", fmt.Errorf("read body: %w", err)
	}
	if len(body) > maxRemoteImageSize {
		return nil, "", fmt.Errorf("image exceeds %d bytes limit", maxRemoteImageSize)
	}

	// If MIME was octet-stream, sniff the real type from body.
	if mimeType == "application/octet-stream" {
		detected := http.DetectContentType(body)
		if strings.HasPrefix(detected, "image/") {
			mimeType = detected
		} else {
			return nil, "", fmt.Errorf("downloaded data is not an image (sniffed: %s)", detected)
		}
	}

	return body, mimeType, nil
}

// extFromURLPath extracts the image file extension from the URL path segment.
func extFromURLPath(rawURL string) string {
	p := path.Ext(path.Base(rawURL))
	switch strings.ToLower(p) {
	case ".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp", ".svg":
		return strings.ToLower(p)
	default:
		return ""
	}
}

View on GitHub (pinned to 988cbb0330)

Solutions

1. Open the image URL in a browser/curl to confirm it actually serves image bytes; fix or replace the URL in the source document.
2. Fix the server/storage to send a correct image Content-Type (e.g. image/png) so sniffing is not needed.
3. Remove any auth or bot-protection requirement on the image URL so fetchers get the real bytes.
4. Check that the URL is not a redirect landing on an HTML error or JSON error body.

Example fix

// before: S3 object stored with wrong content type
aws s3 cp bad.png s3://bucket/bad.png --content-type application/octet-stream
// after
aws s3 cp bad.png s3://bucket/bad.png --content-type image/png

resp, err := http.Get(imgURL)
if err != nil { return err }
ct := resp.Header.Get("Content-Type")
if ct == "" || ct == "application/octet-stream" {
    head := make([]byte, 512)
    n, _ := io.ReadFull(resp.Body, head)
    detected := http.DetectContentType(head[:n])
    if !strings.HasPrefix(detected, "image/") {
        return fmt.Errorf("url %s serves %s, not an image", imgURL, detected)
    }
}

func isImageContentType(contentType string) bool {
    mt, _, err := mime.ParseMediaType(contentType)
    return err == nil && strings.HasPrefix(mt, "image/")
}

body, mimeType, err := resolver.fetchAndStoreRemoteImage(ctx, req, imgURL)
if err != nil {
    if strings.Contains(err.Error(), "not an image") {
        log.Warnf("image URL %s returned non-image data; skipping", imgURL)
        return nil
    }
    return err
}

When it happens

Trigger: fetchAndStoreRemoteImage -> downloadImage got a 200 response with Content-Type application/octet-stream whose body sniffs as text/html, application/json, text/plain, etc. — typically an HTML error/login page or a JSON API response instead of the image bytes.

Common situations: Expired or auth-gated image URLs returning an HTML login/error page with 200; CDN bot protection returning a challenge page; misconfigured object storage serving wrong Content-Type; URL points to an API endpoint rather than the image file.

Related errors

• image exceeds %d bytes limit
• rerank call failed: %w
• failed to do bulk: %w
• failed to delete by query: %w
• failed to get file from KS3: %w

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/ca643ba25ec867ac. Report an issue: GitHub.