Tencent/WeKnora · error

open source file: %w

Error message

open source file: %w

What it means

The parse step (run by Process via the async task) fetches the stored file through fileService.GetFile using document.ResourceRef; any failure there is wrapped as 'open source file'. Root causes live in the file service/storage layer: missing object, bad credentials, or storage backend errors.

Source

Thrown at internal/application/service/temporary_document.go:353

	chunks := make([]types.TemporaryDocumentChunk, 0, len(parts))
	for _, part := range parts {
		chunks = append(chunks, types.TemporaryDocumentChunk{
			Seq: part.Seq, Content: part.Content, ContextHeader: part.ContextHeader,
			Start: part.Start, End: part.End, TokenCount: chunker.ApproxTokenCount(part.EmbeddingContent(), lang),
		})
	}
	chunksJSON, _ := json.Marshal(chunks)
	imagesJSON, _ := json.Marshal(images)
	metadataJSON, _ := json.Marshal(metadata)
	return s.repo.MarkReady(ctx, payload.TenantID, payload.DocumentID, content,
		types.JSON(chunksJSON), types.JSON(imagesJSON), types.JSON(metadataJSON),
		chunker.ApproxTokenCount(content, lang), len(chunks), time.Now())
}

func (s *temporaryDocumentService) parse(ctx context.Context, document *types.TemporaryDocument) (string, []types.TemporaryDocumentImage, map[string]string, error) {
	file, err := s.fileService.GetFile(ctx, document.ResourceRef)
	if err != nil {
		return "", nil, nil, fmt.Errorf("open source file: %w", err)
	}
	defer file.Close()
	data, err := io.ReadAll(io.LimitReader(file, secutils.GetMaxFileSizeMB()*1024*1024+1))
	if err != nil {
		return "", nil, nil, fmt.Errorf("read source file: %w", err)
	}
	ext := document.FileType
	var options types.TemporaryDocumentCreateOptions
	_ = json.Unmarshal(document.ProcessingOptions, &options)
	if options.ParserEngine == "" || options.ParserEngine == "auto" {
		if tenant, ok := ctx.Value(types.TenantInfoContextKey).(*types.Tenant); ok && tenant != nil {
			options.ParserEngine = tenant.ParserEngineConfig.ResolveChatParserEngine(ext)
		}
	}
	if _, ok := temporaryTextExtensions[ext]; ok && (options.ParserEngine == "" || options.ParserEngine == "auto") {
		return string(data), nil, map[string]string{"parser": "plain_text"}, nil
	}
	if docparser.IsAudioFormat(ext) {

View on GitHub (pinned to 988cbb0330)

Solutions

1. Check document.ResourceRef exists in the storage backend and the file service can read it
2. Verify storage credentials/config for the environment the worker runs in
3. Inspect the wrapped cause (%w chain) to distinguish not-found vs permission vs network
4. Re-trigger the parse after fixing storage; or re-upload the source file if the object is gone

Example fix

// before
file, err := s.fileService.GetFile(ctx, document.ResourceRef)
if err != nil {
    return "", nil, nil, fmt.Errorf("open source file: %w", err)
}
// after
file, err := s.fileService.GetFile(ctx, document.ResourceRef)
if err != nil {
    logger.Error("parse: cannot open source", "ref", document.ResourceRef, "doc", document.ID, "err", err)
    return "", nil, nil, fmt.Errorf("open source file: %w", err)
}

// Pre-check the object exists before enqueuing parse:
if _, err := fileService.Stat(ctx, doc.ResourceRef); err != nil { return fmt.Errorf("source missing: %w", err) }

err := svc.Process(ctx, task)
if err != nil && strings.HasPrefix(err.Error(), "open source file:") {
    if errors.Is(err, os.ErrNotExist) { markPermanentFailure(doc) } else { return err } // asynq retries transient
}

When it happens

Trigger: ResourceRef points to a deleted or never-uploaded object; storage credentials invalid/expired; storage backend (S3/GCS/local FS) unreachable; path format changes after migration.

Common situations: Lifecycle rules deleting objects before parsing runs; expired presigned credentials; local dev paths that don't exist in prod; storage region misconfiguration.

Related errors

• read source file: %w
• decode suggestion JSON: %w
• parse document: %w
• attachment %s failed to parse: %s
• readability parse: %w

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/c3184648b7919123. Report an issue: GitHub.