lfnovo/open-notebook · warning · ValueError

Source '{input_data.source_id}' has no text to embed

Error message

Source '{input_data.source_id}' has no text to embed

What it means

ValueError raised by the source embed command when the source exists but full_text is empty or whitespace-only. Sources must have completed text extraction before embedding; this guard fails before deleting/re-creating embeddings.

Source

Thrown at commands/embedding_commands.py:332

    3. Detect content type from file path or content
    4. Chunk text using appropriate splitter
    5. Generate embeddings for all chunks in batches
    6. Bulk INSERT source_embedding records

    Retry Strategy:
    - Retries up to 5 times for transient failures (network, timeout, etc.)
    - Uses exponential-jitter backoff (1-60s)
    - Does NOT retry permanent failures (ValueError for validation errors)
    """

    async def embed() -> Tuple[Dict[str, Any], str]:
        # 1. Load source
        source = await Source.get(input_data.source_id)
        if not source:
            raise ValueError(f"Source '{input_data.source_id}' not found")

        if not source.full_text or not source.full_text.strip():
            raise ValueError(f"Source '{input_data.source_id}' has no text to embed")

        # 2. DELETE existing embeddings (idempotency)
        logger.debug(f"Deleting existing embeddings for source {input_data.source_id}")
        await repo_query(
            "DELETE source_embedding WHERE source = $source_id",
            {"source_id": ensure_record_id(input_data.source_id)},
        )

        # 3. Detect content type from file path if available
        file_path = source.asset.file_path if source.asset else None
        content_type = detect_content_type(source.full_text, file_path)
        logger.debug(f"Detected content type: {content_type.value}")

        # 4. Chunk text using appropriate splitter
        chunks = chunk_text(source.full_text, content_type=content_type)
        total_chunks = len(chunks)

        # Log chunk statistics for debugging

View on GitHub (pinned to a7de90d38a)

Solutions

1. Re-open the source and let ingestion/text-extraction complete, then re-run embedding
2. Check the worker (make worker-start) is running so ingestion jobs actually process
3. Inspect the source record in SurrealDB to confirm full_text is populated

source = await Source.get(source_id)
if not source.full_text or not source.full_text.strip():
    raise RuntimeError(f'source {source_id} ingestion incomplete — full_text empty')

def source_ready_for_embedding(source) -> bool:
    return bool(source and source.full_text and source.full_text.strip())

When it happens

Trigger: embed_source runs on a source whose content fetch or transcription did not finish (or failed silently), leaving full_text empty — e.g. a podcast source where transcription never completed.

Common situations: See trigger scenarios.

Related errors

• {label} '{record_id}' has no content to embed
• Source '{input_data.source_id}' not found
• No chunks created after splitting text
• {label} '{record_id}' not found
• Embedding count mismatch: got {len(embeddings)} embeddings f

AI-assisted analysis of lfnovo/open-notebook@a7de90d38a (2026-08-27). Data as JSON: /api/errors/36a2e4558ce2fdb8. Report an issue: GitHub.