What causes "No chunks created after splitting text" in lfnovo/open-notebook?

Inspect the source's full_text to see what the chunker received

lfnovo/open-notebook · warning · ValueError

No chunks created after splitting text

Error message

No chunks created after splitting text

What it means

ValueError raised by the source embed command when chunk_text() produced zero chunks from non-empty source text. This means the text exists but the chunker's parameters (min size, separators) filtered everything out.

Source

Thrown at commands/embedding_commands.py:360

        file_path = source.asset.file_path if source.asset else None
        content_type = detect_content_type(source.full_text, file_path)
        logger.debug(f"Detected content type: {content_type.value}")

        # 4. Chunk text using appropriate splitter
        chunks = chunk_text(source.full_text, content_type=content_type)
        total_chunks = len(chunks)

        # Log chunk statistics for debugging
        chunk_sizes = [len(c) for c in chunks]
        logger.info(
            f"Created {total_chunks} chunks for source {input_data.source_id} "
            f"(sizes: min={min(chunk_sizes) if chunk_sizes else 0}, "
            f"max={max(chunk_sizes) if chunk_sizes else 0}, "
            f"avg={sum(chunk_sizes) // len(chunk_sizes) if chunk_sizes else 0} chars)"
        )

        if total_chunks == 0:
            raise ValueError("No chunks created after splitting text")

        # 5. Generate embeddings for all chunks in batches
        cmd_id = get_command_id(input_data)
        logger.debug(f"Generating embeddings for {total_chunks} chunks")
        embeddings = await generate_embeddings(chunks, command_id=cmd_id)

        # Verify we got embeddings for all chunks
        if len(embeddings) != len(chunks):
            raise ValueError(
                f"Embedding count mismatch: got {len(embeddings)} embeddings "
                f"for {len(chunks)} chunks"
            )

        # 6. Bulk INSERT source_embedding records
        records = [
            {
                "source": ensure_record_id(input_data.source_id),
                "order": idx,

View on GitHub (pinned to a7de90d38a)

Solutions

Inspect the source's full_text to see what the chunker received
Review chunk_text settings (min chunk size / separators) for your content shape
Log chunk sizes (already logged just above) to confirm the split produced zero pieces

Defensive patterns

Strategy: validation

Validate before calling

chunks = chunk_text(source.full_text)
if not chunks:
    raise RuntimeError('chunker produced no chunks — check chunk settings/content')

Prevention

Sanity-check chunker output on unusual content before embedding
Review min chunk size configuration relative to your source sizes
Log chunk size stats (already emitted) to diagnose empty splits

When it happens

Trigger: Embedding a source whose full_text is non-empty but consists only of content the chunker discards (e.g. all whitespace/separator lines above a min-chunk threshold), or chunking config with a too-large minimum chunk size.

Common situations: Very short sources with aggressive chunk settings, or unusual content (only markup/separators) that chunk_text splits into nothing.

Related errors

AI-assisted analysis of lfnovo/open-notebook@a7de90d38a (2026-08-27). Data as JSON: /api/errors/fcab66ee6cf43754. Report an issue: GitHub.