{"record":{"id":"fcab66ee6cf43754","repo":"lfnovo/open-notebook","slug":"no-chunks-created-after-splitting-text","errorCode":null,"errorMessage":"No chunks created after splitting text","messagePattern":"No chunks created after splitting text","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"commands/embedding_commands.py","lineNumber":360,"sourceCode":"        file_path = source.asset.file_path if source.asset else None\n        content_type = detect_content_type(source.full_text, file_path)\n        logger.debug(f\"Detected content type: {content_type.value}\")\n\n        # 4. Chunk text using appropriate splitter\n        chunks = chunk_text(source.full_text, content_type=content_type)\n        total_chunks = len(chunks)\n\n        # Log chunk statistics for debugging\n        chunk_sizes = [len(c) for c in chunks]\n        logger.info(\n            f\"Created {total_chunks} chunks for source {input_data.source_id} \"\n            f\"(sizes: min={min(chunk_sizes) if chunk_sizes else 0}, \"\n            f\"max={max(chunk_sizes) if chunk_sizes else 0}, \"\n            f\"avg={sum(chunk_sizes) // len(chunk_sizes) if chunk_sizes else 0} chars)\"\n        )\n\n        if total_chunks == 0:\n            raise ValueError(\"No chunks created after splitting text\")\n\n        # 5. Generate embeddings for all chunks in batches\n        cmd_id = get_command_id(input_data)\n        logger.debug(f\"Generating embeddings for {total_chunks} chunks\")\n        embeddings = await generate_embeddings(chunks, command_id=cmd_id)\n\n        # Verify we got embeddings for all chunks\n        if len(embeddings) != len(chunks):\n            raise ValueError(\n                f\"Embedding count mismatch: got {len(embeddings)} embeddings \"\n                f\"for {len(chunks)} chunks\"\n            )\n\n        # 6. Bulk INSERT source_embedding records\n        records = [\n            {\n                \"source\": ensure_record_id(input_data.source_id),\n                \"order\": idx,","sourceCodeStart":342,"sourceCodeEnd":378,"githubUrl":"https://github.com/lfnovo/open-notebook/blob/a7de90d38aaf18ee85fd661854d35c11e44613e2/commands/embedding_commands.py#L342-L378","documentation":"ValueError raised by the source embed command when chunk_text() produced zero chunks from non-empty source text. This means the text exists but the chunker's parameters (min size, separators) filtered everything out.","triggerScenarios":"Embedding a source whose full_text is non-empty but consists only of content the chunker discards (e.g. all whitespace/separator lines above a min-chunk threshold), or chunking config with a too-large minimum chunk size.","commonSituations":"Very short sources with aggressive chunk settings, or unusual content (only markup/separators) that chunk_text splits into nothing.","solutions":["Inspect the source's full_text to see what the chunker received","Review chunk_text settings (min chunk size / separators) for your content shape","Log chunk sizes (already logged just above) to confirm the split produced zero pieces"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"chunks = chunk_text(source.full_text)\nif not chunks:\n    raise RuntimeError('chunker produced no chunks — check chunk settings/content')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Sanity-check chunker output on unusual content before embedding","Review min chunk size configuration relative to your source sizes","Log chunk size stats (already emitted) to diagnose empty splits"],"tags":["python","valueerror","embedding","chunking","source"],"backgroundTag":"text-chunking-empty-result","analyzedSha":"a7de90d38aaf18ee85fd661854d35c11e44613e2","analyzedAt":"2026-08-27T02:39:58.166Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}