sipeed/picoclaw · error

ingest sample %s: %w

Error message

ingest sample %s: %w

What it means

engine.Ingest failed for one LOCOMO sample during bulk ingestion (cmd/membench/ingest.go). All turns of a conversation are submitted under sessionKey "locomo-<sampleID>"; any storage/embedding failure inside the engine bubbles up wrapped with the sample ID, halting the whole ingestion run at that sample.

Source

Thrown at cmd/membench/ingest.go:65

		}

		turns := GetTurns(sample)

		// Convert turns to seahorse messages
		msgs := make([]seahorse.Message, 0, len(turns))
		for _, turn := range turns {
			content := turn.Speaker + ": " + turn.Text
			msgs = append(msgs, seahorse.Message{
				Role:       "user",
				Content:    content,
				TokenCount: len(turn.Text) / 4,
			})
		}

		// Ingest all turns for this sample
		_, err := engine.Ingest(ctx, sessionKey, msgs)
		if err != nil {
			return nil, fmt.Errorf("ingest sample %s: %w", sample.SampleID, err)
		}

		// Get the conversation ID for scoped retrieval
		conv, err := store.GetConversationBySessionKey(ctx, sessionKey)
		if err != nil {
			return nil, fmt.Errorf("get conversation for %s: %w", sample.SampleID, err)
		}
		if conv == nil {
			return nil, fmt.Errorf("conversation not found for %s after ingest", sample.SampleID)
		}
		convMap[sample.SampleID] = conv.ConversationID
		log.Printf("Ingested sample %s: %d turns, convID=%d", sample.SampleID, len(turns), conv.ConversationID)
	}

	log.Printf("Seahorse ingestion complete: %d samples, %d conversations", len(samples), len(convMap))
	return &SeahorseIngestResult{
		Engine:  engine,
		ConvMap: convMap,

View on GitHub (pinned to 49183d7e8d)

Solutions

1. Inspect the wrapped error text — it carries the engine's real cause (locked, full, cancelled)
2. Free space or resolve the DB lock, then simply rerun: the existing-conversation check makes ingestion resume from the first missing sample
3. Increase or disable the context deadline for large corpora
4. Run only one ingestion against a given dbPath at a time

Example fix

// before
if _, err := engine.Ingest(ctx, sessionKey, msgs); err != nil {
    return nil, fmt.Errorf("ingest sample %s: %w", sample.SampleID, err)
}

// after
if _, err := engine.Ingest(ctx, sessionKey, msgs); err != nil {
    if ctx.Err() != nil {
        return nil, fmt.Errorf("ingest cancelled at sample %s: %w", sample.SampleID, ctx.Err())
    }
    log.Printf("sample %s failed (%v); skipping", sample.SampleID, err)
    continue
}

if existing, _ := store.GetConversationBySessionKey(ctx, sessionKey); existing != nil {
    convMap[sample.SampleID] = existing.ConversationID // already ingested; skip
    continue
}

if _, err := engine.Ingest(ctx, sessionKey, msgs); err != nil {
    if ctx.Err() != nil {
        return nil, fmt.Errorf("cancelled at %s: %w", sample.SampleID, ctx.Err())
    }
    log.Printf("skip sample %s: %v", sample.SampleID, err)
    continue // or abort, depending on desired strictness
}

When it happens

Trigger: Sqlite write error (disk full, DB locked by a concurrent reader); context cancelled mid-ingest; an individual message failing engine-internal validation; embedding backend error if the engine computes embeddings inline.

Common situations: Long ingestion runs interrupted by Ctrl-C or context timeout; DB on a volume that fills with conversation data; second concurrent membench run against the same dbPath.

Related errors

• get conversation for %s: %w
• create seahorse engine: %w
• conversation not found for %s after ingest
• create output dir: %w
• marshal result: %w

AI-assisted analysis of sipeed/picoclaw@49183d7e8d (2026-08-15). Data as JSON: /api/errors/0e741233b13f9015. Report an issue: GitHub.