jaegertracing/jaeger · critical

failed to prepare batch: %w

Error message

failed to prepare batch: %w

What it means

Writer.WriteTraces opens a ClickHouse batch insert via conn.PrepareBatch with the sql.InsertSpan INSERT statement. If the driver cannot prepare the batch — connection unavailable, auth failure, the spans table missing or its schema changed so the INSERT statement no longer parses — the error is wrapped as "failed to prepare batch". No span data is written; the whole WriteTraces call fails.

Source

Thrown at internal/storage/v2/clickhouse/tracestore/writer.go:33

)

type Writer struct {
	conn driver.Conn
}

// NewWriter returns a new Writer instance that uses the given ClickHouse connection
// to write trace data.
//
// The provided connection is used for writing traces.
// This connection should not have instrumentation enabled to avoid recursively generating traces.
func NewWriter(conn driver.Conn) *Writer {
	return &Writer{conn: conn}
}

func (w *Writer) WriteTraces(ctx context.Context, td ptrace.Traces) error {
	batch, err := w.conn.PrepareBatch(ctx, sql.InsertSpan)
	if err != nil {
		return fmt.Errorf("failed to prepare batch: %w", err)
	}
	defer batch.Close()
	for _, rs := range td.ResourceSpans().All() {
		for _, ss := range rs.ScopeSpans().All() {
			for _, span := range ss.Spans().All() {
				sr := dbmodel.ToRow(rs.Resource(), ss.Scope(), span)
				err = batch.Append(
					sr.ID,
					sr.TraceID,
					sr.TraceState,
					sr.ParentSpanID,
					sr.Name,
					sr.Kind,
					sr.StartTime,
					sr.StatusCode,
					sr.StatusMessage,
					sr.Duration,
					sr.Attributes.BoolKeys,

View on GitHub (pinned to 806f444784)

Solutions

1. Verify ClickHouse connectivity and credentials from the Jaeger host with clickhouse-client using the DSN values
2. Run the Jaeger ClickHouse schema migrations and confirm the spans table exists: SHOW CREATE TABLE spans
3. Diff sql.InsertSpan's column list against the actual table columns after any upgrade; re-run migrations if they drifted
4. Unwrap the error to distinguish network (dial tcp) vs server (code 60 UNKNOWN_TABLE) vs auth (code 516) causes and fix accordingly
5. Ensure the storage writer's connection is initialized and healthy at startup (connection health check)

Example fix

// before: table missing because migrations were never applied
// Error: failed to prepare batch: code: 60, message: Table default.spans does not exist

// after: apply schema first (migrations) then write
// clickhouse-client --host ... --multiquery < migrations/*.sql

// before writes, verify connectivity and table
if err := conn.Exec(ctx, "SELECT 1"); err != nil {
    return fmt.Errorf("clickhouse unavailable: %w", err)
}
var exists bool
if err := conn.QueryRow(ctx,
    "SELECT exists(SELECT 1 FROM system.tables WHERE database = ? AND name = 'spans')",
    dbName).Scan(&exists); err != nil || !exists {
    return errors.New("spans table missing: run jaeger clickhouse migrations")
}

if err := writer.WriteTraces(ctx, td); err != nil {
    if strings.Contains(err.Error(), "failed to prepare batch") {
        // non-retryable until ops fixes schema/connection
        log.Error("cannot write traces: batch prepare failed", "err", err)
        return err // do NOT drop spans silently; buffer or nack
    }
    return err
}

When it happens

Trigger: Calling WriteTraces when: ClickHouse is unreachable or credentials are wrong, the spans table does not exist (migrations not applied), the table schema was altered so sql.InsertSpan's column list mismatches, or the context is canceled before batch preparation completes.

Common situations: Fresh ClickHouse deployment without running Jaeger schema migrations; mismatched Jaeger/writer version vs table schema (column added/renamed); ClickHouse restarted under load; wrong DSN database name; auth failure after credential rotation.

Related errors

• failed to append span to batch: %w
• failed to send batch: %w
• failed to query trace IDs: %w
• failed to query dependencies: %w
• failed to query trace: %w

AI-assisted analysis of jaegertracing/jaeger@806f444784 (2026-09-01). Data as JSON: /api/errors/eb277d912e6eb381. Report an issue: GitHub.