{"record":{"id":"eb277d912e6eb381","repo":"jaegertracing/jaeger","slug":"failed-to-prepare-batch-w-eb277d","errorCode":null,"errorMessage":"failed to prepare batch: %w","messagePattern":"failed to prepare batch: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"internal/storage/v2/clickhouse/tracestore/writer.go","lineNumber":33,"sourceCode":")\n\ntype Writer struct {\n\tconn driver.Conn\n}\n\n// NewWriter returns a new Writer instance that uses the given ClickHouse connection\n// to write trace data.\n//\n// The provided connection is used for writing traces.\n// This connection should not have instrumentation enabled to avoid recursively generating traces.\nfunc NewWriter(conn driver.Conn) *Writer {\n\treturn &Writer{conn: conn}\n}\n\nfunc (w *Writer) WriteTraces(ctx context.Context, td ptrace.Traces) error {\n\tbatch, err := w.conn.PrepareBatch(ctx, sql.InsertSpan)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to prepare batch: %w\", err)\n\t}\n\tdefer batch.Close()\n\tfor _, rs := range td.ResourceSpans().All() {\n\t\tfor _, ss := range rs.ScopeSpans().All() {\n\t\t\tfor _, span := range ss.Spans().All() {\n\t\t\t\tsr := dbmodel.ToRow(rs.Resource(), ss.Scope(), span)\n\t\t\t\terr = batch.Append(\n\t\t\t\t\tsr.ID,\n\t\t\t\t\tsr.TraceID,\n\t\t\t\t\tsr.TraceState,\n\t\t\t\t\tsr.ParentSpanID,\n\t\t\t\t\tsr.Name,\n\t\t\t\t\tsr.Kind,\n\t\t\t\t\tsr.StartTime,\n\t\t\t\t\tsr.StatusCode,\n\t\t\t\t\tsr.StatusMessage,\n\t\t\t\t\tsr.Duration,\n\t\t\t\t\tsr.Attributes.BoolKeys,","sourceCodeStart":15,"sourceCodeEnd":51,"githubUrl":"https://github.com/jaegertracing/jaeger/blob/806f4447841ecdb60519f408b004a599d515f437/internal/storage/v2/clickhouse/tracestore/writer.go#L15-L51","documentation":"Writer.WriteTraces opens a ClickHouse batch insert via conn.PrepareBatch with the sql.InsertSpan INSERT statement. If the driver cannot prepare the batch — connection unavailable, auth failure, the spans table missing or its schema changed so the INSERT statement no longer parses — the error is wrapped as \"failed to prepare batch\". No span data is written; the whole WriteTraces call fails.","triggerScenarios":"Calling WriteTraces when: ClickHouse is unreachable or credentials are wrong, the spans table does not exist (migrations not applied), the table schema was altered so sql.InsertSpan's column list mismatches, or the context is canceled before batch preparation completes.","commonSituations":"Fresh ClickHouse deployment without running Jaeger schema migrations; mismatched Jaeger/writer version vs table schema (column added/renamed); ClickHouse restarted under load; wrong DSN database name; auth failure after credential rotation.","solutions":["Verify ClickHouse connectivity and credentials from the Jaeger host with clickhouse-client using the DSN values","Run the Jaeger ClickHouse schema migrations and confirm the spans table exists: SHOW CREATE TABLE spans","Diff sql.InsertSpan's column list against the actual table columns after any upgrade; re-run migrations if they drifted","Unwrap the error to distinguish network (dial tcp) vs server (code 60 UNKNOWN_TABLE) vs auth (code 516) causes and fix accordingly","Ensure the storage writer's connection is initialized and healthy at startup (connection health check)"],"exampleFix":"// before: table missing because migrations were never applied\n// Error: failed to prepare batch: code: 60, message: Table default.spans does not exist\n\n// after: apply schema first (migrations) then write\n// clickhouse-client --host ... --multiquery < migrations/*.sql","handlingStrategy":"validation","validationCode":"// before writes, verify connectivity and table\nif err := conn.Exec(ctx, \"SELECT 1\"); err != nil {\n    return fmt.Errorf(\"clickhouse unavailable: %w\", err)\n}\nvar exists bool\nif err := conn.QueryRow(ctx,\n    \"SELECT exists(SELECT 1 FROM system.tables WHERE database = ? AND name = 'spans')\",\n    dbName).Scan(&exists); err != nil || !exists {\n    return errors.New(\"spans table missing: run jaeger clickhouse migrations\")\n}","typeGuard":null,"tryCatchPattern":"if err := writer.WriteTraces(ctx, td); err != nil {\n    if strings.Contains(err.Error(), \"failed to prepare batch\") {\n        // non-retryable until ops fixes schema/connection\n        log.Error(\"cannot write traces: batch prepare failed\", \"err\", err)\n        return err // do NOT drop spans silently; buffer or nack\n    }\n    return err\n}","preventionTips":["Run ClickHouse schema migrations as a deployment step before Jaeger starts","Validate the DSN (host, port, database, auth, TLS) in CI with a live ping","Pin Jaeger writer version and schema migration version together","Health-check the connection at startup and before accepting trace traffic"],"tags":["clickhouse","go","batch-insert","database","schema-mismatch"],"backgroundTag":"table-does-not-exist","analyzedSha":"806f4447841ecdb60519f408b004a599d515f437","analyzedAt":"2026-09-01T02:39:22.140Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T05:18:18.240Z"}