What causes "failed to decode trace ID: %w" in jaegertracing/jaeger?

Inspect the raw trace_id value in ClickHouse for the failing row

failed to decode trace ID: %w

Error message

failed to decode trace ID: %w

What it means

convertSpan wraps any failure from decodeTraceID (invalid hex characters or wrong decoded length) with this contextual message. It indicates the stored trace ID for the row could not be converted into a pcommon.TraceID, so the span cannot be returned.

Source

Thrown at internal/storage/v2/clickhouse/tracestore/dbmodel/from.go:104

func decodeSpanID(s string) (pcommon.SpanID, error) {
	var id pcommon.SpanID
	b, err := hex.DecodeString(s)
	if err != nil {
		return id, err
	}
	if len(b) != len(id) {
		return id, fmt.Errorf("invalid length %d of decoded span ID %q, expected %d bytes", len(b), s, len(id))
	}
	copy(id[:], b)
	return id, nil
}

func convertSpan(sr *SpanRow) (ptrace.Span, error) {
	span := ptrace.NewSpan()
	span.SetStartTimestamp(pcommon.NewTimestampFromTime(sr.StartTime))
	traceId, err := decodeTraceID(sr.TraceID)
	if err != nil {
		return span, fmt.Errorf("failed to decode trace ID: %w", err)
	}
	span.SetTraceID(traceId)
	spanId, err := decodeSpanID(sr.ID)
	if err != nil {
		return span, fmt.Errorf("failed to decode span ID: %w", err)
	}
	span.SetSpanID(spanId)
	if sr.ParentSpanID != "" {
		parentSpanId, err := decodeSpanID(sr.ParentSpanID)
		if err != nil {
			return span, fmt.Errorf("failed to decode parent span ID: %w", err)
		}
		span.SetParentSpanID(parentSpanId)
	}
	span.TraceState().FromRaw(sr.TraceState)
	span.SetName(sr.Name)
	span.SetKind(jptrace.StringToSpanKind(sr.Kind))
	span.SetEndTimestamp(pcommon.NewTimestampFromTime(sr.StartTime.Add(time.Duration(sr.Duration))))

View on GitHub (pinned to 806f444784)

Solutions

Inspect the raw trace_id value in ClickHouse for the failing row
Fix invalid hex at the data level or in the ingestion pipeline
Ensure the writer uses the same hex encoding (lowercase, full 32 chars) as the reader expects
Add ingest-time validation of trace ID format

Example fix

// before
traceId, err := decodeTraceID(sr.TraceID)
// after: guard caller-side
if len(sr.TraceID) != 32 { return span, fmt.Errorf("trace ID %q is not 32 hex chars", sr.TraceID) }
traceId, err := decodeTraceID(sr.TraceID)

Defensive patterns

Strategy: try-catch

Validate before calling

if len(sr.TraceID) != 32 {
    return fmt.Errorf("trace ID %q must be 32 hex chars before conversion", sr.TraceID)
}

Type guard

func hasValidTraceID(sr *SpanRow) bool {
    _, ok := isTraceID(sr.TraceID)
    return ok
}

Try / catch

span, err := FromRow(sr)
if err != nil {
    if strings.Contains(err.Error(), "failed to decode trace ID") {
        metrics.MalformedTraceID.Inc(1)
        continue // skip poisoned row
    }
    return err
}

Prevention

Validate trace IDs at write time
Use a single shared encoder for trace IDs across services
Monitor decode-error metrics to catch producer regressions early

When it happens

Trigger: FromRow is called on a SpanRow whose TraceID string is not valid hex (odd length, non-hex characters) or decodes to a length other than 16 bytes.

Common situations: Reading rows written by external producers; corrupted ClickHouse data; schema drift where trace_id changed representation between storage versions.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of jaegertracing/jaeger@806f444784 (2026-09-01). Data as JSON: /api/errors/ef601e3b7ede9a17. Report an issue: GitHub.