JuliusBrussee/caveman · error

sanitize row %d: %w

Error message

sanitize row %d: %w

What it means

Every row's Attributes must pass telemetry.SanitizeAttributes before import completes; any error it returns is wrapped with the row index. Sanitization enforces attribute-level constraints (size, count, key/value validity) defined in shared/platform/telemetry, so the wrapped error text tells you the specific attribute violation on that row. Like 1188, one bad row fails the whole import.

Source

Thrown at shared/platform/importers/importers.go:103

		rows, err = parseLangfuse(data, opts)
	case FormatHelicone:
		rows, err = parseHelicone(data, opts)
	case FormatGeneric:
		rows, err = parseGeneric(data, opts)
	default:
		// fail-closed: unknown formats are an error, never a silent pass-through.
		return nil, Summary{}, fmt.Errorf("unknown import format %q: supported formats are otlp, caveman-jsonl, langfuse, helicone, generic", format)
	}
	if err != nil {
		return nil, Summary{}, err
	}
	for i := range rows {
		if len(rows[i].EventsJSON) > telemetry.MaxEventsBytes {
			return nil, Summary{}, fmt.Errorf("sanitize row %d: events exceed %d bytes", i, telemetry.MaxEventsBytes)
		}
		sanitized, sanitizeErr := telemetry.SanitizeAttributes(rows[i].Attributes)
		if sanitizeErr != nil {
			return nil, Summary{}, fmt.Errorf("sanitize row %d: %w", i, sanitizeErr)
		}
		rows[i].Attributes = sanitized
	}
	return rows, Summary{Format: format, RowCount: len(rows)}, nil
}

// ---------------------------------------------------------------------------
// Shared helpers
// ---------------------------------------------------------------------------

// chTimeLayout is the ClickHouse DateTime64 textual layout the gateway writer
// uses; importers must produce the same shape ("2006-01-02 15:04:05.000").
const chTimeLayout = "2006-01-02 15:04:05.000"

// applyScope stamps the authenticated org/project onto a span and fills the
// honest defaults that the spans schema expects for non-null operation. It is
// called for EVERY produced span so the uploaded file can never set org/project.
func applyScope(sp *Span, opts Options) {

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Read the wrapped error to identify the exact attribute constraint violated on the indexed row
2. Move oversized values from attributes to a proper storage side-channel, or truncate them pre-import
3. Sanitize/normalize attribute keys and values in your own pipeline before calling Import
4. Re-run import after fixing; the row index in the message points at the offending record

Example fix

// before
attrs := map[string]string{"prompt": giantPromptText}
sp.Attributes = attrs

// after
const maxAttr = 8 << 10
if len(giantPromptText) > maxAttr {
    giantPromptText = giantPromptText[:maxAttr]
}
attrs := map[string]string{"prompt": giantPromptText}
sp.Attributes = attrs

if _, err := telemetry.SanitizeAttributes(sp.Attributes); err != nil { /* fix attributes before import */ }

rows, sum, err := importers.Import(format, data, opts)
if err != nil {
	if i := strings.Index(err.Error(), "sanitize row"); i >= 0 {
		log.Printf("attribute violation on row %d: %v", rowIndexFrom(err), err)
	}
	return err
}

When it happens

Trigger: An attribute value exceeding the per-attribute byte limit (e.g. a full prompt stored as an attribute); invalid attribute keys (empty or control characters); attribute maps exceeding the sanctioned count; non-UTF8 bytes in values.

Common situations: GenAI traces that smuggle request bodies into attributes instead of events; third-party OTLP exporters adding huge resource attributes; binary or binary-encoded values placed in string attributes.

Related errors

• sanitize row %d: events exceed %d bytes
• caveman-jsonl line %d: %w
• generic record %d: %w
• mapped source path %q for target %q was not found in any rec
• generic import requires a non-empty field map

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/48d3261c8b198567. Report an issue: GitHub.