What causes "sanitize row %d: events exceed %d bytes" in JuliusBrussee/caveman?

Reduce event verbosity at the source: sample events or truncate large payloads before capture

sanitize row %d: events exceed %d bytes

Error message

sanitize row %d: events exceed %d bytes

What it means

After format-specific parsing, Import enforces a global cap: each row's EventsJSON may not exceed telemetry.MaxEventsBytes (64 KiB). EventsJSON is the serialized event payload attached to a span; a single oversized event set aborts the entire import with the offending row index. This is a resource guard before anything reaches storage.

Source

Thrown at shared/platform/importers/importers.go:99

		rows, err = parseOTLP(data, opts)
	case FormatCavemanJSONL:
		rows, err = parseCavemanJSONL(data, opts)
	case FormatLangfuse:
		rows, err = parseLangfuse(data, opts)
	case FormatHelicone:
		rows, err = parseHelicone(data, opts)
	case FormatGeneric:
		rows, err = parseGeneric(data, opts)
	default:
		// fail-closed: unknown formats are an error, never a silent pass-through.
		return nil, Summary{}, fmt.Errorf("unknown import format %q: supported formats are otlp, caveman-jsonl, langfuse, helicone, generic", format)
	}
	if err != nil {
		return nil, Summary{}, err
	}
	for i := range rows {
		if len(rows[i].EventsJSON) > telemetry.MaxEventsBytes {
			return nil, Summary{}, fmt.Errorf("sanitize row %d: events exceed %d bytes", i, telemetry.MaxEventsBytes)
		}
		sanitized, sanitizeErr := telemetry.SanitizeAttributes(rows[i].Attributes)
		if sanitizeErr != nil {
			return nil, Summary{}, fmt.Errorf("sanitize row %d: %w", i, sanitizeErr)
		}
		rows[i].Attributes = sanitized
	}
	return rows, Summary{Format: format, RowCount: len(rows)}, nil
}

// ---------------------------------------------------------------------------
// Shared helpers
// ---------------------------------------------------------------------------

// chTimeLayout is the ClickHouse DateTime64 textual layout the gateway writer
// uses; importers must produce the same shape ("2006-01-02 15:04:05.000").
const chTimeLayout = "2006-01-02 15:04:05.000"

View on GitHub (pinned to 27d5a3981a)

Solutions

Reduce event verbosity at the source: sample events or truncate large payloads before capture
Split oversized spans or drop non-essential events in pre-processing so EventsJSON stays under 65536 bytes
Check row index in the error to find which span blew the limit, then re-export that trace with fewer events
If large payloads are required, store them out-of-band and reference them from attributes instead

Example fix

// before
event := map[string]any{"type": "completion", "body": fullResponseBody /* 200KB */}

// after
const maxBody = 32 << 10
if len(fullResponseBody) > maxBody {
    fullResponseBody = fullResponseBody[:maxBody]
}
event := map[string]any{"type": "completion", "body": fullResponseBody}

Defensive patterns

Strategy: validation

Validate before calling

const maxEvents = telemetry.MaxEventsBytes
func eventsWithinLimit(eventsJSON []byte) bool { return len(eventsJSON) <= maxEvents }

Try / catch

if err != nil && strings.Contains(err.Error(), "events exceed") { /* find row index, drop or split its events, re-import */ }

Prevention

Cap event payload size at capture time (truncate bodies to a fixed budget)
Sample high-frequency events before they attach to spans
Alert when any span's serialized events approach 64 KiB

When it happens

Trigger: A span carrying hundreds of large events (e.g. full request/response bodies logged as events); an OTLP or langfuse record with megabyte-scale attribute arrays that the mapper folds into events; machine-generated spans that embed stack traces or base64 payloads per event.

Common situations: Verbose LLM tracing that logs full prompts and completions as events; importing debug-level traces captured with event dumping on; a single trace with thousands of tiny events accumulating past 64 KiB.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/43ca00ea02f1b9a4. Report an issue: GitHub.