JuliusBrussee/caveman · error

telemetry attributes exceed

Error message

telemetry attributes exceed %d keys

What it means

SanitizeAttributes in shared/platform/telemetry/span.go enforces a hard cap of MaxAttributes keys per span. If, after filtering to allowed attribute keys, more than MaxAttributes remain, it fails closed and returns an error instead of silently truncating — the span parse fails rather than emitting an oversized attribute map.

Solutions

1. Reduce the number of attributes per span before sending (drop low-value keys, aggregate tags).
2. Move high-cardinality data out of span attributes into logs or dedicated events.
3. Batch data across multiple spans instead of one span carrying everything.
4. Check for producer bugs that generate unbounded key sets.

Example fix

// before
span.SetAttributes(map[string]string{ "user.tag.0": ..., "user.tag.1": ..., /* hundreds */ })

// after
span.SetAttributes(map[string]string{"user.tag.count": "212", "user.tag.sample": "gold"})

if len(attrs) > maxAttributes /* mirror MaxAttributes constant */ {
    // drop lowest-priority keys or split into another span before Parse
    attrs = pruneToTopN(attrs, maxAttributes)
}

sanitized, err := telemetry.SanitizeAttributes(raw)
if err != nil {
    if strings.Contains(err.Error(), "exceed") && strings.Contains(err.Error(), "keys") {
        raw = reduceAttributes(raw)
        sanitized, err = telemetry.SanitizeAttributes(raw)
    }
    if err != nil { return err }
}

When it happens

Trigger: Calling Parse on span data whose attribute map (after AttributeAllowed filtering) contains more than MaxAttributes keys. Also exercised deliberately by TestSanitizeAttributesFailsClosedOnBounds.

Common situations: Instrumentation attaching a key per tag/label on high-cardinality data; merging attributes from multiple sources into one span; a producer bug duplicating keys with different casing.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• telemetry attribute key exceeds
• telemetry attribute exceeds bytes
• invalid decompression limit
• kms: response exceeds limit
• aes cipher

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/dfca89f0144907ad. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/telemetry/span.go:257

		}
	}
	if strings.Contains(lower, "api_key") || strings.Contains(lower, "access_token") || strings.Contains(lower, "refresh_token") {
		return false
	}
	return true
}

// SanitizeAttributes returns a deterministic copy. Sensitive keys are removed;
// malformed or oversized metadata fails closed so callers can reject the record.
func SanitizeAttributes(attrs map[string]string) (map[string]string, error) {
	keys := make([]string, 0, len(attrs))
	for key := range attrs {
		if AttributeAllowed(key) {
			keys = append(keys, key)
		}
	}
	if len(keys) > MaxAttributes {
		return nil, fmt.Errorf("telemetry attributes exceed %d keys", MaxAttributes)
	}
	sort.Strings(keys)
	out := make(map[string]string, len(keys))
	total := 0
	for _, key := range keys {
		value := attrs[key]
		if len(key) > MaxAttributeKeyBytes {
			return nil, fmt.Errorf("telemetry attribute key exceeds %d bytes", MaxAttributeKeyBytes)
		}
		if len(value) > MaxAttributeValueBytes {
			return nil, fmt.Errorf("telemetry attribute %q exceeds %d bytes", key, MaxAttributeValueBytes)
		}
		total += len(key) + len(value)
		if total > MaxAttributesBytes {
			return nil, fmt.Errorf("telemetry attributes exceed %d bytes", MaxAttributesBytes)
		}
		out[key] = value
	}

View on GitHub (pinned to 3ee70a1026)