vxcontrol/pentagi · error

failed to force flush logger: %w

Error message

failed to force flush logger: %w

What it means

This error is produced by telemetryClient.ForceFlush in backend/pkg/observability/otelclient.go when the underlying OpenTelemetry log provider fails to flush its buffered log records within the given context. It wraps the original SDK error so the caller knows which of the three pipelines (logger, meter, tracer) failed. The function collects errors from all three components and joins them with errors.Join.

Source

Thrown at backend/pkg/observability/otelclient.go:87

	}
	if err := c.meter.Shutdown(ctx); err != nil {
		errs = append(errs, fmt.Errorf("failed to shutdown meter: %w", err))
	}
	if err := c.tracer.Shutdown(ctx); err != nil {
		errs = append(errs, fmt.Errorf("failed to shutdown tracer: %w", err))
	}
	// Always close the connection, even if a provider shutdown failed above, so a
	// stalled flush can't leak the grpc conn.
	if err := c.conn.Close(); err != nil {
		errs = append(errs, fmt.Errorf("failed to close telemetry connection: %w", err))
	}
	return errors.Join(errs...)
}

func (c *telemetryClient) ForceFlush(ctx context.Context) error {
	var errs []error
	if err := c.logger.ForceFlush(ctx); err != nil {
		errs = append(errs, fmt.Errorf("failed to force flush logger: %w", err))
	}
	if err := c.meter.ForceFlush(ctx); err != nil {
		errs = append(errs, fmt.Errorf("failed to force flush meter: %w", err))
	}
	if err := c.tracer.ForceFlush(ctx); err != nil {
		errs = append(errs, fmt.Errorf("failed to force flush tracer: %w", err))
	}
	return errors.Join(errs...)
}

func NewTelemetryClient(ctx context.Context, cfg *config.Config) (TelemetryClient, error) {
	if cfg.TelemetryEndpoint == "" {
		return nil, fmt.Errorf("telemetry endpoint is not set: %w", ErrNotConfigured)
	}

	// grpc.NewClient is non-blocking: it never dials during startup, so a
	// set-but-unreachable collector can't stall main(), and the connection is
	// established (and re-established) lazily in the background — a collector that

View on GitHub (pinned to ea665308ba)

Solutions

1. Check the wrapped cause (%w) to see whether it is a context deadline or an export failure and address that first
2. Verify the OTLP collector at cfg.TelemetryEndpoint is reachable (docker compose observability stack up, correct port 4317)
3. Increase the flush timeout passed via ctx or retry the flush with a fresh context
4. If seen at shutdown, ensure the collector is healthy before stopping the app, or accept and log the dropped logs

Example fix

// before
if err := client.ForceFlush(context.Background()); err != nil {
    return err
}
// after
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := client.ForceFlush(ctx); err != nil {
    log.Warnf("telemetry flush incomplete: %v", err) // don't fail shutdown on telemetry
}

if client == nil { /* telemetry disabled, skip flush */ }

if err := client.ForceFlush(ctx); err != nil {
    var joined interface{ Unwrap() []error }
    if errors.As(err, &joined) { /* inspect each leg */ }
    log.Warnf("flush failed (non-fatal): %v", err)
}

When it happens

Trigger: Calling ForceFlush(ctx) when the sdklog LoggerProvider's batch processor cannot export pending log records: ctx deadline exceeded before the batch exporter finishes, the OTLP gRPC exporter returning an error, or the collector rejecting logs.

Common situations: Shutdown-time flushes during application exit with an unreachable or slow OpenTelemetry Collector; short-lived contexts cancelled before the batch processor can drain; network partitions between the app and the collector endpoint.

Related errors

• failed to force flush meter: %w
• failed to force flush tracer: %w
• failed to create telemetry connection: %w
• failed to create log exporter: %w
• failed to create metric exporter: %w

AI-assisted analysis of vxcontrol/pentagi@ea665308ba (2026-09-01). Data as JSON: /api/errors/b83de67e67bad06f. Report an issue: GitHub.