sgl-project/sglang · error · RuntimeError

initialize opentelemetry error:{e}. Please set correct otlp

Error message

initialize opentelemetry error:{e}. Please set correct otlp endpoint.

What it means

process_tracing_init builds an OTLP exporter/span processor and registers a tracer provider; any exception during that setup (bad OTLP endpoint URL, unreachable collector, invalid sampling/batch options) is wrapped into this RuntimeError with the underlying message.

Source

Thrown at python/sglang/srt/observability/trace.py:252

            }
        )
        tracer_provider = TracerProvider(
            resource=resource, id_generator=TraceCustomIdGenerator()
        )

        schedule_delay_millis = envs.SGLANG_OTLP_EXPORTER_SCHEDULE_DELAY_MILLIS.get()
        max_export_batch_size = envs.SGLANG_OTLP_EXPORTER_MAX_EXPORT_BATCH_SIZE.get()

        processor = BatchSpanProcessor(
            span_exporter=get_otlp_span_exporter(otlp_endpoint),
            schedule_delay_millis=schedule_delay_millis,
            max_export_batch_size=max_export_batch_size,
        )
        tracer_provider.add_span_processor(processor)
        trace.set_tracer_provider(tracer_provider)
    except Exception as e:
        opentelemetry_initialized = False
        raise RuntimeError(
            f"initialize opentelemetry error:{e}. Please set correct otlp endpoint."
        )

    opentelemetry_initialized = True
    tracer = trace.get_tracer("sglang server")

    # Auto-start async trace exporter when SGLANG_TRACE_ASYNC=1
    if envs.SGLANG_TRACE_ASYNC.get():
        from sglang.srt.observability.trace_async import start_trace_exporter

        start_trace_exporter(otlp_endpoint, server_name, trace_modules=trace_modules)


def get_global_tracing_enabled():
    return opentelemetry_initialized


def get_otlp_span_exporter(endpoint):

View on GitHub (pinned to 0132848349)

Solutions

Fix the OTLP endpoint: use a full valid URL like http://collector-host:4317 and confirm the collector is reachable (curl the health port)
Verify scheme/port match your exporter protocol (grpc 4317 vs http 4318)
If TLS is involved, install/point to the right CA cert; retest after collector startup

Example fix

# before
--otlp-endpoint localhost:4317  # malformed, init fails
# after
--otlp-endpoint http://collector:4317

Defensive patterns

Strategy: validation

Validate before calling

from urllib.parse import urlparse
u = urlparse(otlp_endpoint)
assert u.scheme in ('http', 'https', 'grpc') and u.hostname and u.port, 'invalid OTLP endpoint'
import socket
socket.create_connection((u.hostname, u.port), timeout=2).close()  # reachability precheck

Try / catch

try:
    process_tracing_init(...)
except RuntimeError as e:
    log_fatal(f'tracing init failed: {e}'); start_without_tracing_or_exit()

Prevention

Start the collector before sglang; use full http(s)://host:port endpoints
Validate endpoint URL format and port/protocol match (4317 grpc vs 4318 http) in deploy scripts

When it happens

Trigger: Setting an malformed or unreachable OTLP endpoint (e.g. --otlp-endpoint http:localhost:4317 missing slashes, or collector not running / TLS mismatch), or invalid batch size arguments.

Common situations: Collector not started before sglang; endpoint scheme mismatch (http vs https/grpc); typos in the endpoint flag; firewall/DNS issues in k8s.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c583e8920e260ed2. Report an issue: GitHub.