{"record":{"id":"3f423c9de5df4659","repo":"pinpoint-apm/pinpoint","slug":"failed-to-insert-spanbo","errorCode":null,"errorMessage":"Failed to insert spanBo","messagePattern":"Failed to insert spanBo","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"otlptrace/otlptrace-collector/src/main/java/com/navercorp/pinpoint/otlp/trace/collector/service/OtlpTraceExportService.java","lineNumber":150,"sourceCode":"\n        final OtlpTraceMapperData otlpTraceMapperData = otlpTraceMapper.map(resourceSpanList);\n        // Mapping rejects (invalid ids, unlinkable/orphan spans) are client-side data faults.\n        final OtlpTraceCollectorRejectedSpan clientRejected = otlpTraceMapperData.getRejectedSpan();\n        for (Map.Entry<OtlpTraceRejectReason, Long> rejected : clientRejected.countByReason().entrySet()) {\n            ingestMetrics.spanRejected(transport, rejected.getKey(), rejected.getValue());\n        }\n        ingestMetrics.spanStored(transport, otlpTraceMapperData.getSpanBoList().size());\n        ingestMetrics.spanChunkStored(transport, otlpTraceMapperData.getSpanChunkBoList().size());\n\n        int insertErrorCount = 0;\n        for (SpanBo spanBo : otlpTraceMapperData.getSpanBoList()) {\n            for (TraceService traceService : traceServiceList) {\n                try {\n                    traceService.insertSpan(spanBo);\n                } catch (Exception e) {\n                    insertErrorCount++;\n                    spanInsertErrorCounter.increment();\n                    throttledLogger.warn(\"Failed to insert spanBo\", e);\n                }\n            }\n        }\n\n        for (SpanChunkBo spanChunkBo : otlpTraceMapperData.getSpanChunkBoList()) {\n            for (TraceService traceService : traceServiceList) {\n                try {\n                    traceService.insertSpanChunk(spanChunkBo);\n                } catch (Exception e) {\n                    insertErrorCount++;\n                    spanChunkInsertErrorCounter.increment();\n                    throttledLogger.warn(\"Failed to insert spanChunkBo\", e);\n                }\n            }\n        }\n\n        int agentInfoErrorCount = 0;\n        for (AgentInfoBo agentInfoBo : otlpTraceMapperData.getAgentInfoBoList()) {","sourceCodeStart":132,"sourceCodeEnd":168,"githubUrl":"https://github.com/pinpoint-apm/pinpoint/blob/744c3d3075e595656abb1ae331ad2c0e4c9eb996/otlptrace/otlptrace-collector/src/main/java/com/navercorp/pinpoint/otlp/trace/collector/service/OtlpTraceExportService.java#L132-L168","documentation":"OtlpTraceExportService.export loops over every configured TraceService calling insertSpan(spanBo); a synchronous exception from any of them (the OTLP-mapped span could not be handed to one sink) increments the insert error count, bumps spanInsertErrorCounter, and logs this throttled warning. The export continues with the remaining sinks but the failure is included in the returned serverErrorCount.","triggerScenarios":"traceService.insertSpan(spanBo) throws synchronously for any registered TraceService while exporting mapped OTLP spans — e.g. HBase client exception, rejected executor submission, or mapping/data validation failure.","commonSituations":"A secondary trace sink (v1/v2 HBase writer) misconfigured or down; executor queue full under load spikes; malformed span after OTLP mapping.","solutions":["Check the throttled-logged stack trace to see which TraceService threw.","Verify each configured trace sink's HBase tables and connectivity.","Inspect the export result's serverErrorCount/serverMessage returned to the ingest path for aggregate failure info.","Fix or disable the failing TraceService in the collector configuration."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"if (otlpTraceMapperData.getSpanBoList() == null || otlpTraceMapperData.getSpanBoList().isEmpty()) {\n    return; // nothing to export, skip loop entirely\n}","typeGuard":null,"tryCatchPattern":"for (TraceService traceService : traceServiceList) {\n    try {\n        traceService.insertSpan(spanBo);\n    } catch (Exception e) {\n        spanInsertErrorCounter.increment();\n        logger.warn(\"Failed to insert spanBo into {}\", traceService.getClass().getSimpleName(), e);\n    }\n}","preventionTips":["Check the export result's serverErrorCount before confirming successful ingestion.","Verify every configured TraceService sink is reachable at collector startup.","Keep per-sink executor queues sized for peak span rates to avoid rejected submissions.","Alert on spanInsertErrorCounter rather than relying on throttled logs alone."],"tags":["hbase","span","export-failure"],"backgroundTag":"database-write-failed","analyzedSha":"744c3d3075e595656abb1ae331ad2c0e4c9eb996","analyzedAt":"2026-09-07T18:48:45.289Z","contentChangedAt":"2026-09-07T18:48:45.289Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}