{"record":{"id":"e8f7b55f303e974e","repo":"pinpoint-apm/pinpoint","slug":"failed-to-insert-spanchunkbo","errorCode":null,"errorMessage":"Failed to insert spanChunkBo","messagePattern":"Failed to insert spanChunkBo","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"otlptrace/otlptrace-collector/src/main/java/com/navercorp/pinpoint/otlp/trace/collector/service/OtlpTraceExportService.java","lineNumber":162,"sourceCode":"            for (TraceService traceService : traceServiceList) {\n                try {\n                    traceService.insertSpan(spanBo);\n                } catch (Exception e) {\n                    insertErrorCount++;\n                    spanInsertErrorCounter.increment();\n                    throttledLogger.warn(\"Failed to insert spanBo\", e);\n                }\n            }\n        }\n\n        for (SpanChunkBo spanChunkBo : otlpTraceMapperData.getSpanChunkBoList()) {\n            for (TraceService traceService : traceServiceList) {\n                try {\n                    traceService.insertSpanChunk(spanChunkBo);\n                } catch (Exception e) {\n                    insertErrorCount++;\n                    spanChunkInsertErrorCounter.increment();\n                    throttledLogger.warn(\"Failed to insert spanChunkBo\", e);\n                }\n            }\n        }\n\n        int agentInfoErrorCount = 0;\n        for (AgentInfoBo agentInfoBo : otlpTraceMapperData.getAgentInfoBoList()) {\n            if (agentIdCache.getIfPresent(agentInfoBo.getAgentId()) == null) {\n                try {\n                    agentInfoService.insert(agentInfoBo);\n                    applicationIndexV2Service.insert(DEFAULT_SERVICE_UID, agentInfoBo);\n                    agentIdCache.put(agentInfoBo.getAgentId(), Boolean.TRUE);\n                } catch (Exception e) {\n                    agentInfoErrorCount++;\n                    agentInfoInsertErrorCounter.increment();\n                    throttledLogger.warn(\"Failed to insert agentInfoBo\", e);\n                }\n            }\n        }","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/pinpoint-apm/pinpoint/blob/744c3d3075e595656abb1ae331ad2c0e4c9eb996/otlptrace/otlptrace-collector/src/main/java/com/navercorp/pinpoint/otlp/trace/collector/service/OtlpTraceExportService.java#L144-L180","documentation":"OtlpTraceExportService.export iterates all TraceService instances calling insertSpanChunk(spanChunkBo); a synchronous exception while persisting a span chunk (partial spans/spans without a root) increments insertErrorCount and spanChunkInsertErrorCounter and logs this throttled warning. Export proceeds, but the chunk is lost for that sink and counted in serverErrorCount.","triggerScenarios":"traceService.insertSpanChunk(spanChunkBo) throws for any sink — HBase write error, rejected execution, or invalid chunk data — during export of otlpTraceMapperData.getSpanChunkBoList().","commonSituations":"SpanChunk HBase table missing or full; collector under load with saturated write executors; failing secondary trace sink after a version upgrade.","solutions":["Read the throttled stack trace to identify the failing TraceService.","Verify the span chunk HBase table schema and write permissions.","Tune write executor capacity / HBase client buffers for chunk throughput.","Fix or remove the broken trace sink from the collector configuration."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"if (spanChunkBo == null || spanChunkBo.getSpanChunkBoList() == null) {\n    logger.warn(\"Skipping invalid spanChunkBo during export\");\n    continue;\n}","typeGuard":null,"tryCatchPattern":"try {\n    traceService.insertSpanChunk(spanChunkBo);\n} catch (Exception e) {\n    spanChunkInsertErrorCounter.increment();\n    logger.warn(\"Failed to insert spanChunkBo via {}\", traceService.getClass().getSimpleName(), e);\n}","preventionTips":["Validate span chunk HBase table schema after pinpoint version upgrades.","Size write executors to absorb chunk bursts (chunks spike with async/incomplete spans).","Monitor spanChunkInsertErrorCounter for per-sink degradation.","Keep agent SDK chunk emission sane to avoid oversized chunk batches."],"tags":["hbase","span-chunk","export-failure"],"backgroundTag":"database-write-failed","analyzedSha":"744c3d3075e595656abb1ae331ad2c0e4c9eb996","analyzedAt":"2026-09-07T18:48:45.289Z","contentChangedAt":"2026-09-07T18:48:45.289Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}