pinpoint-apm/pinpoint · error

INTERNAL_SERVER_ERROR

INTERNAL_SERVER_ERROR

Error message

Failed to handle apiMetaData={}

What it means

Catch-all failure path of GrpcApiMetaDataHandler.handleApiMetaData. Any exception while building the ApiMetaDataBo or inserting it via apiMetaDataService.insert (an HBase-backed write of the method-to-apiId mapping) is logged at WARN with a debug-rendered apiMetaData and stack trace, then converted into PResults.INTERNAL_SERVER_ERROR returned to the agent. The real cause is hidden from the gRPC response on purpose.

Source

Thrown at collector/src/main/java/com/navercorp/pinpoint/collector/handler/grpc/GrpcApiMetaDataHandler.java:86

                logger.warn("Service not found. serviceName={}, serviceUid={}, applicationName={}, agentId={}",
                        header.getServiceName(), serviceUid, header.getApplicationName(), header.getAgentId());
                return PResults.serviceNotFound(header.getServiceName());
            }

            final String agentId = header.getAgentId();
            final long agentStartTime = header.getAgentStartTime();
            final int line = LineNumber.defaultLineNumber(apiMetaData.getLine());

            final MethodTypeEnum type = MethodTypeEnum.defaultValueOf(apiMetaData.getType());

            final ApiMetaDataBo apiMetaDataBo = new ApiMetaDataBo.Builder(serviceUid, agentId, agentStartTime, apiMetaData.getApiId(), line, type, apiMetaData.getApiInfo())
                    .setLocation(apiMetaData.getLocation())
                    .build();

            this.apiMetaDataService.insert(apiMetaDataBo);
            return PResults.SUCCESS;
        } catch (Exception e) {
            logger.warn("Failed to handle apiMetaData={}", MessageFormatUtils.debugLog(apiMetaData), e);
            // Avoid detailed error messages.
            return PResults.INTERNAL_SERVER_ERROR;
        }
    }
}

View on GitHub (pinned to 744c3d3075)

Solutions

1. Read the WARN log 'Failed to handle apiMetaData=' in the collector for the underlying stack trace.
2. Check HBase availability and that the ApiMetaData table exists with the schema matching your Pinpoint version (run hbase schema scripts).
3. Verify collector HBase client configuration (zookeeper quorum, table namespace).
4. Inspect the payload in the log (debugLog renders the apiMetaData) for null/absurd fields; update or fix the offending agent if its bytecode instrumentation produces bad descriptors.
5. Restart affected agents so they re-send api metadata once storage is healthy.

Example fix

// before
return PResults.INTERNAL_SERVER_ERROR;

// after: keep the exception in the log (it already is) and ensure the table exists
// hbase shell: create 'ApiMetaData', ...
this.apiMetaDataService.insert(apiMetaDataBo); // throws -> logged with full stack trace
return PResults.INTERNAL_SERVER_ERROR;

// pre-check before registering api metadata
if (apiMetaData == null || apiMetaData.getAgentId() == null || apiMetaData.getApplicationName() == null)
    throw new IllegalArgumentException("apiMetaData requires agentId and applicationName");
// verify table exists: hbase shell: exists 'ApiMetaData'

// Java: validate the gRPC payload shape before the call
private static boolean isWellFormedApiMetaData(PS_ApiMetaData m) {
    return m != null && !m.getApiDescriptor().isEmpty() && m.getApplicationName() != null;
}

try {
    PResult result = handler.handleApiMetaData(apiMetaData);
    if (!PResults.isSuccess(result)) {
        backoffRetry(() -> handler.handleApiMetaData(apiMetaData));
    }
} catch (StatusRuntimeException e) {
    logger.warn("apiMetaData RPC failed: {}", e.getStatus(), e);
}

When it happens

Trigger: Any exception in handleApiMetaData: malformed PS_ApiMetaData request fields, exception while composing ApiMetaDataBo, or a storage exception from apiMetaDataService.insert.

Common situations: HBase unavailable or ApiMetaData table missing/mis-scoped (wrong table name in collector config); schema drift after Pinpoint upgrade (e.g. apiUid columns added without table migration); duplicate/racing inserts from many agents hammering the same method signature; corrupt or oversized method descriptors (very long line numbers/locations) from instrumented agents.

Related errors

• INTERNAL_SERVER_ERROR
• INTERNAL_SERVER_ERROR
• INTERNAL_SERVER_ERROR
• INTERNAL_SERVER_ERROR
• invalid metadata serviceUid: ${serviceUid}

AI-assisted analysis of pinpoint-apm/pinpoint@744c3d3075 (2026-09-07). Data as JSON: /api/errors/2396cebc85845cfc. Report an issue: GitHub.