pinpoint-apm/pinpoint · error

INTERNAL_SERVER_ERROR

INTERNAL_SERVER_ERROR

Error message

Failed to handle applicationName={}, sqlMetaData={}

What it means

Catch-all failure path of GrpcSqlMetaDataHandler.handleSqlMetaData. The handler iterates all configured SqlMetaDataService backends, inserts the SqlMetaDataBo into each, and if any insert throws, it logs 'Failed to handle applicationName={}, sqlMetaData={}' (details suppressed, stack trace appended) and marks the overall result false, which yields PResults.INTERNAL_SERVER_ERROR to the agent. An outer catch does the same for unexpected exceptions.

Source

Thrown at collector/src/main/java/com/navercorp/pinpoint/collector/handler/grpc/GrpcSqlMetaDataHandler.java:79

        }

        try {
            ServiceUid serviceUid = header.getServiceUid().get();
            if (serviceUid == null || ServiceUid.UNKNOWN.equals(serviceUid) || ServiceUid.ERROR.equals(serviceUid)) {
                logger.warn("Service not found. serviceName={}, serviceUid={}, applicationName={}, agentId={}",
                        header.getServiceName(), serviceUid, header.getApplicationName(), header.getAgentId());
                return PResults.serviceNotFound(header.getServiceName());
            }

            final SqlMetaDataBo sqlMetaDataBo = mapSqlMetaDataBo(header, sqlMetaData, serviceUid);

            boolean result = true;
            for (SqlMetaDataService sqlMetaDataService : sqlMetaDataServices) {
                try {
                    sqlMetaDataService.insert(sqlMetaDataBo);
                } catch (Exception e) {
                    // Avoid detailed error messages.
                    logger.warn("Failed to handle applicationName={}, sqlMetaData={}", header.getApplicationName(), MessageFormatUtils.debugLog(sqlMetaData), e);
                    result = false;
                }
            }

            return newResult(result);
        } catch (Exception e) {
            logger.warn("Failed to handle applicationName={}, sqlMetaData={}", header.getApplicationName(), MessageFormatUtils.debugLog(sqlMetaData), e);
            return PResults.INTERNAL_SERVER_ERROR;
        }
    }

    private static SqlMetaDataBo mapSqlMetaDataBo(ServerHeader agentInfo, PSqlMetaData sqlMetaData, ServiceUid serviceUid) {
        final String agentId = agentInfo.getAgentId();
        final long agentStartTime = agentInfo.getAgentStartTime();
        final int sqlId = sqlMetaData.getSqlId();
        final String sql = sqlMetaData.getSql();

        return new SqlMetaDataBo(serviceUid, agentId, agentStartTime, sqlId, sql);

View on GitHub (pinned to 744c3d3075)

Solutions

1. Read the collector WARN log 'Failed to handle applicationName={}, sqlMetaData=' for the stack trace identifying which backend failed.
2. Check HBase health and that the SqlMetaData table exists with the correct schema for your Pinpoint version.
3. If multiple sqlMetaDataServices are configured, verify each backend is reachable (the per-service try/catch logs one line per failing backend).
4. Validate the logged sqlMetaData payload (sql, sqlId, hashCode) for null/invalid values; fix or upgrade the agent/plugin generating it.
5. Restart agents after storage recovery so sql metadata is re-sent.

Example fix

// before
sqlMetaDataService.insert(sqlMetaDataBo); // throws -> result=false -> INTERNAL_SERVER_ERROR

// after: ensure backend is healthy before deploying
// hbase shell: exists 'SqlMetaData'; and inspect per-service WARN lines to find the failing service
try {
    sqlMetaDataService.insert(sqlMetaDataBo);
} catch (Exception e) {
    logger.warn("Failed to handle applicationName={}, sqlMetaData={}", header.getApplicationName(), MessageFormatUtils.debugLog(sqlMetaData), e);
    result = false;
}

// verify backends before bulk sql metadata registration
// hbase shell: exists 'SqlMetaData'
if (header == null || header.getApplicationName() == null)
    throw new IllegalArgumentException("applicationName is required for sql metadata");

// Java: check the request header before the call
private static boolean hasValidHeader(PS_MetaDataHeader header) {
    return header != null && header.getApplicationName() != null && !header.getApplicationName().isEmpty();
}

try {
    PResult result = handler.handleSqlMetaData(header, sqlMetaData);
    if (!PResults.isSuccess(result)) {
        backoffRetry(() -> handler.handleSqlMetaData(header, sqlMetaData));
    }
} catch (StatusRuntimeException e) {
    logger.warn("sqlMetaData RPC failed: {}", e.getStatus(), e);
}

When it happens

Trigger: A PS_MetaData request whose sqlMetaDataBo insert throws in one or more SqlMetaDataService instances: HBase write failure, malformed sql statement/params, or null applicationName in the header; also any exception in the outer request-parsing/mapping block.

Common situations: HBase outage or SqlMetaData table missing/misconfigured; multi-backend setups (e.g. legacy + uid metadata services) where one backend is down but others work; Pinpoint version upgrade with unmigrated SqlMetaData schema; corrupt SQL metadata produced by a misconfigured jdbc plugin.

Related errors

• INTERNAL_SERVER_ERROR
• INTERNAL_SERVER_ERROR
• INTERNAL_SERVER_ERROR
• INTERNAL_SERVER_ERROR
• RemoteAddress is null

AI-assisted analysis of pinpoint-apm/pinpoint@744c3d3075 (2026-09-07). Data as JSON: /api/errors/28795962f24b7a6e. Report an issue: GitHub.