{"record":{"id":"649f853652ee2978","repo":"apache/druid","slug":"fail-to-get-protobuf-schema-because-of-can-not-con","errorCode":null,"errorMessage":"Fail to get protobuf schema because of can not connect to registry or failed http request!","messagePattern":"Fail to get protobuf schema because of can not connect to registry or failed http request!","errorType":"exception","errorClass":"ParseException","httpStatus":null,"severity":"error","filePath":"extensions-core/protobuf-extensions/src/main/java/org/apache/druid/data/input/protobuf/SchemaRegistryBasedProtobufBytesDecoder.java","lineNumber":187,"sourceCode":"   * @param bytes ByteBuffer containing the Confluent Schema Registry formatted protobuf message\n   * @return DynamicMessage parsed from the protobuf bytes\n   * @throws ParseException if the schema cannot be retrieved or the message cannot be parsed\n   */\n  @Override\n  public DynamicMessage parse(ByteBuffer bytes)\n  {\n    bytes.get(); // ignore first \\0 byte\n    int id = bytes.getInt(); // extract schema registry id\n    bytes.get(); // ignore \\0 byte before PB message\n    int length = bytes.limit() - 2 - 4;\n    Descriptors.Descriptor descriptor;\n    try {\n      ProtobufSchema schema = (ProtobufSchema) registry.getSchemaById(id);\n      descriptor = schema.toDescriptor();\n    }\n    catch (RestClientException e) {\n      LOGGER.error(e.getMessage());\n      throw new ParseException(\n          null,\n          e,\n          \"Fail to get protobuf schema because of can not connect to registry or failed http request!\"\n      );\n    }\n    catch (IOException e) {\n      LOGGER.error(e.getMessage());\n      throw new ParseException(null, e, \"Fail to get protobuf schema because of invalid schema!\");\n    }\n    try {\n      byte[] rawMessage = new byte[length];\n      bytes.get(rawMessage, 0, length);\n      return DynamicMessage.parseFrom(descriptor, rawMessage);\n    }\n    catch (Exception e) {\n      LOGGER.error(e.getMessage());\n      throw new ParseException(null, e, \"Fail to decode protobuf message!\");\n    }","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/protobuf-extensions/src/main/java/org/apache/druid/data/input/protobuf/SchemaRegistryBasedProtobufBytesDecoder.java#L169-L205","documentation":"parse() fetches the protobuf schema for the id embedded in the Confluent wire-format envelope via registry.getSchemaById(id). If the schema registry REST call fails (RestClientException — connectivity problems, unknown id, bad HTTP response), the decoder wraps it in a ParseException saying it could not get the protobuf schema.","triggerScenarios":"Consuming a message whose 4-byte schema id is not present in the configured registry (wrong registry, schema deleted, subject purged) or the registry is unreachable/timing out, raising RestClientException inside SchemaRegistryBasedProtobufBytesDecoder.parse.","commonSituations":"Pointing Druid at the wrong schema registry environment; schema auto-deregistered due to cleanup policies; network/firewall blocking Druid workers from the registry; registry returning 40403 unknown schema for an id.","solutions":["Verify 'urls' points to the schema registry that actually produced the messages and that it is reachable from Druid workers (curl the /schemas/ids/<id> endpoint)","Re-register the missing schema / restore the subject if it was deleted","Check registry auth (credentials/basic auth config) and network policies; increase timeouts if the registry is slow"],"exampleFix":"// before: wrong environment registry\n\"schemaRegistry\": {\"urls\": [\"http://staging-registry:8081\"]}\n// after\n\"schemaRegistry\": {\"urls\": [\"http://prod-registry:8081\"], \"config\": {\"basic.auth.credentials.source\": \"USER_INFO\", \"basic.auth.user.info\": \"user:pass\"}}","handlingStrategy":"try-catch","validationCode":"int schemaId = ByteBuffer.wrap(bytes, 1, 4).getInt();\ncurl -f http://registry:8081/schemas/ids/<schemaId>  // must return 200 with the schema","typeGuard":null,"tryCatchPattern":"try {\n  byte[] parsed = decoder.parse(bytes);\n} catch (ParseException e) {\n  if (e.getMessage().contains(\"can not connect to registry\")) {\n    log.error(\"Schema registry unreachable or schema id missing; check urls/network\", e.getCause());\n    // retry with backoff or route to DLQ\n  }\n}","preventionTips":["Pre-fetch every schema id expected on the topic via the registry REST API before starting ingestion","Open network/firewall access from Druid workers to the registry and set sane timeouts","Disable subject/schema deletion (or use immortal cleanup policy) for production subjects"],"tags":["network","schema-registry","rest-client"],"backgroundTag":"http-request-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}