{"record":{"id":"a3c516053bc6523b","repo":"apache/beam","slug":"unable-to-get-latest-schema-metadata-for-subject","errorCode":null,"errorMessage":"Unable to get latest schema metadata for subject: ","messagePattern":"Unable to get latest schema metadata for subject: ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ConfluentSchemaRegistryDeserializerProvider.java","lineNumber":147,"sourceCode":"    return deserializer;\n  }\n\n  @Override\n  public Coder<T> getCoder(CoderRegistry coderRegistry) {\n    return (Coder<T>) AvroCoder.of(getAvroSchema());\n  }\n\n  private Schema getAvroSchema() {\n    return new Schema.Parser().parse(getSchemaMetadata().getSchema());\n  }\n\n  private SchemaMetadata getSchemaMetadata() {\n    try {\n      return (version == null)\n          ? getSchemaRegistryClient().getLatestSchemaMetadata(subject)\n          : getSchemaRegistryClient().getSchemaMetadata(subject, version);\n    } catch (IOException | RestClientException e) {\n      throw new RuntimeException(\"Unable to get latest schema metadata for subject: \" + subject, e);\n    }\n  }\n\n  private SchemaRegistryClient getSchemaRegistryClient() {\n    return this.schemaRegistryClientProviderFn.apply(null);\n  }\n}\n\nclass ConfluentSchemaRegistryDeserializer extends KafkaAvroDeserializer {\n  Schema readerSchema;\n\n  ConfluentSchemaRegistryDeserializer(SchemaRegistryClient client, Schema readerSchema) {\n    super(client);\n    this.readerSchema = readerSchema;\n  }\n\n  @Override\n  public Object deserialize(String s, byte[] bytes) {","sourceCodeStart":129,"sourceCodeEnd":165,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ConfluentSchemaRegistryDeserializerProvider.java#L129-L165","documentation":"ConfluentSchemaRegistryDeserializerProvider.getSchemaMetadata() fetches the schema metadata from the Confluent Schema Registry, either the latest version or a pinned version. Any IOException or RestClientException from the registry client is rethrown as a RuntimeException prefixed \"Unable to get latest schema metadata for subject: \". This indicates the pipeline could not resolve the subject's schema at startup.","triggerScenarios":"getSchemaMetadata() called (directly or via getAvroSchema) when the schema registry is unreachable, the subject does not exist, the version doesn't exist, or auth/TLS to the registry fails.","commonSituations":"Wrong schema.registry.url; subject renamed or deleted (schema cleanup policies); missing basic auth credentials; registry behind TLS with untrusted certs; network segmentation between workers and registry.","solutions":["Verify the subject exists: curl the registry at /subjects and /subjects/<subject>/versions.","Check the registry URL, credentials (auth), and TLS trust configuration passed to the provider.","If a specific version was pinned, confirm that version still exists (registry cleanup may have removed it) or drop the version pin to use latest.","Confirm workers can reach the registry host/port (network/VPC/firewall)."],"exampleFix":"// before\nConfluentSchemaRegistryDeserializerProvider.of(\"bad-url\", \"my-topic-value\", null, ...)\n// after\nConfluentSchemaRegistryDeserializerProvider.of(\"https://schema-registry:8081\", \"my-topic-value\", null,\n  Collections.singletonMap(\"basic.auth.credentials.source\", \"USER_INFO\"), ...)","handlingStrategy":"retry","validationCode":"// pre-flight check against the registry\nHttpGet req = new HttpGet(registryUrl + \"/subjects/\" + subject + \"/versions/latest\");\n// expect 200 with auth headers; 404 means subject missing","typeGuard":null,"tryCatchPattern":"try { getAvroSchema(subject); }\ncatch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Unable to get latest schema metadata\")) {\n    // check subject existence / registry reachability, retry with backoff\n  }\n  throw e;\n}","preventionTips":["Verify subject names follow the TopicNameStrategy used by producers (topic-value by default).","Configure registry auth and TLS correctly for worker environments.","Avoid pinning schema versions subject to registry cleanup; or pin and monitor versions."],"tags":["kafka","schema-registry","network"],"backgroundTag":"entity-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}