{"record":{"id":"ceb59544e906060c","repo":"apache/beam","slug":"unexpected-null-schema-for-entry-getkey","errorCode":null,"errorMessage":"Unexpected null schema for ${entry.getKey()}","messagePattern":"Unexpected null schema for (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableSchemaCache.java","lineNumber":292,"sourceCode":"                    @Nullable SchemaHolder schemaHolder = cachedSchemas.get(entry.getKey());\n                    return schemaHolder != null\n                        && schemaHolder.getVersion() >= entry.getValue().getTargetVersion();\n                  });\n        } finally {\n          tableUpdateMonitor.leave();\n        }\n      }\n\n      // Query all the tables for their schema.\n      final Map<String, @Nullable TableSchema> schemas = refreshAll(localTablesToRefresh);\n\n      runUnderMonitor(\n          () -> {\n            // Update the cache schemas.\n            for (Map.Entry<String, @Nullable TableSchema> entry : schemas.entrySet()) {\n              SchemaHolder schemaHolder = cachedSchemas.get(entry.getKey());\n              if (schemaHolder == null) {\n                throw new RuntimeException(\"Unexpected null schema for \" + entry.getKey());\n              }\n\n              if (entry.getValue() == null) {\n                // There was an error fetching the schema. Reschedule it.\n                Refresh oldRefresh =\n                    Preconditions.checkStateNotNull(localTablesToRefresh.get(entry.getKey()));\n                Refresh existingRefresh = this.tablesToRefresh.get(entry.getKey());\n                if (existingRefresh == null\n                    || oldRefresh.getTargetVersion() > existingRefresh.getTargetVersion()) {\n                  this.tablesToRefresh.put(entry.getKey(), oldRefresh);\n                }\n              } else {\n                SchemaHolder newSchema =\n                    SchemaHolder.of(entry.getValue(), schemaHolder.getVersion() + 1);\n                cachedSchemas.put(entry.getKey(), newSchema);\n              }\n            }\n          });","sourceCodeStart":274,"sourceCodeEnd":310,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableSchemaCache.java#L274-L310","documentation":"TableSchemaCache's refresh thread updates cached schemas from a map of fetched results. Every key returned by the refresh loop must already exist in cachedSchemas; if it does not, the internal invariant is broken and this RuntimeException is thrown naming the missing table key.","triggerScenarios":"The background refresh thread completes schema fetches and iterates the result map, but a table key in the results is absent from cachedSchemas — typically because the entry was removed or never registered while refreshes were in flight.","commonSituations":"Concurrent stop/eviction racing with the refresh thread, a table being removed from the cache while its refresh was queued, or a bug in cache bookkeeping during dynamic destination churn.","solutions":["Report this as a bug to the Apache Beam project with pipeline details — it indicates an internal race/invariant violation","Avoid stopping or mutating the cache concurrently with active refreshes; check lifecycle ordering","Upgrade Beam to pick up any fix for the cache race","Log the table key and reproduce with a minimal pipeline before filing the issue","As a workaround, restart the affected worker/pipeline stage"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"SchemaHolder holder = cachedSchemas.get(key); if (holder == null) { /* skip or re-register before update */ }","typeGuard":null,"tryCatchPattern":"try {\n  schemaCache.refreshSchema(tableRef, service, options);\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Unexpected null schema for\")) {\n    LOG.error(\"Cache invariant violation for table; restarting stage\", e);\n    // fail pipeline or recreate cache\n  } else throw e;\n}","preventionTips":["Don't evict/modify cache entries concurrently with the refresh thread","Keep stop/cleanup strictly ordered after refresh completion","Pin a Beam version without this race; upgrade when fixes land","File a bug with repro details if it occurs"],"tags":["java","google-bigquery","concurrency","invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}