Performance degraded as records with different schema is…

Error message

Performance degraded as records with different schema is generated for the same table. Likely the DynamicRecord.schema is not reused. Reuse the same instance if the record schema is the same to improve performance

What it means

TableMetadataCache keeps an LRU of input Schemas mapped to resolved schema info. Each entry registers inputSchemaEvictionListener, which fires when a Schema is evicted from the cache. Frequent evictions mean DynamicRecords arriving with structurally identical schemas are carrying distinct Schema instances, defeating the cache and degrading sink performance. This is a performance warning, not a failure.

Solutions

Ensure DynamicRecord producers reuse a single Schema instance per distinct schema (intern/canonicalize schemas before emitting records).
Increase the input schema cache maximum size (cache maximum size config) if many genuinely distinct schemas are in use.
Check upstream operators for unnecessary serialization/deserialization of DynamicRecords that re-create Schema objects.

Example fix

// before
for (Row row : input) {
  out.collect(new DynamicRecord(table, new Schema(...), row)); // new Schema every record
}

// after
private static final Schema SCHEMA = new Schema(...);
for (Row row : input) {
  out.collect(new DynamicRecord(table, SCHEMA, row)); // reuse instance
}

Defensive patterns

Strategy: type-guard

Validate before calling

// Before emitting DynamicRecords, canonicalize schemas:
Schema canonical = schemaCache.computeIfAbsent(candidateSchema, s -> s);
// then construct records only with `canonical`

Type guard

private boolean isCanonical(Schema emitted, java.util.concurrent.ConcurrentHashMap<Schema, Schema> cache) {
  return cache.getOrDefault(emitted, emitted) == emitted;
}

Prevention

Intern Schema instances at the pipeline source (one instance per distinct schema).
Avoid creating Schema objects inside per-record loops or map functions.
Size the input schema cache for the actual number of distinct live schemas.

When it happens

Trigger: The inputSchema LRUCache (capacity inputSchemaCacheMaximumSize) evicts entries because upstream operators deserialize DynamicRecords independently, creating new Schema objects per record instead of reusing one instance for equal schemas.

Common situations: High-churn dynamic sink pipelines where records pass through serialization boundaries (e.g. network shuffle) without schema canonicalization; many distinct tables/schemas exceeding the cache size so legit churn looks like reuse failure.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b92567720428d3b8. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableMetadataCache.java:280

    private CacheItem(
        long createdTimestampMillis,
        boolean tableExists,
        Set<String> branches,
        Map<Integer, Schema> tableSchemas,
        Map<Integer, PartitionSpec> specs,
        int inputSchemaCacheMaximumSize) {
      this.createdTimestampMillis = createdTimestampMillis;
      this.tableExists = tableExists;
      this.branches = branches;
      this.tableSchemas = tableSchemas;
      this.specs = specs;
      this.inputSchemas =
          new LRUCache<>(inputSchemaCacheMaximumSize, CacheItem::inputSchemaEvictionListener);
    }

    private static void inputSchemaEvictionListener(
        Map.Entry<Schema, ResolvedSchemaInfo> evictedEntry) {
      LOG.warn(
          "Performance degraded as records with different schema is generated for the same table. "
              + "Likely the DynamicRecord.schema is not reused. "
              + "Reuse the same instance if the record schema is the same to improve performance");
    }

    @VisibleForTesting
    Map<Schema, ResolvedSchemaInfo> inputSchemas() {
      return inputSchemas;
    }
  }

  static class ResolvedSchemaInfo {
    private final Schema resolvedTableSchema;
    private final CompareSchemasVisitor.Result compareResult;
    private final DataConverter recordConverter;

    ResolvedSchemaInfo(
        Schema tableSchema,

View on GitHub (pinned to 86d9c8fc54)