apache/beam · error · IllegalStateException

Table schema is null.

Error message

Table schema is null.

What it means

After resolving a snapshot via version or timestamp, expand() fetches its schema via snapshot.getSchema(). Delta Kernel should always return a schema; a null means the connector's internal invariant was violated, so it throws IllegalStateException instead of producing a Beam schema.

Solutions

1. Verify the table is a valid Delta table (complete _delta_log with valid schema in the latest commit).
2. Align the Delta Kernel dependency version with the table's protocol/minReaderVersion — upgrade the kernel if the table uses newer features.
3. Re-check the resolved version/timestamp; try reading the latest snapshot to see if only a historical version is corrupt.
4. If the table is corrupt, restore from a recent commit or rewrite the table.

Example fix

// before
snapshot = table.getSnapshotAsOfTimestamp(engine, ts); // corrupt/partial _delta_log
// after
// validate the table first (e.g. with delta-spark): DESCRIBE HISTORY / VALIDATE, or read latest snapshot
snapshot = table.getLatestSnapshot(engine);
StructType schema = snapshot.getSchema();
if (schema == null) throw new IllegalStateException("table at " + path + " has no valid schema");

// validate the table is readable before launching Beam
// run: DESCRIBE HISTORY <path> (delta-spark) or read latest snapshot with Kernel to confirm schema

try { pipeline.run(); } catch (IllegalStateException e) { if (e.getMessage().equals("Table schema is null.")) { validateTableAndKernelVersion(); } else { throw e; } }

When it happens

Trigger: snapshot.getSchema() returning null — unexpected Delta Kernel behavior, a corrupted/missing Delta log that yields a degenerate snapshot, or a connector/Kernel version mismatch.

Common situations: Pointing at a directory with a partial or corrupt _delta_log; using a Delta Kernel version incompatible with the table protocol/writer features; a table where schema was never properly committed.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Beam write property ' ' is not supported. Writing to Delta…
• Can't compare with NULL
• cannot encode a null value
• cannot encode a null value
• Cannot set both endVersion and endTimestamp.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/88d9e96ec1ad6588. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/delta/src/main/java/org/apache/beam/sdk/io/delta/DeltaIO.java:164

          conf.set(entry.getKey(), entry.getValue());
        }
      }
      Engine engine = DefaultEngine.create(conf);
      Table table = Table.forPath(engine, path);
      Snapshot snapshot;
      Long versionVal = getVersion();
      String timestampVal = getTimestamp();
      if (versionVal != null) {
        snapshot = table.getSnapshotAsOfVersion(engine, versionVal);
      } else if (timestampVal != null) {
        long timestampMillis = java.time.Instant.parse(timestampVal).toEpochMilli();
        snapshot = table.getSnapshotAsOfTimestamp(engine, timestampMillis);
      } else {
        snapshot = table.getLatestSnapshot(engine);
      }
      StructType deltaSchema = snapshot.getSchema();
      if (deltaSchema == null) {
        throw new IllegalStateException("Table schema is null.");
      }
      Schema beamSchema = convertToBeamSchema(deltaSchema);

      return input
          .apply("Create Path", Create.of(path))
          .apply(
              "Plan Files",
              ParDo.of(new CreateReadTasksDoFn(hadoopConfig, getVersion(), getTimestamp())))
          .apply("Read Logical Data", ParDo.of(new DeltaSourceDoFn(hadoopConfig)))
          .setRowSchema(beamSchema);
    }

    static Schema convertToBeamSchema(StructType deltaSchema) {
      Schema.Builder builder = Schema.builder();
      for (StructField field : deltaSchema.fields()) {
        builder.addField(
            Schema.Field.of(field.getName(), convertToBeamFieldType(field.getDataType()))
                .withNullable(field.isNullable()));

View on GitHub (pinned to 12126d8942)