apache/beam · error · UnsupportedOperationException

Cannot read format: {}

Error message

Cannot read format: {}

What it means

ScanTaskReader.advance() builds an iterator over a data file's records and only implements readers for the file formats it knows (AVRO, PARQUET via specific builders). If the task's DataFile has a format outside the handled switch cases, it throws UnsupportedOperationException with the file's format. It means the connector cannot read this type of Iceberg data file.

Source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/ScanTaskReader.java:181

          iterable = parquetReader.build();
          break;
        case AVRO:
          LOG.info("Preparing Avro input.");
          Avro.ReadBuilder avroReader =
              Avro.read(input)
                  .split(fileTask.start(), fileTask.length())
                  .project(requiredSchema)
                  .createReaderFunc(
                      fileSchema -> DataReader.create(requiredSchema, fileSchema, idToConstants));

          if (nameMapping != null) {
            avroReader.withNameMapping(NameMappingParser.fromJson(nameMapping));
          }

          iterable = avroReader.build();
          break;
        default:
          throw new UnsupportedOperationException("Cannot read format: " + file.format());
      }
      GenericDeleteFilter deleteFilter =
          new GenericDeleteFilter(
              checkStateNotNull(io), fileTask, fileTask.schema(), requiredSchema);
      iterable = deleteFilter.filter(iterable);

      iterable = ReadUtils.maybeApplyFilter(iterable, source.getScanConfig());
      currentIterator = iterable.iterator();
    } while (true);

    return false;
  }

  @Override
  public Row getCurrent() throws NoSuchElementException {
    if (current == null) {
      throw new NoSuchElementException();
    }

View on GitHub (pinned to 12126d8942)

Solutions

1. Rewrite the table's files to a supported format: CALL catalog.system.rewrite_data_files(table => '...', options => map('format','parquet')).
2. Set the table's write format to parquet/avro (write.format.default) and re-ingest affected data.
3. If ORC support is required, upgrade the Beam IO iceberg connector or contribute/implement an ORC reader branch.

Example fix

// before
table.updateProperties().set("write.format.default", "orc").commit();
// after
table.updateProperties().set("write.format.default", "parquet").commit();

for (FileScanTask task : tasks) {
  FileFormat fmt = task.file().format();
  if (!fmt.equals(FileFormat.AVRO) && !fmt.equals(FileFormat.PARQUET)) {
    throw new IllegalStateException("Table contains unsupported file format: " + fmt + "; rewrite to parquet/avro first");
  }
}

try {
  reader.start();
} catch (UnsupportedOperationException e) {
  // trigger a rewrite job for the offending file format, then retry the pipeline
  throw new IllegalStateException("Unsupported data file format; rewrite table files", e);
}

When it happens

Trigger: A CombinedScanTask file whose FileFormat is not one of the implemented cases (e.g. ORC) is encountered while advancing the reader in ScanTaskReader.

Common situations: The Iceberg table contains ORC-written files but the pipeline expects AVRO/Parquet-only support; a table was populated by another engine using an unsupported format; format migration left mixed-format files.

Related errors

• Unsupported format: {format}
• smallBatchTag and largeBatchTag must be different
• Threshold must be a positive integer
• Encountered an error when parsing filter: '{filter}'
• Unsupported operation '{operation}' in filter expression: {c

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5cba244818f92bee. Report an issue: GitHub.