{"record":{"id":"5cba244818f92bee","repo":"apache/beam","slug":"cannot-read-format","errorCode":null,"errorMessage":"Cannot read format: {}","messagePattern":"Cannot read format: (.+?)","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/ScanTaskReader.java","lineNumber":181,"sourceCode":"          iterable = parquetReader.build();\n          break;\n        case AVRO:\n          LOG.info(\"Preparing Avro input.\");\n          Avro.ReadBuilder avroReader =\n              Avro.read(input)\n                  .split(fileTask.start(), fileTask.length())\n                  .project(requiredSchema)\n                  .createReaderFunc(\n                      fileSchema -> DataReader.create(requiredSchema, fileSchema, idToConstants));\n\n          if (nameMapping != null) {\n            avroReader.withNameMapping(NameMappingParser.fromJson(nameMapping));\n          }\n\n          iterable = avroReader.build();\n          break;\n        default:\n          throw new UnsupportedOperationException(\"Cannot read format: \" + file.format());\n      }\n      GenericDeleteFilter deleteFilter =\n          new GenericDeleteFilter(\n              checkStateNotNull(io), fileTask, fileTask.schema(), requiredSchema);\n      iterable = deleteFilter.filter(iterable);\n\n      iterable = ReadUtils.maybeApplyFilter(iterable, source.getScanConfig());\n      currentIterator = iterable.iterator();\n    } while (true);\n\n    return false;\n  }\n\n  @Override\n  public Row getCurrent() throws NoSuchElementException {\n    if (current == null) {\n      throw new NoSuchElementException();\n    }","sourceCodeStart":163,"sourceCodeEnd":199,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/ScanTaskReader.java#L163-L199","documentation":"ScanTaskReader.advance() builds an iterator over a data file's records and only implements readers for the file formats it knows (AVRO, PARQUET via specific builders). If the task's DataFile has a format outside the handled switch cases, it throws UnsupportedOperationException with the file's format. It means the connector cannot read this type of Iceberg data file.","triggerScenarios":"A CombinedScanTask file whose FileFormat is not one of the implemented cases (e.g. ORC) is encountered while advancing the reader in ScanTaskReader.","commonSituations":"The Iceberg table contains ORC-written files but the pipeline expects AVRO/Parquet-only support; a table was populated by another engine using an unsupported format; format migration left mixed-format files.","solutions":["Rewrite the table's files to a supported format: CALL catalog.system.rewrite_data_files(table => '...', options => map('format','parquet')).","Set the table's write format to parquet/avro (write.format.default) and re-ingest affected data.","If ORC support is required, upgrade the Beam IO iceberg connector or contribute/implement an ORC reader branch."],"exampleFix":"// before\ntable.updateProperties().set(\"write.format.default\", \"orc\").commit();\n// after\ntable.updateProperties().set(\"write.format.default\", \"parquet\").commit();","handlingStrategy":"validation","validationCode":"for (FileScanTask task : tasks) {\n  FileFormat fmt = task.file().format();\n  if (!fmt.equals(FileFormat.AVRO) && !fmt.equals(FileFormat.PARQUET)) {\n    throw new IllegalStateException(\"Table contains unsupported file format: \" + fmt + \"; rewrite to parquet/avro first\");\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  reader.start();\n} catch (UnsupportedOperationException e) {\n  // trigger a rewrite job for the offending file format, then retry the pipeline\n  throw new IllegalStateException(\"Unsupported data file format; rewrite table files\", e);\n}","preventionTips":["Set write.format.default=parquet on all Iceberg tables read by Beam.","Audit tables written by other engines for ORC/unsupported files before running pipelines.","Schedule rewrite_data_files maintenance jobs to normalize formats."],"tags":["iceberg","java","apache-beam","file-format","reader"],"backgroundTag":"unsupported-dtype","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}