{"record":{"id":"649179f42cd65e9c","repo":"apache/iceberg","slug":"not-an-avro-file","errorCode":null,"errorMessage":"Not an Avro file","messagePattern":"Not an Avro file","errorType":"exception","errorClass":"InvalidAvroMagicException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/avro/AvroIO.java","lineNumber":163,"sourceCode":"\n  static long findStartingRowPos(Supplier<SeekableInputStream> open, long start) {\n    long totalRows = 0;\n    try (SeekableInputStream in = open.get()) {\n      // use a direct decoder that will not buffer so the position of the input stream is accurate\n      BinaryDecoder decoder = DecoderFactory.get().directBinaryDecoder(in, null);\n\n      // an Avro file's layout looks like this:\n      //   header|block|block|...\n      // the header contains:\n      //   magic|string-map|sync\n      // each block consists of:\n      //   row-count|compressed-size-in-bytes|block-bytes|sync\n\n      // it is necessary to read the header here because this is the only way to get the expected\n      // file sync bytes\n      byte[] magic = MAGIC_READER.read(decoder, null);\n      if (!Arrays.equals(AVRO_MAGIC, magic)) {\n        throw new InvalidAvroMagicException(\"Not an Avro file\");\n      }\n\n      META_READER.read(decoder, null); // ignore the file metadata, it isn't needed\n      byte[] fileSync = SYNC_READER.read(decoder, null);\n\n      // the while loop reads row counts and seeks past the block bytes until the next sync pos is\n      // >= start, which\n      // indicates that the next sync is the start of the split.\n      byte[] blockSync = new byte[16];\n      long nextSyncPos = in.getPos();\n\n      while (nextSyncPos < start) {\n        if (nextSyncPos != in.getPos()) {\n          in.seek(nextSyncPos);\n          SYNC_READER.read(decoder, blockSync);\n\n          if (!Arrays.equals(fileSync, blockSync)) {\n            throw new RuntimeIOException(\"Invalid sync at %s\", nextSyncPos);","sourceCodeStart":145,"sourceCodeEnd":181,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/avro/AvroIO.java#L145-L181","documentation":"During Avro.IO.findStartingRowPos, the file header's magic bytes are read to locate the sync marker; if the magic does not match the expected Avro magic (\"Obj\\u0001\"), an InvalidAvroMagicException('Not an Avro file') is thrown. This indicates the file being position-scanned is not an Avro file, or its header is corrupted.","triggerScenarios":"Calling findStartingRowPos on a file whose bytes are not Avro (e.g. a Parquet/ORC data file misregistered as Avro in the manifest, or a metadata JSON path pointing at the wrong file), or on a truncated/corrupted Avro file whose header magic was overwritten.","commonSituations":"Manifests referencing the wrong file format after manual edits or catalog corruption; deleted files replaced by zero-byte or garbage content; cross-format file reuse bugs where delete files and data files were written with different formats.","solutions":["Verify the file's format in the manifest matches Avro (check file_path and file_format)","Inspect the first bytes of the file (should be 'Obj\\u0001') to confirm it is Avro","Fix or recompute the manifest entries pointing at the wrong files","Restore or rewrite the corrupted file if the header is damaged"],"exampleFix":"// before\n// manifest lists a Parquet file with format=AVRO\nlong pos = Avro.IO.findStartingRowPos(...) // InvalidAvroMagicException\n\n// after\n// ensure FileFormat.AVRO files only; or read with the correct format model\nCloseableIterable<T> rows = Parquet.read(file).project(schema).build();","handlingStrategy":"try-catch","validationCode":"try (FSDataInputStream in = io.newInputFile(path).newStream()) { byte[] head = new byte[4]; in.readFully(head); if (!Arrays.equals(head, new byte[]{'O','b','j',1})) { throw new IllegalStateException(path + \" is not an Avro file\"); } }","typeGuard":null,"tryCatchPattern":"try { return Avro.IO.findStartingRowPos(...); } catch (InvalidAvroMagicException e) { /* validate manifest format entry and file header; fail the scan with a clear error */ throw e; }","preventionTips":["Keep file_format entries in manifests accurate","Validate file headers after external writes or restores","Never manually swap data files across formats"],"tags":["avro","magic-bytes","file-format","corruption"],"backgroundTag":"schema-validation-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}