apache/iceberg · error · RuntimeIOException

Invalid sync at

Error message

Invalid sync at %s

What it means

AvroIO.findStartingRowPos validates that each Avro block it skips over carries the same 16-byte sync marker as the file footer. When the block's sync marker differs from the file's declared sync marker, the reader cannot trust that the bytes at that offset belong to this file, so it throws RuntimeIOException to prevent reading corrupt or foreign data.

Solutions

1. Re-open the data file (invalidate any cached FileIO/ContentFile handle) and retry the scan
2. Verify the file was not rewritten after the snapshot: re-read the current table metadata and re-plan the scan
3. Check for concurrent writers writing to the same file path (fix job-level concurrent-write issues)
4. If corruption persists, validate the file integrity (re-copy from source or restore from backup) and rewrite the affected data files with a rewrite action

Example fix

// before: reusing a stale cached file handle across commits
InputFile in = table.io().newInputFile(staleLocation);
AvroIterable<Record> it = Avro.read(in).project(schema).build();
// after: re-resolve the file from current table metadata
ContentFile<?> current = findFile(currentSnapshot, dataFile.location());
AvroIterable<Record> it = Avro.read(table.io().newInputFile(current.location()))
    .project(schema)
    .build();

long len = inputFile.getLength();
if (startOffset < 0 || startOffset >= len) {
  throw new IllegalArgumentException("Offset " + startOffset + " outside file of length " + len);
}
// ensure the file has not changed since planning: compare with manifest-recorded size/checksum

try {
  iterable.forEachRemaining(rec -> consume(rec));
} catch (RuntimeIOException e) {
  if (isSyncMismatch(e)) { replanScanFromCurrentSnapshot(); } else { throw e; }
}

When it happens

Trigger: Calling GenericAvroReader/AvroIterable row-position seeking (e.g. during delete-file positional skip or split alignment) where in.getPos() at nextSyncPos does not contain the expected sync marker, typically because the underlying file was modified, truncated-and-rewritten, or the seek offset passed to the reader is stale.

Common situations: A data file was overwritten or compacted after a task snapshot captured its length/offsets; corrupted object-store uploads; copying a file without copying footer metadata; using cached file handles (FileIO lazy handles) after the remote object changed under the same path.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• Cannot read manifest list file
• Cannot read manifest list file
• Cannot seek to position after end of file
• Decoding datum failed
• Failed to check range end

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b47ebf3f2604d76a. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/avro/AvroIO.java:181

        throw new InvalidAvroMagicException("Not an Avro file");
      }

      META_READER.read(decoder, null); // ignore the file metadata, it isn't needed
      byte[] fileSync = SYNC_READER.read(decoder, null);

      // the while loop reads row counts and seeks past the block bytes until the next sync pos is
      // >= start, which
      // indicates that the next sync is the start of the split.
      byte[] blockSync = new byte[16];
      long nextSyncPos = in.getPos();

      while (nextSyncPos < start) {
        if (nextSyncPos != in.getPos()) {
          in.seek(nextSyncPos);
          SYNC_READER.read(decoder, blockSync);

          if (!Arrays.equals(fileSync, blockSync)) {
            throw new RuntimeIOException("Invalid sync at %s", nextSyncPos);
          }
        }

        long rowCount = decoder.readLong();
        long compressedBlockSize = decoder.readLong();

        totalRows += rowCount;
        nextSyncPos = in.getPos() + compressedBlockSize;
      }

      return totalRows;

    } catch (EOFException e) {
      return totalRows;

    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to read stream while finding starting row position");
    }

View on GitHub (pinned to 86d9c8fc54)