{"record":{"id":"b47ebf3f2604d76a","repo":"apache/iceberg","slug":"invalid-sync-at-s","errorCode":null,"errorMessage":"Invalid sync at %s","messagePattern":"Invalid sync at (.+?)","errorType":"exception","errorClass":"RuntimeIOException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/avro/AvroIO.java","lineNumber":181,"sourceCode":"        throw new InvalidAvroMagicException(\"Not an Avro file\");\n      }\n\n      META_READER.read(decoder, null); // ignore the file metadata, it isn't needed\n      byte[] fileSync = SYNC_READER.read(decoder, null);\n\n      // the while loop reads row counts and seeks past the block bytes until the next sync pos is\n      // >= start, which\n      // indicates that the next sync is the start of the split.\n      byte[] blockSync = new byte[16];\n      long nextSyncPos = in.getPos();\n\n      while (nextSyncPos < start) {\n        if (nextSyncPos != in.getPos()) {\n          in.seek(nextSyncPos);\n          SYNC_READER.read(decoder, blockSync);\n\n          if (!Arrays.equals(fileSync, blockSync)) {\n            throw new RuntimeIOException(\"Invalid sync at %s\", nextSyncPos);\n          }\n        }\n\n        long rowCount = decoder.readLong();\n        long compressedBlockSize = decoder.readLong();\n\n        totalRows += rowCount;\n        nextSyncPos = in.getPos() + compressedBlockSize;\n      }\n\n      return totalRows;\n\n    } catch (EOFException e) {\n      return totalRows;\n\n    } catch (IOException e) {\n      throw new RuntimeIOException(e, \"Failed to read stream while finding starting row position\");\n    }","sourceCodeStart":163,"sourceCodeEnd":199,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/avro/AvroIO.java#L163-L199","documentation":"AvroIO.findStartingRowPos validates that each Avro block it skips over carries the same 16-byte sync marker as the file footer. When the block's sync marker differs from the file's declared sync marker, the reader cannot trust that the bytes at that offset belong to this file, so it throws RuntimeIOException to prevent reading corrupt or foreign data.","triggerScenarios":"Calling GenericAvroReader/AvroIterable row-position seeking (e.g. during delete-file positional skip or split alignment) where in.getPos() at nextSyncPos does not contain the expected sync marker, typically because the underlying file was modified, truncated-and-rewritten, or the seek offset passed to the reader is stale.","commonSituations":"A data file was overwritten or compacted after a task snapshot captured its length/offsets; corrupted object-store uploads; copying a file without copying footer metadata; using cached file handles (FileIO lazy handles) after the remote object changed under the same path.","solutions":["Re-open the data file (invalidate any cached FileIO/ContentFile handle) and retry the scan","Verify the file was not rewritten after the snapshot: re-read the current table metadata and re-plan the scan","Check for concurrent writers writing to the same file path (fix job-level concurrent-write issues)","If corruption persists, validate the file integrity (re-copy from source or restore from backup) and rewrite the affected data files with a rewrite action"],"exampleFix":"// before: reusing a stale cached file handle across commits\nInputFile in = table.io().newInputFile(staleLocation);\nAvroIterable<Record> it = Avro.read(in).project(schema).build();\n// after: re-resolve the file from current table metadata\nContentFile<?> current = findFile(currentSnapshot, dataFile.location());\nAvroIterable<Record> it = Avro.read(table.io().newInputFile(current.location()))\n    .project(schema)\n    .build();","handlingStrategy":"validation","validationCode":"long len = inputFile.getLength();\nif (startOffset < 0 || startOffset >= len) {\n  throw new IllegalArgumentException(\"Offset \" + startOffset + \" outside file of length \" + len);\n}\n// ensure the file has not changed since planning: compare with manifest-recorded size/checksum","typeGuard":null,"tryCatchPattern":"try {\n  iterable.forEachRemaining(rec -> consume(rec));\n} catch (RuntimeIOException e) {\n  if (isSyncMismatch(e)) { replanScanFromCurrentSnapshot(); } else { throw e; }\n}","preventionTips":["Never rewrite data files in place — write new files and commit atomically","Invalidate cached InputFile handles after each commit","Compare file size with the manifest's file-size-in-bytes before reading","Use snapshot isolation so reads see a consistent set of files"],"tags":["avro","io","corruption","seek"],"backgroundTag":"checksum-mismatch","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}