{"record":{"id":"979730191f2dbdfb","repo":"apache/beam","slug":"checksum-operation-failed","errorCode":null,"errorMessage":"Checksum operation failed","messagePattern":"Checksum operation failed","errorType":"exception","errorClass":"BeamIOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/aws/s3filesystem.py","lineNumber":283,"sourceCode":"    except Exception as e:  # pylint: disable=broad-except\n      raise BeamIOError(\"last_updated operation failed\", {path: e})\n\n  def checksum(self, path):\n    \"\"\"Fetch checksum metadata of a file on the\n    :class:`~apache_beam.io.filesystem.FileSystem`.\n\n    Args:\n      path: string path of a file.\n\n    Returns: string containing checksum\n\n    Raises:\n      ``BeamIOError``: if path isn't a file or doesn't exist.\n    \"\"\"\n    try:\n      return s3io.S3IO(options=self._options).checksum(path)\n    except Exception as e:  # pylint: disable=broad-except\n      raise BeamIOError(\"Checksum operation failed\", {path: e})\n\n  def metadata(self, path):\n    \"\"\"Fetch metadata fields of a file on the FileSystem.\n\n    Args:\n      path: string path of a file.\n\n    Returns:\n      :class:`~apache_beam.io.filesystem.FileMetadata`.\n\n    Raises:\n      ``BeamIOError``: if path isn't a file or doesn't exist.\n    \"\"\"\n    try:\n      file_metadata = s3io.S3IO(options=self._options)._status(path)\n      return FileMetadata(\n          path, file_metadata['size'], file_metadata['last_updated'])\n    except Exception as e:  # pylint: disable=broad-except","sourceCodeStart":265,"sourceCodeEnd":301,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/aws/s3filesystem.py#L265-L301","documentation":"Raised by S3FileSystem.checksum when S3IO.checksum(path) raises; wrapped as BeamIOError keyed by path. Checksum only works for existing S3 objects; S3 provides ETag-based checksums, so non-file paths or missing objects fail.","triggerScenarios":"Calling fs.checksum('s3://bucket/dir/') on a directory/prefix rather than a file; nonexistent object; boto3 failure during the underlying request.","commonSituations":"Passing an S3 prefix (folder) instead of an object key; comparing checksums of objects that were replaced during transfer; missing read permissions.","solutions":["Ensure the path points to an object, not a prefix/directory","Verify the object exists and the caller has s3:GetObject permission","Catch BeamIOError and fall back to size+mtime comparison for change detection","Check credentials/region if all checksum calls fail"],"exampleFix":"// before\nck = fs.checksum(prefix + '/')  # not a file\n// after\nif not prefix.endswith('/'):\n    ck = fs.checksum(prefix)\nelse:\n    raise ValueError('checksum requires a file path')","handlingStrategy":"validation","validationCode":"if path.endswith('/'):\n    raise ValueError('checksum requires a file path, not an s3 prefix')","typeGuard":null,"tryCatchPattern":"try:\n    ck = fs.checksum(path)\nexcept BeamIOError as e:\n    log.error('checksum failed for %s: %s', path, e.exception_details)\n    ck = None","preventionTips":["Only pass object keys, never prefixes, to checksum","Fall back to size+mtime comparison when checksum is unavailable","Verify object existence first"],"tags":["s3","io","aws","checksum"],"backgroundTag":"file-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}