{"record":{"id":"4386bcc5e55d1cf1","repo":"apache/beam","slug":"could-not-find-python-snappy-google-crc32c-or-crcmod-to","errorCode":null,"errorMessage":"Could not find python-snappy, google-crc32c, or crcmod. To allow execution to succeed, make sure that one of these packages is installed or pip install apache-beam[tfrecord]","messagePattern":"Could not find python-snappy, google-crc32c, or crcmod\\. To allow execution to succeed, make sure that one of these packages is installed or pip install apache-beam\\[tfrecord\\]","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"sdks/python/apache_beam/io/tfrecordio.py","lineNumber":74,"sourceCode":"      if getattr(snappy, '_crc32c', None):\n        _default_crc32c_fn.fn = snappy._crc32c  # pylint: disable=protected-access\n      elif getattr(snappy, '_snappy', None):\n        _default_crc32c_fn.fn = snappy._snappy._crc32c  # pylint: disable=protected-access\n    except ImportError:\n      pass\n\n  if not _default_crc32c_fn.fn:\n    try:\n      import google_crc32c  # pylint: disable=import-error\n\n      if getattr(google_crc32c, 'value', None):\n        _default_crc32c_fn.fn = google_crc32c.value  # pylint: disable=protected-access\n    except ImportError:\n      pass\n\n    if not _default_crc32c_fn.fn:\n      if crcmod is None:\n        raise RuntimeError(\n            'Could not find python-snappy, google-crc32c, or crcmod. To allow '\n            'execution to succeed, make sure that one of these packages is '\n            'installed or pip install apache-beam[tfrecord]')\n      _LOGGER.warning(\n          'Couldn\\'t find python-snappy or google-crc32c so the '\n          'implementation of _TFRecordUtil._masked_crc32c is not as fast '\n          'as it could be.')\n      _default_crc32c_fn.fn = crcmod.predefined.mkPredefinedCrcFun('crc-32c')\n  return _default_crc32c_fn.fn(value)\n\n\n_default_crc32c_fn.fn = None  # type: ignore\n\n\nclass _TFRecordUtil(object):\n  \"\"\"Provides basic TFRecord encoding/decoding with consistency checks.\n\n  For detailed TFRecord format description see:","sourceCodeStart":56,"sourceCodeEnd":92,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/tfrecordio.py#L56-L92","documentation":"_default_crc32c_fn needs a CRC32C implementation (python-snappy, google-crc32c, or crcmod) to compute TFRecord checksums. If none is importable and crcmod is None, it raises RuntimeError because TFRecord reading requires a CRC function.","triggerScenarios":"Reading TFRecord files (_TFRecordUtil._masked_crc32c) in an environment lacking all three packages.","commonSituations":"Installing apache-beam without the [tfrecord] extra; slim runtime images on Dataflow/Flink runners.","solutions":["pip install apache-beam[tfrecord]","Or install one package directly: pip install crcmod, google-crc32c, or python-snappy","Add the package to requirements.txt so remote workers get it"],"exampleFix":"// before\n# no crc32c provider installed\n// after\npip install apache-beam[tfrecord]","handlingStrategy":"fallback","validationCode":"try:\n    import crcmod  # or google_crc32c / snappy\n    CRC_OK = True\nexcept ImportError:\n    CRC_OK = False","typeGuard":null,"tryCatchPattern":"try:\n    data = beam.io.ReadFromTFRecord(path)\nexcept RuntimeError as e:\n    if 'python-snappy' in str(e):\n        raise SystemExit('Install apache-beam[tfrecord] before running')\n    raise","preventionTips":["Use the apache-beam[tfrecord] extra when installing","Add crcmod or google-crc32c to requirements.txt for remote runners","Verify extras in Docker images used for workers"],"tags":["python","beam","tfrecord","dependency"],"backgroundTag":"missing-optional-dependency","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}