apache/beam · error · ValueError

Unsupported cache format

Error message

Unsupported cache format: '%s'.

What it means

The RecordingCacheManager constructor validates the cache_format argument against the set of supported formats ('batch'/'text' style map in _available_formats). An unknown format string raises ValueError before reader/writer classes are selected.

Solutions

1. Pass one of the supported formats exactly as listed in cache_manager._available_formats (e.g. 'text' or the default 'batch').
2. Fix typos in the cache_format string.
3. Omit the argument to use the default format.
4. Check the installed apache-beam version's cache_manager.py if migrating code across versions, since supported formats changed.

Example fix

// before
cm = RecordingCacheManager(cache_dir, cache_format='json')
// after
cm = RecordingCacheManager(cache_dir, cache_format='text')

from apache_beam.runners.interactive import cache_manager
fmt = 'json'
if fmt not in cache_manager.RecordingCacheManager._available_formats:
    fmt = next(iter(cache_manager.RecordingCacheManager._available_formats))
cm = cache_manager.RecordingCacheManager(cache_dir, cache_format=fmt)

def valid_cache_format(fmt, mgr_cls):
    return fmt in mgr_cls._available_formats

try:
    cm = RecordingCacheManager(cache_dir, cache_format=fmt)
except ValueError as e:
    if 'Unsupported cache format' in str(e):
        cm = RecordingCacheManager(cache_dir)  # default format

When it happens

Trigger: Creating RecordingCacheManager(cache_dir, cache_format='parquet' or any unsupported string); passing a misspelled format like 'textt' or 'json'; constructing the manager programmatically instead of via the default.

Common situations: Custom interactive cache setups copying old code that used formats since removed; typos when overriding the default cache format; version drift where a format was dropped from _available_formats.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• GCS cache paths are not currently supported for streaming…
• Timed out waiting for cache file for PCollection
• Values given to streaming cache should be either…
• An unsupported type of cache was passed in. Received
• capacity of cache cannot be negative, got

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d66b32a1af96cf72. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/interactive/cache_manager.py:182

          lambda path: textio.ReadFromText(
              path, coder=Base64Coder(), compression_type=filesystems.
              CompressionTypes.BZIP2), lambda path: textio.WriteToText(
                  path, coder=Base64Coder(), compression_type=filesystems.
                  CompressionTypes.BZIP2)),
      'tfrecord': (tfrecordio.ReadFromTFRecord, tfrecordio.WriteToTFRecord)
  }

  def __init__(self, cache_dir=None, cache_format='text'):
    if cache_dir:
      self._cache_dir = cache_dir
    else:
      self._cache_dir = tempfile.mkdtemp(
          prefix='ib-', dir=os.environ.get('TEST_TMPDIR', None))
    self._versions = collections.defaultdict(lambda: self._CacheVersion())
    self.cache_format = cache_format

    if cache_format not in self._available_formats:
      raise ValueError("Unsupported cache format: '%s'." % cache_format)
    self._reader_class, self._writer_class = self._available_formats[
        cache_format]
    self._default_pcoder = (
        SafeFastPrimitivesCoder() if cache_format == 'text' else None)

    # List of saved pcoders keyed by PCollection path. It is OK to keep this
    # list in memory because once FileBasedCacheManager object is
    # destroyed/re-created it loses the access to previously written cache
    # objects anyways even if cache_dir already exists. In other words,
    # it is not possible to resume execution of Beam pipeline from the
    # saved cache if FileBasedCacheManager has been reset.
    #
    # However, if we are to implement better cache persistence, one needs
    # to take care of keeping consistency between the cached PCollection
    # and its PCoder type.
    self._saved_pcoders = {}

  def size(self, *labels):

View on GitHub (pinned to 12126d8942)