apache/beam · error · ValueError

Unrecognized unit.

Error message

Unrecognized unit.

What it means

After extracting the trailing non-digit suffix of a storage size string, _parse_storage_size_str checks it against a fixed unit table ('TB','GB','MB','KB','B'). A suffix not in that table is rejected with this error. Units like 'MiB', 'GiB' or 'm' are not accepted.

Solutions

1. Use exactly one of the supported units: TB, GB, MB, KB, B (correct case, no plural).
2. Convert binary units manually: 512MiB -> '536870912B' or approximately '537MB'.
3. Normalize/strip the value of whitespace before passing it, e.g. ' 512MB ' -> '512MB' if stray spaces cause a bad suffix.

Example fix

// before
add_resource_hint('storage=2GiB')
// after
add_resource_hint('storage=2GB')

UNITS = {'TB', 'GB', 'MB', 'KB', 'B'}
suffix = re.search(r'(\D+)$', value).group(1) if re.search(r'\D+$', value) else ''
assert suffix in UNITS, f"{value!r}: unit {suffix!r} not in {UNITS}"

def has_supported_unit(value: str) -> bool:
    m = re.search(r'(\D+)$', value)
    return m is not None and m.group(1) in {'TB','GB','MB','KB','B'}

try:
    hints = parse_resource_hints(raw_hints)
except ValueError as e:
    if 'Unrecognized unit' in str(e):
        raw_hints = {k: normalize_unit(v) for k, v in raw_hints.items()}
        hints = parse_resource_hints(raw_hints)

When it happens

Trigger: Passing a hint value such as '512MiB', '4G', or '2mb' (case-sensitive lookup) to a storage resource hint; parse succeeds on the regex but the suffix is absent from the units dict.

Common situations: Mixing binary prefixes (MiB/GiB) with Beam's decimal units (MB/GB); lowercase unit spellings; inventing units like 'TBs' or 'gigabytes'.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Input must be a string.
• Input must be a string or integer.
• Input must be an integer.
• Resource hint has invalid value .
• Unknown resource hint

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0fcb987826606350. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/resources.py:138

        'PiB': 2**50,
        'TiB': 2**40,
        'GiB': 2**30,
        'MiB': 2**20,
        'KiB': 2**10,
        'PB': 10**15,
        'TB': 10**12,
        'GB': 10**9,
        'MB': 10**6,
        'KB': 10**3,
        'B': 1,
    }
    match = re.match(r'.*?(\D+)$', value)
    if not match:
      raise ValueError("Unrecognized value pattern.")

    suffix = match.group(1)
    if suffix not in units:
      raise ValueError("Unrecognized unit.")
    multiplier = units[suffix]
    value = value[:-len(suffix)]

    return str(round(float(value) * multiplier)).encode('ascii')

  @staticmethod
  def _use_max(v1, v2):
    return str(max(int(v1), int(v2))).encode('ascii')

  @staticmethod
  def _use_sum(v1, v2):
    return str(int(v1) + int(v2)).encode('ascii')


class AcceleratorHint(ResourceHint):
  """Describes desired hardware accelerators in execution environment."""
  urn = resource_hints.ACCELERATOR.urn

View on GitHub (pinned to 12126d8942)