{"record":{"id":"e1586069755d3e65","repo":"apache/beam","slug":"cache-root-gcs-bucket-path-is-invalid","errorCode":null,"errorMessage":"cache_root GCS bucket path is invalid.","messagePattern":"cache_root GCS bucket path is invalid\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/interactive/interactive_environment.py","lineNumber":731,"sourceCode":"    if set_user_pipeline:\n      if chain.user_pipeline and chain.user_pipeline is not pipeline:\n        raise ValueError(\n            'The beam_sql magic tries to query PCollections from multiple '\n            'pipelines: %s and %s',\n            chain.user_pipeline,\n            pipeline)\n      chain.user_pipeline = pipeline\n    return chain\n\n  def _get_gcs_cache_dir(self, pipeline, cache_dir):\n    cache_dir_path = PurePath(cache_dir)\n    if len(cache_dir_path.parts) < 2:\n      _LOGGER.error(\n          'GCS bucket cache path \"%s\" is too short to be valid. See '\n          'https://cloud.google.com/storage/docs/naming-buckets for '\n          'the expected format.',\n          cache_dir)\n      raise ValueError('cache_root GCS bucket path is invalid.')\n    bucket_name = cache_dir_path.parts[1]\n    assert_bucket_exists(bucket_name)\n    return 'gs://{}/{}'.format('/'.join(cache_dir_path.parts[1:]), id(pipeline))\n\n  @property\n  def computing_pcollections(self):\n    return self._computing_pcolls\n\n  def mark_pcollection_computing(self, pcolls):\n    \"\"\"Marks the given pcolls as currently being computed.\"\"\"\n    self._computing_pcolls.update(pcolls)\n\n  def unmark_pcollection_computing(self, pcolls):\n    \"\"\"Removes the given pcolls from the computing set.\"\"\"\n    self._computing_pcolls.difference_update(pcolls)\n\n  def is_pcollection_computing(self, pcoll):\n    \"\"\"Checks if the given pcollection is currently being computed.\"\"\"","sourceCodeStart":713,"sourceCodeEnd":749,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/interactive/interactive_environment.py#L713-L749","documentation":"When cache_root points to a GCS path (gs://...), _get_gcs_cache_dir parses it and requires at least two path parts — a bucket name plus at least one object prefix component — before computing the per-pipeline cache dir. If the path is too short or oddly formed it logs the detailed reason and raises this ValueError, protecting the cache from writing to an invalid GCS location.","triggerScenarios":"Setting ib.options.cache_root to 'gs://' alone, a bucket-only path like 'gs://mybucket' that parses to fewer than 2 parts, a malformed URL missing the bucket (e.g. 'gs:///folder'), or a typo'd path; then calling get_cache_manager via any recording/show/compute/cache usage.","commonSituations":"Notebooks configured for Dataproc/GCP: typos in cache_root; copying a bucket URL without a folder suffix; switching from a local temp dir to GCS without updating the format; bucket names with invalid characters breaking parsing.","solutions":["Set cache_root to a full GCS path with a bucket and at least one folder, e.g. 'gs://my-bucket/cache'.","Verify the bucket name follows GCS naming rules (3-63 chars, lowercase, no underscores) per the docs linked in the log message.","Confirm the bucket exists and is accessible (assert_bucket_exists runs next and would fail otherwise).","Check for typos/extra slashes: 'gs:///x' or 'gs://' are invalid; use exactly 'gs://<bucket>/<prefix>'.","If remote caching isn't needed, use a local absolute directory path so the GCS branch is skipped."],"exampleFix":"// before: ib.options.cache_root = 'gs://my-bucket' | // after: ib.options.cache_root = 'gs://my-bucket/interactive_cache'","handlingStrategy":"validation","validationCode":"from pathlib import PurePosixPath; def valid_gcs_cache_root(path): return True if not (isinstance(path, str) and path.startswith('gs://')) else (len(PurePosixPath(path[5:]).parts) >= 2 and bool(PurePosixPath(path[5:]).parts[0])); assert valid_gcs_cache_root(ib.options.cache_root)","typeGuard":"def is_well_formed_gcs_path(s): return isinstance(s, str) and s.startswith('gs://') and '/' in s[5:]","tryCatchPattern":"try: ib.options.cache_root = candidate  # first recording builds cache manager | except ValueError as e: (set cache_root to 'gs://my-bucket/interactive_cache' if 'cache_root GCS' in str(e) else raise)","preventionTips":["Always include a bucket AND a folder: gs://<bucket>/<prefix>.","Validate bucket names against GCS naming rules before setting the option.","Never leave cache_root as bare 'gs://' or 'gs://bucket'.","Use a dedicated cache folder per notebook for predictable cleanup."],"tags":["python","apache-beam","interactive","gcs","configuration","value-error"],"backgroundTag":"invalid-url-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}