{"record":{"id":"8a495bcd9244449a","repo":"pathwaycom/pathway","slug":"incorrect-s3-path-s3-path","errorCode":null,"errorMessage":"Incorrect S3 path: {s3_path}","messagePattern":"Incorrect S3 path: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/internals/_io_helpers.py","lineNumber":178,"sourceCode":"                f\"https://s3.amazonaws.com/{bucket}\",\n                allow_redirects=False,\n                timeout=S3_REGION_DETECTION_TIMEOUT_S,\n            )\n            region = response.headers.get(\"x-amz-bucket-region\")\n            if region is None:\n                raise ValueError(\n                    f\"Failed to detect the region of S3 bucket {bucket!r} \"\n                    f\"(HTTP status {response.status_code}): the bucket may not exist. \"\n                    \"If it does, pass AwsS3Settings with an explicit region\"\n                )\n\n            return cls(\n                bucket_name=bucket,\n                region=region,\n            )\n\n        # If it doesn't start with a valid S3 prefix, it's not a full S3 path\n        raise ValueError(f\"Incorrect S3 path: {s3_path}\")\n\n    def authorize(self):\n        \"\"\"Fills in the credentials that the downstream libraries can't deduce.\n\n        The DeltaLake library resolves environment variables and instance\n        credentials on its own, but does not read AWS profile files — those are\n        resolved here, with the official AWS SDK chain (environment, profile\n        files, SSO, assume-role, IMDS) built into the engine.\n        \"\"\"\n        if self._access_key is not None and self._secret_access_key is not None:\n            return\n        env_access_key = os.environ.get(\"AWS_ACCESS_KEY_ID\")\n        env_secret_access_key = os.environ.get(\"AWS_SECRET_ACCESS_KEY\")\n        if env_access_key and env_secret_access_key:\n            # pinned explicitly instead of left for the engine to rediscover:\n            # this snapshot is immune to the credential vars that delta-rs\n            # writes into the process env for previously opened tables\n            self._access_key = env_access_key","sourceCodeStart":160,"sourceCodeEnd":196,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/internals/_io_helpers.py#L160-L196","documentation":"With output_table_type=\"snapshot\", pw.io.clickhouse.write maintains the current row state keyed by primary key columns (using ClickHouse ReplacingMergeTree-style versioning), so it must know which column(s) identify a row. primary_key is mandatory in that mode.","triggerScenarios":"Calling pw.io.clickhouse.write(..., output_table_type=\"snapshot\") with primary_key omitted, None, or an empty list.","commonSituations":"User switches from default append mode (where primary_key is not needed) to snapshot mode to get up-to-date rows in ClickHouse and forgets to add the key; or passes primary_key_columns (an older/different spelling) instead of primary_key.","solutions":["Pass primary_key with the identifying column(s), e.g. primary_key=t.id or primary_key=[t.id, t.tenant_id].","Ensure the key columns come from the same table being written.","If you actually wanted append-only output, drop output_table_type=\"snapshot\" instead."],"exampleFix":"# before\npw.io.clickhouse.write(t, \"db.pets\", output_table_type=\"snapshot\")\n\n# after\npw.io.clickhouse.write(t, \"db.pets\", output_table_type=\"snapshot\", primary_key=t.id)","handlingStrategy":"validation","validationCode":"if output_table_type == \"snapshot\":\n    assert primary_key is not None and len(list(primary_key)) > 0, (\n        \"output_table_type='snapshot' requires primary_key\"\n    )\npw.io.clickhouse.write(t, ..., output_table_type=output_table_type, primary_key=primary_key)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Model the writer config so snapshot mode cannot be built without a primary_key field.","Pick key columns that truly identify a row across updates (stable ids, not mutable attributes)."],"tags":["pathway","clickhouse","snapshot","primary-key","validation"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}