{"record":{"id":"65403a3b0f174e43","repo":"apache/beam","slug":"keys-to-be-deleted-from-cloud-datastore-must-be-complete-s","errorCode":null,"errorMessage":"Keys to be deleted from Cloud Datastore must be complete:\n%s","messagePattern":"Keys to be deleted from Cloud Datastore must be complete:\n(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/datastore/v1new/datastoreio.py","lineNumber":598,"sourceCode":"        be deleted.\n      throttle_rampup: Whether to enforce a gradual ramp-up.\n      hint_num_workers: A hint for the expected number of workers, used to\n                        estimate appropriate limits during ramp-up throttling.\n    \"\"\"\n    mutate_fn = DeleteFromDatastore._DatastoreDeleteFn(project)\n    super().__init__(mutate_fn, throttle_rampup, hint_num_workers)\n\n  class _DatastoreDeleteFn(_Mutate.DatastoreMutateFn):\n    def element_to_client_batch_item(self, element):\n      if not isinstance(element, types.Key):\n        raise ValueError(\n            'apache_beam.io.gcp.datastore.v1new.datastoreio.Key'\n            ' expected, got: %s' % type(element))\n      if not element.project:\n        element.project = self._project\n      client_key = element.to_client_key()\n      if client_key.is_partial:\n        raise ValueError(\n            'Keys to be deleted from Cloud Datastore must be '\n            'complete:\\n%s' % client_key)\n      return client_key\n\n    def add_to_batch(self, client_key):\n      self._batch.delete(client_key)\n\n    def display_data(self):\n      return {\n          'mutation': 'Delete',\n          'project': self._project,\n      }\n","sourceCodeStart":580,"sourceCodeEnd":611,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/datastore/v1new/datastoreio.py#L580-L611","documentation":"DeleteFromDatastore requires every Key to be complete — a partial key (missing the final id/name) does not uniquely identify an entity, so the sink refuses to delete it, raising ValueError after checking client_key.is_partial.","triggerScenarios":"Deleting a v1new types.Key built without an id/name on its last path element, e.g. types.Key('Person', project=project), or a key read from an entity whose key was never finalized.","commonSituations":"Constructing delete keys from incomplete records; missing the id field in input data due to schema changes; assuming wild-card/partial-key deletes are supported (they are not in the Beam sink).","solutions":["Supply the full key including id or name: types.Key(kind, id_or_name, project=project).","Fix the upstream data extraction so the identifying field is present (check for None/missing ids before the sink).","Filter out or dead-letter elements with incomplete keys using a validation step prior to DeleteFromDatastore."],"exampleFix":"// before\nkey = types.Key('Person', project=project)\n| 'delete' >> DeleteFromDatastore(project)\n// after\nkey = types.Key('Person', record['person_id'], project=project)\n| 'delete' >> DeleteFromDatastore(project)","handlingStrategy":"validation","validationCode":"def assert_complete_delete_key(key):\n    from apache_beam.io.gcp.datastore.v1new import types\n    if not isinstance(key, types.Key):\n        raise TypeError('need types.Key')\n    if key.id is None and key.name is None:\n        raise ValueError('cannot delete: key is partial (no id/name)')","typeGuard":"def is_complete_key(key):\n    return isinstance(key, types.Key) and (key.id is not None or key.name is not None)","tryCatchPattern":"bad = pcoll | 'filter_partial' >> beam.Filter(lambda k: k.id is None and k.name is None)\ntry:\n    _ = pcoll | DeleteFromDatastore(project)\nexcept ValueError as e:\n    if 'must be complete' in str(e):\n        raise RuntimeError('partial keys reached delete sink') from e","preventionTips":["Verify source records actually contain the id/name before building keys","Drop or dead-letter partial keys with beam.Filter before the sink","Log and monitor filtered-out partial keys to catch upstream schema drift","Never assume wild-card deletes on partial keys"],"tags":["python","datastore","validation","beam-io"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}