{"record":{"id":"54ceb36363be11ef","repo":"apache/beam","slug":"failed-to-parse-datastore-key-from-bytes","errorCode":null,"errorMessage":"Failed to parse DataStore key from bytes.","messagePattern":"Failed to parse DataStore key from bytes\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/RowToEntity.java","lineNumber":154,"sourceCode":"    }\n\n    /**\n     * Create a random key for a {@code Row} without a keyField or use a user-specified key by\n     * parsing it from byte array when keyField is set.\n     *\n     * @param row {@code Row} to construct a key for.\n     * @return resulting {@code Key}.\n     */\n    private com.google.datastore.v1.Key constructKeyFromRow(Row row) {\n      if (!useNonRandomKey) {\n        // When key field is not present - use key supplier to generate a random one.\n        return makeKey(kind, keySupplier.get()).build();\n      }\n      byte[] keyBytes = row.getBytes(keyField);\n      try {\n        return com.google.datastore.v1.Key.parseFrom(keyBytes);\n      } catch (InvalidProtocolBufferException e) {\n        throw new IllegalStateException(\"Failed to parse DataStore key from bytes.\");\n      }\n    }\n\n    /**\n     * Converts a {@code Row} value to an appropriate DataStore {@code Value} object.\n     *\n     * @param value {@code Row} value to convert.\n     * @return resulting {@code Value}.\n     * @throws IllegalStateException when no mapping function for object of given type exists.\n     */\n    private Value mapObjectToValue(Object value) {\n      if (value == null) {\n        return Value.newBuilder().build();\n      }\n\n      if (Boolean.class.equals(value.getClass())) {\n        return makeValue((Boolean) value).build();\n      } else if (Byte.class.equals(value.getClass())) {","sourceCodeStart":136,"sourceCodeEnd":172,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/RowToEntity.java#L136-L172","documentation":"DatastoreV1 IO's RowToEntity can build a Datastore Key from a Beam Row either from kind/name fields or from a raw protobuf-serialized Key stored in a byte[] column. When the byte[] column is used, the bytes are parsed with Key.parseFrom; if the bytes are not a valid serialized com.google.datastore.v1.Key protobuf, an IllegalStateException with this message is thrown, aborting the element.","triggerScenarios":"The schema field designated as the key field contains bytes that are not a valid datastore.v1.Key protobuf message — e.g. the column holds a plain string key, a differently-serialized key, truncated bytes, or a key serialized with a different protobuf type/version.","commonSituations":"Migrating from another sink where the key column was written as a UTF-8 string rather than Key.toByteString(); pipeline version mismatch where the writer serialized keys differently than the reader expects; corrupted or hand-crafted test rows.","solutions":["Verify the key byte[] column contains exactly com.google.datastore.v1.Key.toByteString() output from a datastore client, not a raw string or another protobuf type","In the upstream pipeline, serialize keys with makeKey(...).build().toByteString().toByteArray() before writing the row","Check that no encoding/compression step between writer and reader corrupted the bytes (e.g. re-encoding as UTF-8 string)","As a workaround, use the kind + key-field name mode of RowToEntity instead of raw key bytes"],"exampleFix":"// before\nrow.getString(\"id\") // stored UTF-8 string in byte[] key column -> parseFrom fails\n// after\nbyte[] keyBytes = com.google.datastore.v1.Key.newBuilder()\n    .addPartitionId(...).build().toByteString().toByteArray(); // write valid Key protobuf bytes","handlingStrategy":"validation","validationCode":"if (keyBytes == null || keyBytes.length == 0) throw new IllegalArgumentException(\"empty key bytes\");\ntry { com.google.datastore.v1.Key.parseFrom(keyBytes); } catch (InvalidProtocolBufferException e) { throw new IllegalArgumentException(\"column is not a datastore Key protobuf\"); }","typeGuard":"boolean isValidDatastoreKey(byte[] b) {\n  if (b == null || b.length == 0) return false;\n  try { com.google.datastore.v1.Key.parseFrom(b); return true; }\n  catch (InvalidProtocolBufferException e) { return false; }\n}","tryCatchPattern":"try {\n  Key key = com.google.datastore.v1.Key.parseFrom(row.getBytes(keyField));\n} catch (InvalidProtocolBufferException | IllegalStateException e) {\n  LOG.error(\"Invalid key bytes in row\", e); // route to dead-letter / skip\n}","preventionTips":["Always write key columns with Key.toByteString().toByteArray()","Round-trip test: parseFrom(serialize(key)) in unit tests","Don't pass raw string ids into byte[] key columns; use kind+name mode instead"],"tags":["java","google-cloud-datastore","serialization","protobuf"],"backgroundTag":"protobuf-unmarshal-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}