{"record":{"id":"063b600edc370f88","repo":"apache/beam","slug":"unable-to-perform-expansion-for-transform","errorCode":null,"errorMessage":"Unable to perform expansion for transform ","messagePattern":"Unable to perform expansion for transform ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/graph/SplittableParDoExpander.java","lineNumber":312,"sourceCode":"              generateUniquePCollectonName(\n                  splittableParDo.getUniqueName() + \"/ProcessSizedElementsAndRestrictions\",\n                  existingComponents));\n          processSizedElementsAndRestrictions.setSpec(\n              FunctionSpec.newBuilder()\n                  .setUrn(\n                      PTransformTranslation.SPLITTABLE_PROCESS_SIZED_ELEMENTS_AND_RESTRICTIONS_URN)\n                  .setPayload(splittableParDo.getSpec().getPayload()));\n          processSizedElementsAndRestrictions.setEnvironmentId(splittableParDo.getEnvironmentId());\n          rval.getComponentsBuilder()\n              .putTransforms(\n                  processSizedElementsAndRestrictionsId,\n                  processSizedElementsAndRestrictions.build());\n        }\n        newCompositeRoot.addSubtransforms(processSizedElementsAndRestrictionsId);\n        rval.setPtransform(newCompositeRoot);\n        return rval.build();\n      } catch (IOException e) {\n        throw new RuntimeException(\"Unable to perform expansion for transform \" + transformId, e);\n      }\n    }\n  }\n\n  private static String getOrAddDoubleCoder(\n      ComponentsOrBuilder existingComponents, MessageWithComponents.Builder out) {\n    for (Map.Entry<String, Coder> coder : existingComponents.getCodersMap().entrySet()) {\n      if (ModelCoders.DOUBLE_CODER_URN.equals(coder.getValue().getSpec().getUrn())) {\n        return coder.getKey();\n      }\n    }\n    String doubleCoderId = generateUniqueId(\"DoubleCoder\", existingComponents::containsCoders);\n    out.getComponentsBuilder()\n        .putCoders(\n            doubleCoderId,\n            Coder.newBuilder()\n                .setSpec(FunctionSpec.newBuilder().setUrn(ModelCoders.DOUBLE_CODER_URN))\n                .build());","sourceCodeStart":294,"sourceCodeEnd":330,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/graph/SplittableParDoExpander.java#L294-L330","documentation":"Thrown by SplittableParDoExpander.getReplacement when expanding a SPLITTABLE_PROCESS_URN transform raises an IOException while constructing the replacement composite (building restriction coders, PairWithRestriction, ProcessSizedElementsAndRestrictions transforms). It marks that the automatic expansion of the splittable DoFn transform could not be performed.","triggerScenarios":"Calling getReplacement for a splittable ParDo transform where building sub-transform protos throws IOException — most commonly serializing the DoFn or coder payload via SerializableUtils/proto serialization inside the expansion.","commonSituations":"Splittable DoFns (e.g. file-based sources) with non-serializable restriction coders or DoFn state; classpath issues where referenced classes cannot be resolved during payload serialization; custom runners lacking splittable-DoFn expansion support.","solutions":["Inspect the chained IOException cause — it usually points to a serialization failure of the DoFn, coder, or restriction type.","Ensure the element/restriction coders are registered and serializable (check Coder verification with getCoderArguments).","Verify all classes referenced by the DoFn exist on the classpath used by the expansion code.","If your runner cannot support splittable DoFn, use a non-splittable source or register an expansion fallback."],"exampleFix":"// before: unserializable custom coder\nnew MyCoder() // holds non-serializable field\n// after\nclass MyCoder extends CustomCoder<MyT> {\n  void encode(MyT v, OutputStream out) { ... } // stateless, serializable\n}","handlingStrategy":"try-catch","validationCode":"// ensure DoFn and coders are serializable before expansion\ntry { java.io.ObjectOutputStream oos = new java.io.ObjectOutputStream(new java.io.ByteArrayOutputStream()); oos.writeObject(dofn); oos.close(); } catch (java.io.IOException e) { throw new IllegalArgumentException(\"DoFn is not serializable; expansion will fail\", e); }","typeGuard":"boolean isSerializable(Object o) { return o instanceof java.io.Serializable || org.apache.beam.sdk.util.SerializableUtils.serializeToByteArray(o) != null; }","tryCatchPattern":"try { replacement = expander.getReplacement(transformNode); } catch (RuntimeException e) { log.error(\"Splittable ParDo expansion failed for {}: {}\", transformId, e.getCause() != null ? e.getCause().getMessage() : e.getMessage(), e); throw e; }","preventionTips":["Make restriction coders stateless and fully serializable","Verify coder encode/decode round-trips in tests before using SplittableDoFn","Keep DoFn-enclosed fields serializable or transient-free for Beam closures"],"tags":["java","beam","splittable-dofn","expansion","serialization"],"backgroundTag":"api-request-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}