{"record":{"id":"93324c691a50c4d0","repo":"apache/beam","slug":"unmarshalling-partitionfn-data","errorCode":null,"errorMessage":"unmarshalling partitionFn data","messagePattern":"unmarshalling partitionFn data","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"sdks/go/pkg/beam/partition.go","lineNumber":180,"sourceCode":"\tkey := args[1]\n\tvalue := args[2]\n\n\tn := f.fnKV.Call2x1(key, value).(int)\n\tif n < 0 || n >= f.n {\n\t\treturn []any{errors.Errorf(\"partitionFn(%v) = %v, want [0,%v)\", value, n, f.n)}\n\t}\n\n\temit := args[n+3]\n\treflectx.MakeFunc3x0(emit).Call3x0(timestamp, key, value)\n\n\tvar err error\n\treturn []any{err}\n}\n\nfunc makePartitionFn(name string, t reflect.Type, enc []byte) reflectx.Func {\n\tvar data partitionData\n\tif err := json.Unmarshal(enc, &data); err != nil {\n\t\tpanic(errors.WithContext(err, \"unmarshalling partitionFn data\"))\n\t}\n\tif data.KV {\n\t\treturn &partitionFnKV{\n\t\t\tname: name,\n\t\t\tt:    t,\n\t\t\tn:    data.N,\n\t\t\tfnKV: reflectx.ToFunc2x1(data.Fn.Fn),\n\t\t}\n\t}\n\treturn &partitionFn{\n\t\tname: name,\n\t\tt:    t,\n\t\tn:    data.N,\n\t\tfn:   reflectx.ToFunc1x1(data.Fn.Fn),\n\t}\n}\n","sourceCodeStart":162,"sourceCodeEnd":197,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/partition.go#L162-L197","documentation":"makePartitionFn regenerates the partition DoFn on workers by JSON-unmarshaling previously serialized partitionData. If the encoded bytes cannot be unmarshaled, it panics with the context \"unmarshalling partitionFn data\". This is an internal deserialization failure of the transform's payload, usually caused by data corruption or a Beam version mismatch between job submission and worker.","triggerScenarios":"Worker-side expansion of a Partition transform where the marshaled partitionData bytes are corrupted, truncated, or were produced by an incompatible Beam SDK version (schema changed between releases).","commonSituations":"Mixed SDK versions: pipeline submitted with one version of Beam Go SDK, workers/staging containers running another; corrupted staged artifacts; custom builds of the SDK diverging on partitionData fields.","solutions":["Pin the same Apache Beam Go SDK version for submission and workers (matching container images, --workerHarnessContainerImage or environment config).","Clear stale staged artifacts/jar-less staging dirs and resubmit the job.","Reproduce locally with a direct runner; if it persists, inspect the panic's wrapped error for exact JSON offset/type failure and file a Beam issue with versions."],"exampleFix":"// before: mismatched versions\n// go.mod: github.com/apache/beam/sdks/v2 v2.45.0, workers built from v2.50.0\n\n// after: align versions\ngo get github.com/apache/beam/sdks/v2@v2.50.0\ngo mod tidy\n// rebuild worker image with the same SDK version","handlingStrategy":"validation","validationCode":"// pre-flight: verify SDK version consistency\n// go.mod SDK version must match worker container image tag\nif sdkVersion != workerImageVersion {\n    log.Fatalf(\"SDK mismatch: go.mod=%s worker=%s\", sdkVersion, workerImageVersion)\n}","typeGuard":null,"tryCatchPattern":"defer func() { if r := recover(); r != nil { err = fmt.Errorf(\"partitionFn deserialization failed: %v\", r) } }()","preventionTips":["Keep Beam Go SDK version identical across submission and worker images","Avoid custom local SDK builds in worker containers","Resubmit jobs with clean staging if payloads may be stale or corrupted"],"tags":["go","apache-beam","partition","json","unmarshal","worker"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}