{"record":{"id":"0f8689a094b0ac18","repo":"pentaho/pentaho-kettle","slug":"avroinput-error-encounteredaprimitivepriortomapexpansion","errorCode":null,"errorMessage":"AvroInput.Error.EncounteredAPrimitivePriorToMapExpansion","messagePattern":"AvroInput\\.Error\\.EncounteredAPrimitivePriorToMapExpansion","errorType":"exception","errorClass":"KettleException","httpStatus":null,"severity":"error","filePath":"plugins/avro-format/core/src/main/java/org/pentaho/di/trans/steps/avro/input/AvroNestedReader.java","lineNumber":1119,"sourceCode":"            // either have a map or primitive here\n            if ( value instanceof Map ) {\n              // now have to look for the schema of the map\n              Schema mapSchema = null;\n              for ( Schema ts : valueType.getTypes() ) {\n                if ( ts.getType() == Schema.Type.MAP ) {\n                  mapSchema = ts;\n                  break;\n                }\n              }\n              if ( mapSchema == null ) {\n                throw new KettleException( BaseMessages.getString( PKG,\n                  \"AvroInput.Error.UnableToFindSchemaForUnionMap\" ) );\n              }\n              valueType = mapSchema;\n            } else {\n              // We shouldn't have a primitive here\n              if ( !ignoreMissing ) {\n                throw new KettleException( BaseMessages.getString( PKG,\n                  \"AvroInput.Error.EncounteredAPrimitivePriorToMapExpansion\" ) );\n              }\n              Object[][] result = new Object[ 1 ][ m_outputRowMeta.size() + RowDataUtil.OVER_ALLOCATE_SIZE ];\n              return result;\n            }\n          }\n        }\n\n        // what have we got?\n        if ( valueType.getType() == Schema.Type.RECORD ) {\n          return convertToKettleValues( (GenericData.Record) value, valueType, defaultSchema, space, ignoreMissing );\n        } else if ( valueType.getType() == Schema.Type.ARRAY ) {\n          return convertToKettleValues( (GenericData.Array) value, valueType, defaultSchema, space, ignoreMissing );\n        } else if ( valueType.getType() == Schema.Type.MAP ) {\n          return convertToKettleValues( (Map<Utf8, Object>) value, valueType, defaultSchema, space, ignoreMissing );\n        } else {\n          // we shouldn't have a primitive at this point. If we are\n          // extracting a particular key from the map then we're not to the","sourceCodeStart":1101,"sourceCodeEnd":1137,"githubUrl":"https://github.com/pentaho/pentaho-kettle/blob/f3058517a153da500bf4551f46d79b91bf8ec552/plugins/avro-format/core/src/main/java/org/pentaho/di/trans/steps/avro/input/AvroNestedReader.java#L1101-L1137","documentation":"Thrown by AvroNestedReader while walking an Avro map path: the schema resolved to a primitive type where a map was expected before the map-expansion phase could run. The reader cannot extract a keyed value from a primitive, so it aborts the row unless 'ignore missing paths' is enabled, in which case it returns an empty row.","triggerScenarios":"A field path in the Avro Input step targets a map (e.g. 'mymap.key') but the actual Avro schema at that point in the (possibly union-resolved) schema is a primitive (string, int, etc.), often because the schema changed between rows or the union branch selected is a primitive.","commonSituations":"Schema evolution: producer upgraded a field from a map<string,int> to a plain int; per-row schema switching where one version has a map and another a scalar; copy-pasted path definitions from an older schema.","solutions":["Correct the field path so it does not traverse into a map at a schema location that is a primitive","Verify the Avro schema used by the step matches the data (re-read the schema file / schema registry entry)","Enable 'Ignore missing paths' in the step if a blank row is acceptable for such records","Pin to a single schema version instead of per-row schema switching"],"exampleFix":"// before (path assumes map but schema has primitive)\npath: orders.total\n// after\npath: orders            // read scalar directly, no map key traversal\n// or fix the schema so 'orders' is map<string,int>","handlingStrategy":"validation","validationCode":"// Before configuring the path, check the schema branch at the path position\nSchema fieldSchema = schema.getField(\"orders\").schema();\nboolean isMap = fieldSchema.getType() == Schema.Type.MAP\n    || (fieldSchema.getType() == Schema.Type.UNION\n        && fieldSchema.getTypes().stream().anyMatch(t -> t.getType() == Schema.Type.MAP));\nif (!isMap) throw new IllegalArgumentException(\"orders is not a map in this schema\");","typeGuard":"boolean isMapSchema(Schema s) {\n  if (s.getType() == Schema.Type.UNION) {\n    return s.getTypes().stream().anyMatch(t -> t.getType() == Schema.Type.MAP);\n  }\n  return s.getType() == Schema.Type.MAP;\n}","tryCatchPattern":"try {\n  Object[][] rows = reader.convertMap(...);\n} catch (KettleException e) {\n  if (e.getMessage().contains(\"EncounteredAPrimitivePriorToMapExpansion\")) {\n    logMinimal(\"Path targets a primitive; skipping row\");\n    // return empty row / route to error handling\n  } else throw e;\n}","preventionTips":["Regenerate step paths whenever the Avro schema changes","Enable 'Ignore missing paths' for schemas that evolve across versions","Avoid per-row schema switching unless paths are validated per version","Inspect union branches before writing map-traversal paths"],"tags":["avro","schema","path-parsing"],"backgroundTag":"schema-validation-failed","analyzedSha":"f3058517a153da500bf4551f46d79b91bf8ec552","analyzedAt":"2026-09-13T14:04:16.340Z","contentChangedAt":"2026-09-13T14:04:16.340Z","schemaVersion":2},"datasetVersion":"2026-09-20T23:17:15.980Z"}