What causes "XMLInputStream.Log.TooManyNestedElements" in pentaho/pentaho-kettle?

Flatten or pre-process the XML (XSLT or an external preprocessing step) to reduce nesting depth below the limit.

pentaho/pentaho-kettle · error · KettleException

XMLInputStream.Log.TooManyNestedElements

Error message

XMLInputStream.Log.TooManyNestedElements

What it means

XMLInputStream.processEvent throws a KettleException with 'XMLInputStream.Log.TooManyNestedElements' when the XML document nesting depth exceeds PARENT_ID_ALLOCATE_SIZE - 1 during a START_ELEMENT event. The step pre-allocates the elementParentID tracking array and refuses documents nested deeper than that fixed limit.

Solutions

Flatten or pre-process the XML (XSLT or an external preprocessing step) to reduce nesting depth below the limit.
Raise the source-side limit: increase PARENT_ID_ALLOCATE_SIZE in XMLInputStream.java and rebuild the plugin (check the installed version - later PDI versions allocate more).
Use the non-streaming 'Get XML Data' step or XPath-based parsing, which does not track element levels in a fixed array.
Point the streaming reader at a shallower subtree (adjust the encoding/loop settings) so deep branches are not traversed.

Example fix

// before (plugin source)
private static final int PARENT_ID_ALLOCATE_SIZE = 500;
// after (rebuild plugin)
private static final int PARENT_ID_ALLOCATE_SIZE = 5000; // supports deeper documents

Defensive patterns

Strategy: validation

Validate before calling

// Count max nesting depth before streaming the document
int maxDepth( Node n, int d ) {
  int m = d;
  for ( Node c = n.getFirstChild(); c != null; c = c.getNextSibling() ) {
    if ( c.getNodeType() == Node.ELEMENT_NODE ) m = Math.max( m, maxDepth( c, d + 1 ) );
  }
  return m;
}
// reject if maxDepth( doc.getDocumentElement(), 1 ) > 499

Try / catch

try { trans.execute( null ); } catch ( KettleException e ) { if ( e.getMessage().contains( "TooManyNestedElements" ) ) { log.error( "Document nesting exceeds plugin limit - pre-process with XSLT" ); } else { throw e; } }

Prevention

Know the depth limit of your PDI version's XML Input Stream plugin.
Flatten deeply nested supplier XML with XSLT before streaming.
Prefer GetXMLData/XPath parsing for documents with unbounded nesting.

When it happens

Trigger: processEvent (called from getRowFromXML) on XMLStreamConstants.START_ELEMENT: data.elementLevel++ pushes elementLevel beyond PARENT_ID_ALLOCATE_SIZE - 1, i.e. the streamed document has more nesting levels than the step supports.

Common situations: Processing deeply nested XML generated by recursive schemas, machine-generated XML with excessive wrapper depth, or XML where the chosen encoding/loop path forces the reader to traverse very deep subtrees.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/16d3f51dd4519800. Report an issue: GitHub.

Appendix: source

Thrown at plugins/xml/core/src/main/java/org/pentaho/di/trans/steps/xmlinputstream/XMLInputStream.java:363

        // unknown eventType
        outputRowData[data.pos_xml_data_type_description] = eventDescription[0] + "(" + eventType + ")";
      } else {
        outputRowData[data.pos_xml_data_type_description] = eventDescription[eventType];
      }
    }
    if ( data.pos_xml_location_line != -1 ) {
      outputRowData[data.pos_xml_location_line] = new Long( e.getLocation().getLineNumber() );
    }
    if ( data.pos_xml_location_column != -1 ) {
      outputRowData[data.pos_xml_location_column] = new Long( e.getLocation().getColumnNumber() );
    }

    switch ( eventType ) {

      case XMLStreamConstants.START_ELEMENT:
        data.elementLevel++;
        if ( data.elementLevel > PARENT_ID_ALLOCATE_SIZE - 1 ) {
          throw new KettleException(
            BaseMessages.getString( PKG, "XMLInputStream.Log.TooManyNestedElements", PARENT_ID_ALLOCATE_SIZE ) );
        }
        if ( data.elementParentID[data.elementLevel] == null ) {
          data.elementParentID[data.elementLevel] = data.elementID;
        }
        data.elementID++;
        data.elementLevelID[data.elementLevel] = data.elementID;

        String xml_data_name;
        if ( meta.isEnableNamespaces() ) {
          String prefix = e.asStartElement().getName().getPrefix();
          if ( Utils.isEmpty( prefix ) ) {
            xml_data_name = e.asStartElement().getName().getLocalPart();
          } else { // add namespace prefix:
            xml_data_name = prefix + ":" + e.asStartElement().getName().getLocalPart();
          }
        } else {
          xml_data_name = e.asStartElement().getName().getLocalPart();

View on GitHub (pinned to f3058517a1)