pentaho/pentaho-kettle · critical · KettleFileException

GroupBy.Exception.UnableToCreateTemporaryFile

GroupBy.Exception.UnableToCreateTemporaryFile

Error message

GroupBy.Exception.UnableToCreateTemporaryFile

What it means

When GroupBy's row buffer exceeds its size and sorting is enabled, addToBuffer spills the oldest rows to a temporary file. File.createTempFile / FileOutputStream creation failures are wrapped in a KettleFileException with this localized message. It means the step could not create or open its spill temp file, so grouping cannot continue.

Solutions

1. Free disk space in the temp directory (or point java.io.tmpdir / Kettle 'java.io.tmpdir' option to a volume with room).
2. Verify the configured temp path exists and is writable by the user running Pentaho/Kettle.
3. In Spoon, set the transformation's temp directory or start Kettle with -Djava.io.tmpdir=/path/writable.
4. Increase the sort/group buffer size so less spilling occurs, or reduce input size.
5. In containers, mount a writable tmp volume instead of relying on a read-only overlay.

Example fix

// before: default tmpdir full or read-only
java -jar pentaho-kettle.jar
// after
java -Djava.io.tmpdir=/data/kettle-tmp -jar pentaho-kettle.jar
# with: mkdir -p /data/kettle-tmp && chmod 770 /data/kettle-tmp

String tmp = System.getProperty( "java.io.tmpdir" );
File dir = new File( tmp );
if ( !dir.isDirectory() || !dir.canWrite() || dir.getUsableSpace() < 512L * 1024 * 1024 ) {
  throw new IllegalStateException( "Temp dir unusable or low on space: " + tmp );
}

try {
  trans.execute( null );
} catch ( KettleFileException e ) {
  if ( e.getMessage().contains( "UnableToCreateTemporaryFile" ) ) {
    logError( "Temp dir full/unwritable - free space or set -Djava.io.tmpdir" );
  }
}

When it happens

Trigger: addToBuffer calls File.createTempFile(prefix, ".tmp", new File(pathToTmp)) and an IOException occurs: java.io.tmpdir (or the configured temp directory) doesn't exist, is read-only, is full, or the OS denies file creation for the running user.

Common situations: Disk full on /tmp after long-running transformations; tmp dir cleaned by a cron job while the transformation runs; Kettle's internal.tmpdir setting pointing to a nonexistent path; container running with a read-only filesystem or tiny tmpfs.

Understand the failure class

Background: "open() failed", "failed to open file", "cannot create file" — what a file open error means and how to fix it — this error's family across 42 libraries.

Related errors

• Error opening new file
• Error processing temp-file!
• ex.getMessage()
• GPG.ErrorCreatingTempFile
• GPG.ErrorWritingTempFile

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/5a6b2aabb8c98724. Report an issue: GitHub.

Appendix: source

Thrown at engine/src/main/java/org/pentaho/di/trans/steps/groupby/GroupBy.java:796

  }

  // Method is defined as package-protected in order to be accessible by unit tests
  void addToBuffer( Object[] row ) throws KettleFileException {
    data.bufferList.add( row );
    if ( data.bufferList.size() > 5000 && data.rowsOnFile == 0 ) {
      String pathToTmp = environmentSubstitute( getMeta().getDirectory() );
      try {
        File ioFile = new File( pathToTmp );
        if ( !ioFile.exists() ) {
          // try to resolve as Apache VFS file
          pathToTmp = retrieveVfsPath( pathToTmp );
        }
        data.tempFile = File.createTempFile( getMeta().getPrefix(), ".tmp", new File( pathToTmp ) );
        data.fosToTempFile = new FileOutputStream( data.tempFile );
        data.dosToTempFile = new DataOutputStream( data.fosToTempFile );
        data.firstRead = true;
      } catch ( IOException e ) {
        throw new KettleFileException( BaseMessages.getString( PKG, "GroupBy.Exception.UnableToCreateTemporaryFile" ),
            e );
      }
      // OK, save the oldest rows to disk!
      Object[] oldest = data.bufferList.get( 0 );
      data.inputRowMeta.writeData( data.dosToTempFile, oldest );
      data.bufferList.remove( 0 );
      data.rowsOnFile++;
    }
  }

  // Method is defined as public in order to be accessible by unit tests
  public String retrieveVfsPath( String pathToTmp ) throws KettleFileException {
    FileObject vfsFile = KettleVFS.getInstance( getTransMeta().getBowl() ).getFileObject( pathToTmp );
    String path = vfsFile.getName().getPath();
    return path;
  }

  private Object[] getRowFromBuffer() throws KettleFileException {

View on GitHub (pinned to f3058517a1)