pentaho/pentaho-kettle · error · KettleFileException

GroupBy.Exception.UnableToReadBackRowFromTemporaryFile

GroupBy.Exception.UnableToReadBackRowFromTemporaryFile

Error message

GroupBy.Exception.UnableToReadBackRowFromTemporaryFile

What it means

GroupBy buffered its rows to a temporary file when the group was too large for memory. This error wraps an IOException thrown while opening a FileInputStream/DataInputStream on that temp file to read rows back during getRowFromBuffer. It means the temp file that was just written is now unreadable.

Solutions

1. Check that the java.io.tmpdir / Kettle temp directory exists, is writable, and is not purged while the transformation runs
2. Ensure the transformation isn't run in multiple copies/clustered mode such that copies overwrite each other's temp files
3. Verify sufficient disk space for the sorted group data
4. Call init() so data.tempFile is created before processRow; never construct the step data manually
5. Upgrade/patch: ensure closeOutput resets firstRead=true so a second pass re-opens the file cleanly

Example fix

// before
try {
  data.fisToTmpFile = new FileInputStream( data.tempFile );
// after
try {
  if ( data.tempFile == null || !data.tempFile.exists() ) {
    throw new KettleFileException( "Temp file missing: " + data.tempFile );
  }
  data.fisToTmpFile = new FileInputStream( data.tempFile );

// before running the transformation
File tmp = new File( System.getProperty( "java.io.tmpdir" ) );
if ( !tmp.isDirectory() || !tmp.canWrite() ) throw new IllegalStateException( "Temp dir not writable" );
if ( tmp.getUsableSpace() < 100L * 1024 * 1024 ) throw new IllegalStateException( "Low temp disk space" );

try {
  step.processRow();
} catch ( KettleFileException e ) {
  if ( e.getMessage().contains( "UnableToReadBackRowFromTemporaryFile" ) ) {
    log.warn( "GroupBy temp file unreadable; check {} for cleanup jobs", System.getProperty( "java.io.tmpdir" ), e );
  } else { throw e; }
}

When it happens

Trigger: getRowFromBuffer opens data.tempFile via new FileInputStream(data.tempFile) after firstRead was set false by the writer; the file was deleted, the path is invalid, disk/permissions changed, or the stream was already consumed and reset failed.

Common situations: Temp directory cleaned by another process or tmpwatch mid-transformation; multiple transformation copies sharing data.tempFile; disk full or mount removed; running clustered where temp files aren't shared across nodes.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• GroupBy.Exception.UnableToCloseInputStream
• Exception reading line using NIO:
• GroupByMeta.Exception.UnexpectedErrorInReadingStepInfoFromRepository
• GroupByMeta.Exception.UnableToLoadStepInfoFromXML
• JsonOutput.Error.OpenNewFile

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/0a9c17ab94a76c05. Report an issue: GitHub.

Appendix: source

Thrown at engine/src/main/java/org/pentaho/di/trans/steps/groupby/GroupBy.java:823

  }

  // Method is defined as public in order to be accessible by unit tests
  public String retrieveVfsPath( String pathToTmp ) throws KettleFileException {
    FileObject vfsFile = KettleVFS.getInstance( getTransMeta().getBowl() ).getFileObject( pathToTmp );
    String path = vfsFile.getName().getPath();
    return path;
  }

  private Object[] getRowFromBuffer() throws KettleFileException {
    if ( data.rowsOnFile > 0 ) {
      if ( data.firstRead ) {
        // Open the inputstream first...
        try {
          data.fisToTmpFile = new FileInputStream( data.tempFile );
          data.disToTmpFile = new DataInputStream( data.fisToTmpFile );
          data.firstRead = false;
        } catch ( IOException e ) {
          throw new KettleFileException( BaseMessages.getString(
              PKG, "GroupBy.Exception.UnableToReadBackRowFromTemporaryFile" ), e );
        }
      }

      // Read one row from the file!
      Object[] row;
      try {
        row = data.inputRowMeta.readData( data.disToTmpFile );
      } catch ( SocketTimeoutException e ) {
        throw new KettleFileException( e ); // Shouldn't happen on files
      }
      data.rowsOnFile--;

      return row;
    } else {
      if ( data.bufferList.size() > 0 ) {
        Object[] row = data.bufferList.get( 0 );
        data.bufferList.remove( 0 );

View on GitHub (pinned to f3058517a1)