{"record":{"id":"477d41b407a9111d","repo":"pentaho/pentaho-kettle","slug":"s3csvinput-exception-errorpreparingparallelrun","errorCode":"S3CsvInput.Exception.ErrorPreparingParallelRun","errorMessage":"S3CsvInput.Exception.ErrorPreparingParallelRun","messagePattern":"S3CsvInput\\.Exception\\.ErrorPreparingParallelRun","errorType":"error_code","errorClass":"KettleException","httpStatus":null,"severity":"error","filePath":"plugins/s3csvinput/core/src/main/java/org/pentaho/di/trans/steps/s3csvinput/S3CsvInput.java","lineNumber":187,"sourceCode":"          data.startFilenr = i;\n\n\n          // How many bytes do we skip in that first file?\n          //\n          if ( data.startPosition == 0 ) {\n            data.bytesToSkipInFirstFile = 0L;\n          } else {\n            data.bytesToSkipInFirstFile = data.startPosition - totalFileSize;\n          }\n\n          break;\n        }\n        totalFileSize += size;\n      }\n\n      logBasic( Messages.getString( \"S3CsvInput.Log.ParallelFileNrAndPositionFeedback\", data.filenames[data.filenr], Long.toString( data.fileSizes.get( data.filenr ) ), Long.toString( data.bytesToSkipInFirstFile ), Long.toString( data.blockToRead ) ) );\n    } catch ( Exception e ) {\n      throw new KettleException( Messages.getString( \"S3CsvInput.Exception.ErrorPreparingParallelRun\" ), e );\n    }\n  }\n\n  private void getFilenamesFromPreviousSteps() throws KettleException {\n    List<String> filenames = new ArrayList<String>();\n    boolean firstRow = true;\n    int index = -1;\n    Object[] row = getRow();\n    while ( row != null ) {\n\n      if ( firstRow ) {\n        firstRow = false;\n\n        // Get the filename field index...\n        //\n        String filenameField = environmentSubstitute( meta.getFilenameField() );\n        index = getInputRowMeta().indexOfValue( filenameField );\n        if ( index < 0 ) {","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/pentaho/pentaho-kettle/blob/f3058517a153da500bf4551f46d79b91bf8ec552/plugins/s3csvinput/core/src/main/java/org/pentaho/di/trans/steps/s3csvinput/S3CsvInput.java#L169-L205","documentation":"S3CsvInput.prepareToRunInParallel computes file sizes and block offsets so multiple step copies can split a large S3 CSV file. Any exception during that setup (S3 metadata fetch failures, null sizes, bad filenames) is rethrown as a KettleException with message key S3CsvInput.Exception.ErrorPreparingParallelRun.","triggerScenarios":"Running the S3 CSV Input step in 'parallel run' mode when data.filenames lookup, S3 object metadata retrieval (size), or the size/block calculation throws — e.g. object missing, credentials invalid, or filenames list empty/null.","commonSituations":"Misconfigured parallel/clustered transformation with a non-existent S3 key; expired S3 credentials; running the step with 0 input files; region mismatch making head-object calls fail.","solutions":["Verify each file in the step's file list exists in the bucket and credentials can read it (aws s3 ls / head-object)","If filenames come from a previous step, confirm rows actually arrive before this step executes","Check the wrapped cause (e.getCause()) in the KettleException log for the real failure (NoSuchBucket, 403, etc.)","Disable or correctly configure parallel execution if the step copy receives no files"],"exampleFix":"// before\n// filenames resolved from previous step may be empty\nprepareToRunInParallel( layouts );\n// after\nif ( data.filenames == null || data.filenames.length == 0 ) {\n  logBasic( \"No files to process; skipping parallel preparation\" );\n  return;\n}\nprepareToRunInParallel( layouts );","handlingStrategy":"validation","validationCode":"for ( String f : filenames ) {\n  if ( f == null || f.isEmpty() ) throw new IllegalArgumentException( \"empty filename\" );\n  if ( !s3.doesObjectExist( bucket, key( f ) ) ) throw new IllegalStateException( \"missing: \" + f );\n}","typeGuard":null,"tryCatchPattern":"try {\n  prepareToRunInParallel( layouts );\n} catch ( KettleException e ) {\n  logError( \"parallel prep failed\", e.getCause() );\n  throw e;\n}","preventionTips":["Verify all configured S3 files exist and are readable before running","Ensure upstream filename-producing steps emit rows before this step","Avoid enabling parallel mode with an empty file list","Check credentials/region for head-object access"],"tags":["s3","kettle","csv","parallel-execution"],"backgroundTag":"api-request-failed","analyzedSha":"f3058517a153da500bf4551f46d79b91bf8ec552","analyzedAt":"2026-09-13T14:04:16.340Z","contentChangedAt":"2026-09-13T14:04:16.340Z","schemaVersion":2},"datasetVersion":"2026-09-20T23:17:15.980Z"}