{"record":{"id":"009c1dbb7526ea5b","repo":"pentaho/pentaho-kettle","slug":"error-defining-data-file","errorCode":null,"errorMessage":"Error defining data file!","messagePattern":"Error defining data file!","errorType":"exception","errorClass":"KettleException","httpStatus":null,"severity":"error","filePath":"plugins/terafast-bulk-loader/impl/src/main/java/org/pentaho/di/trans/steps/terafastbulkloader/TeraFast.java","lineNumber":398,"sourceCode":"  /**\n   * Invoke loading with loading commands.\n   *\n   * @throws KettleException\n   *           ...\n   */\n  private void invokeLoadingCommand() throws KettleException {\n    final FastloadControlBuilder builder = new FastloadControlBuilder();\n    builder.setSessions( this.meta.getSessions().getValue() );\n    builder.setErrorLimit( this.meta.getErrorLimit().getValue() );\n    builder.logon( this.meta.getDbMeta().getHostname(), this.meta.getDbMeta().getUsername(), this.meta\n      .getDbMeta().getPassword() );\n    builder.setRecordFormat( FastloadControlBuilder.RECORD_VARTEXT );\n    try {\n      builder.define(\n        this.meta.getRequiredFields( this.getTransMeta() ), meta.getTableFieldList(), resolveFileName( this.meta\n          .getDataFile().getValue() ) );\n    } catch ( Exception ex ) {\n      throw new KettleException( \"Error defining data file!\", ex );\n    }\n    builder.show();\n    builder.beginLoading( this.meta.getDbMeta().getPreferredSchemaName(), this.meta.getTargetTable().getValue() );\n\n    builder.insert( this.meta.getRequiredFields( this.getTransMeta() ), meta.getTableFieldList(), this.meta\n      .getTargetTable().getValue() );\n    builder.endLoading();\n    builder.logoff();\n    final String control = builder.toString();\n    try {\n      logDetailed( \"Control file: \" + control );\n      IOUtils.write( control, this.fastload );\n    } catch ( IOException e ) {\n      throw new KettleException( \"Error while execution control command [controlCommand=\" + control + \"]\", e );\n    } finally {\n      IOUtils.closeQuietly( this.fastload );\n    }\n  }","sourceCodeStart":380,"sourceCodeEnd":416,"githubUrl":"https://github.com/pentaho/pentaho-kettle/blob/f3058517a153da500bf4551f46d79b91bf8ec552/plugins/terafast-bulk-loader/impl/src/main/java/org/pentaho/di/trans/steps/terafastbulkloader/TeraFast.java#L380-L416","documentation":"TeraFast (Teradata fastload bulk loader step) wraps any failure while asking the FastloadControlBuilder to emit the fastload DEFINE directive for the data file. The builder.define() call maps the incoming row fields to the target table columns and the local data file; any exception there (bad field metadata, missing/unresolvable data file name, builder state issues) is rethrown as a KettleException.","triggerScenarios":"invokeLoadingCommand() calls builder.define(meta.getRequiredFields(transMeta), meta.getTableFieldList(), resolveFileName(meta.getDataFile().getValue())) and it throws — e.g. requiredFields lookup returned metadata that does not match the tableFieldList, the data filename cannot be resolved, or the builder is in an invalid state.","commonSituations":"Target table was altered after the step was configured so required fields no longer align; data file path contains variables that do not resolve; fastload utility misconfigured; step metadata serialized from an older PDI version.","solutions":["Verify the step's field mapping (required fields vs table field list) matches the actual target table in Teradata","Check that the data file path resolves to a writable, valid location (environment variables resolve)","Inspect the wrapped cause (ex) in the KettleException stack trace for the root error from FastloadControlBuilder","Reopen and re-save the step dialog to regenerate metadata after table changes"],"exampleFix":"// before\nbuilder.define( meta.getRequiredFields( transMeta ), meta.getTableFieldList(), resolveFileName( meta.getDataFile().getValue() ) );\n// after\nString dataFile = environmentSubstitute( meta.getDataFile().getValue() );\nif ( dataFile == null || dataFile.trim().isEmpty() ) {\n  throw new KettleException( \"Data file path is not set\" );\n}\nbuilder.define( meta.getRequiredFields( transMeta ), meta.getTableFieldList(), dataFile );","handlingStrategy":"try-catch","validationCode":"// before executing the transformation\nTeraFastMeta meta = ...;\nif ( meta.getDbMeta() == null ) throw new KettleException( \"No DB connection set\" );\nif ( meta.getDataFile() == null || meta.getDataFile().getValue() == null ) throw new KettleException( \"No data file set\" );\nRowMetaInterface req = meta.getRequiredFields( transMeta );\nif ( req == null || req.size() == 0 ) throw new KettleException( \"No required fields resolved for target table\" );","typeGuard":"boolean isUsable( TeraFastMeta m ) {\n  return m != null && m.getDbMeta() != null && m.getDataFile() != null\n    && m.getDataFile().getValue() != null && m.getTargetTable() != null;\n}","tryCatchPattern":"try {\n  step.run();\n} catch ( KettleException e ) {\n  if ( e.getMessage().contains( \"Error defining data file\" ) ) {\n    logError( \"Fastload DEFINE failed: \" + e.getCause(), e ); // inspect cause\n  } else { throw e; }\n}","preventionTips":["Keep target table metadata in sync with the step's field mapping after any ALTER TABLE","Always set a concrete (or resolvable-variable) data file path","Test the step with 'Preview' before running full loads","Log the generated control file (detailed logging) to diagnose builder failures"],"tags":["teradata","fastload","kettle","etl","bulk-loader"],"backgroundTag":"database-write-failed","analyzedSha":"f3058517a153da500bf4551f46d79b91bf8ec552","analyzedAt":"2026-09-13T14:04:16.340Z","contentChangedAt":"2026-09-13T14:04:16.340Z","schemaVersion":2},"datasetVersion":"2026-09-20T23:17:15.980Z"}