{"record":{"id":"113bc14c871f5a06","repo":"apache/hadoop","slug":"output-format-not-set-for-lazyoutputformat","errorCode":null,"errorMessage":"Output Format not set for LazyOutputFormat","messagePattern":"Output Format not set for LazyOutputFormat","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/LazyOutputFormat.java","lineNumber":64,"sourceCode":"   * Set the underlying output format for LazyOutputFormat.\n   * @param job the {@link Job} to modify\n   * @param theClass the underlying class\n   */\n  @SuppressWarnings(\"unchecked\")\n  public static void  setOutputFormatClass(Job job, \n                                     Class<? extends OutputFormat> theClass) {\n      job.setOutputFormatClass(LazyOutputFormat.class);\n      job.getConfiguration().setClass(OUTPUT_FORMAT, \n          theClass, OutputFormat.class);\n  }\n\n  @SuppressWarnings(\"unchecked\")\n  private void getBaseOutputFormat(Configuration conf) \n  throws IOException {\n    baseOut =  ((OutputFormat<K, V>) ReflectionUtils.newInstance(\n      conf.getClass(OUTPUT_FORMAT, null), conf));\n    if (baseOut == null) {\n      throw new IOException(\"Output Format not set for LazyOutputFormat\");\n    }\n  }\n\n  @Override\n  public RecordWriter<K, V> getRecordWriter(TaskAttemptContext context)\n  throws IOException, InterruptedException {\n    if (baseOut == null) {\n      getBaseOutputFormat(context.getConfiguration());\n    }\n    return new LazyRecordWriter<K, V>(baseOut, context);\n  }\n  \n  @Override\n  public void checkOutputSpecs(JobContext context) \n  throws IOException, InterruptedException {\n    if (baseOut == null) {\n      getBaseOutputFormat(context.getConfiguration());\n    }","sourceCodeStart":46,"sourceCodeEnd":82,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/LazyOutputFormat.java#L46-L82","documentation":"Thrown as IOException from LazyOutputFormat.getBaseOutputFormat (LazyOutputFormat.java:64). LazyOutputFormat only creates output files when the first record is written; it needs the real OutputFormat class in mapreduce.output.lazyoutputformat.outputformat. If that key is absent, ReflectionUtils.newInstance(null, conf) yields null and this IOException is raised when getRecordWriter/checkOutputSpecs/getOutputCommitter runs.","triggerScenarios":"Setting job.setOutputFormatClass(LazyOutputFormat.class) manually (or via conf) instead of calling LazyOutputFormat.setOutputFormatClass(job, ActualFormat.class); the latter both registers LazyOutputFormat AND stores the delegate under OUTPUT_FORMAT. Also losing the key when a Configuration is rebuilt/filtered (e.g. whitelisting config keys for tasks) before task start.","commonSituations":"Recipes copied from docs that show setOutputFormatClass(LazyOutputFormat.class) plus a hand-written conf key with a typo; security-filtered configurations (Configuration.loadResources / mapred-site trimming) dropping the key; using LazyOutputFormat together with MultipleOutputs but wiring it manually.","solutions":["Configure via the API: LazyOutputFormat.setOutputFormatClass(job, TextOutputFormat.class) — never set the LazyOutputFormat class by hand","If configuring raw properties, set both: mapreduce.job.outputformat.class=org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat AND mapreduce.output.lazyoutputformat.outputformat=<delegate FQCN>","Check for config whitelisting/scrubbing in your framework that strips the lazyoutputformat key before tasks launch"],"exampleFix":"// before: wrapper registered without its delegate\njob.setOutputFormatClass(LazyOutputFormat.class);\n\n// after: helper sets wrapper + delegate key together\nLazyOutputFormat.setOutputFormatClass(job, TextOutputFormat.class);","handlingStrategy":"validation","validationCode":"// before submit: both keys must be present\nConfiguration conf = job.getConfiguration();\nif (job.getOutputFormatClass().equals(LazyOutputFormat.class)\n    && conf.get(\"mapreduce.output.lazyoutputformat.outputformat\") == null) {\n  throw new IllegalStateException(\"LazyOutputFormat configured without its delegate\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use LazyOutputFormat.setOutputFormatClass(job, Actual.class) only","Do not hand-set mapreduce.job.outputformat.class to LazyOutputFormat","Watch for config whitelists stripping lazyoutputformat keys"],"tags":["hadoop","mapreduce","lazy-output-format","output-format","configuration"],"backgroundTag":"outputformat-delegate-not-set","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}