{"record":{"id":"9277dcae17c418a3","repo":"apache/hadoop","slug":"writablevaluebytes-doesn-t-support-record-compress","errorCode":null,"errorMessage":"WritableValueBytes doesn't support RECORD compression","messagePattern":"WritableValueBytes doesn't support RECORD compression","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/SequenceFileAsBinaryOutputFormat.java","lineNumber":73,"sourceCode":"      this.value = null;\n    }\n    \n    public WritableValueBytes(BytesWritable value) {\n      this.value = value;\n    }\n\n    public void reset(BytesWritable value) {\n      this.value = value;\n    }\n\n    public void writeUncompressedBytes(DataOutputStream outStream)\n        throws IOException {\n      outStream.write(value.getBytes(), 0, value.getLength());\n    }\n\n    public void writeCompressedBytes(DataOutputStream outStream)\n        throws IllegalArgumentException, IOException {\n      throw new UnsupportedOperationException(\n        \"WritableValueBytes doesn't support RECORD compression\"); \n    }\n    \n    public int getSize(){\n      return value.getLength();\n    }\n  }\n\n  /**\n   * Set the key class for the {@link SequenceFile}\n   * <p>This allows the user to specify the key class to be different \n   * from the actual class ({@link BytesWritable}) used for writing </p>\n   * \n   * @param job the {@link Job} to modify\n   * @param theClass the SequenceFile output key class.\n   */\n  static public void setSequenceFileOutputKeyClass(Job job, \n      Class<?> theClass) {","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/SequenceFileAsBinaryOutputFormat.java#L55-L91","documentation":"SequenceFileAsBinaryOutputFormat writes raw BytesWritable keys/values through an adapter (WritableValueBytes). SequenceFile record compression needs per-value compressed writes, which the adapter cannot do, so writeCompressedBytes() throws UnsupportedOperationException. Normally this configuration is rejected up front by checkOutputSpecs (InvalidJobConfException, error 4345); this runtime exception only appears when a writer is driven with RECORD compression through a path that bypassed that check (custom record-writer wiring, tests, direct SequenceFile.Writer-style usage).","triggerScenarios":"The record writer reaching writeCompressedBytes() because output compression is enabled with CompressionType.RECORD - i.e. mapreduce.output.fileoutputformat.compress=true and mapreduce.output.fileoutputformat.compression.type=RECORD while writing via SequenceFileAsBinaryOutputFormat's BinarySequenceFileRecordWriter outside the normal checked submit path.","commonSituations":"Test code constructing the RecordWriter directly; a job where compression settings were changed after checkOutputSpecs ran; shared job-conf templates that default compression type to RECORD (the SequenceFile default).","solutions":["Switch compression type to BLOCK: conf.set(\"mapreduce.output.fileoutputformat.compression.type\", \"BLOCK\").","Or disable compression for this output: FileOutputFormat.setCompressOutput(job, false).","Let the framework call checkOutputSpecs (it fails fast at submit) instead of bypassing job submission."],"exampleFix":"// before\nconf.setBoolean(\"mapreduce.output.fileoutputformat.compress\", true);\nconf.set(\"mapreduce.output.fileoutputformat.compression.type\", \"RECORD\");\n\n// after\nconf.setBoolean(\"mapreduce.output.fileoutputformat.compress\", true);\nconf.set(\"mapreduce.output.fileoutputformat.compression.type\", \"BLOCK\");","handlingStrategy":"validation","validationCode":"// guard before obtaining/using the record writer\nif (FileOutputFormat.getCompressOutput(job)\n    && FileOutputFormat.getOutputCompressionType(job) == CompressionType.RECORD) {\n  throw new InvalidJobConfException(\n      \"SequenceFileAsBinaryOutputFormat supports only NONE or BLOCK compression\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always pair setCompressOutput(true) with an explicit setOutputCompressionType(BLOCK) for binary sequence output.","Prefer the framework's checkOutputSpecs path so misconfiguration fails at submit, not at write time.","Remember RECORD is the SequenceFile default type - enabling compression alone is not safe."],"tags":["hadoop","sequencefile","compression","mapreduce","output-format"],"backgroundTag":"unsupported-compression-type","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}