apache/hadoop · error · IllegalStateException

Incorrect state to start a Meta Block: {state}

Error message

Incorrect state to start a Meta Block: {state}

What it means

Thrown by TFile.Writer.prepareMetaBlock(String name, String compressName) when the writer is not in the READY state, i.e. when a key or value append stream is still open. Meta blocks (named auxiliary blocks such as TFile's own metadata) can only be written between records, not in the middle of one. IllegalStateException carries the offending state name.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java:611

     * active. No more key-value insertion is allowed after a meta data block
     * has been added to TFile.
     * 
     * @param name
     *          Name of the meta block.
     * @param compressName
     *          Name of the compression algorithm to be used. Must be one of the
     *          strings returned by
     *          {@link TFile#getSupportedCompressionAlgorithms()}.
     * @return A DataOutputStream that can be used to write Meta Block data.
     *         Closing the stream would signal the ending of the block.
     * @throws IOException raised on errors performing I/O.
     * @throws MetaBlockAlreadyExists
     *           the Meta Block with the same name already exists.
     */
    public DataOutputStream prepareMetaBlock(String name, String compressName)
        throws IOException, MetaBlockAlreadyExists {
      if (state != State.READY) {
        throw new IllegalStateException(
            "Incorrect state to start a Meta Block: " + state.name());
      }

      finishDataBlock(true);
      DataOutputStream outputStream =
          writerBCF.prepareMetaBlock(name, compressName);
      return outputStream;
    }

    /**
     * Obtain an output stream for creating a meta block. This function may not
     * be called when there is a key append stream or value append stream
     * active. No more key-value insertion is allowed after a meta data block
     * has been added to TFile. Data will be compressed using the default
     * compressor as defined in Writer's constructor.
     * 
     * @param name
     *          Name of the meta block.

View on GitHub (pinned to 2add963021)

Solutions

1. Finish the in-flight record (close key and value streams) before calling prepareMetaBlock
2. Structure writes as: all key-value appends, then meta blocks, then close(); this matches the intended TFile layout
3. Close meta block streams promptly too, so the writer returns to READY for the next data record

Example fix

// before
DataOutputStream k = writer.prepareAppendKey(-1);
k.write(key);
DataOutputStream meta = writer.prepareMetaBlock("stats", "gz"); // IllegalStateException

// after
try (DataOutputStream k = writer.prepareAppendKey(-1)) {
  k.write(key);
}
DataOutputStream meta = writer.prepareMetaBlock("stats", "gz"); // OK: READY

// Only write meta blocks between records: finish data first, then meta
for (Record r : records) {
  writer.append(r.key(), r.value());
}
try (DataOutputStream meta = writer.prepareMetaBlock("stats", TFile.COMPRESSION_GZ)) {
  meta.write(statsBytes);
}
writer.close();

catch (IllegalStateException e) {
  // a key/value stream was open; close it, then retry the meta block on a consistent writer only if no bytes were mid-record
}

When it happens

Trigger: Calling prepareMetaBlock(name, compressName) while a stream from prepareAppendKey() or prepareAppendValue() is unclosed; interleaving meta block writes into the middle of a key-value insertion sequence.

Common situations: Code that writes a meta block on a timer or callback without coordinating with the record-writing loop; exception in the record loop leaving a stream open, followed by a finally-block meta write; refactored writers that emit meta blocks after N records but forget to finish the current record first.

Related errors

• Cannot close TFile in the middle of key-value insertion.
• Incorrect state to start a new key: {state}
• Incorrect state to start a new value: {state}
• Incorrect key length: expected={expectedLength} actual={len}
• Keys are not added in sorted order

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a7c3d54500539b04. Report an issue: GitHub.