apache/beam · error · IllegalStateException

Unable to read data

Error message

Unable to read data: {}

What it means

HadoopInputFormatReader.getCurrent() calls RecordReader.getCurrentKey()/getCurrentValue() and applies the translation functions; an IOException or InterruptedException during that is logged and rethrown as this IllegalStateException, since the reader is now in an unrecoverable state for the current record.

Solutions

1. Validate/repair the input data (run hdfs fsck, exclude corrupt files)
2. Wrap translation functions to catch their own exceptions and map bad records instead of failing
3. Check Hadoop logs for the chained cause to identify the underlying I/O problem

Example fix

// before
new ValueTranslateFn<MyIn, MyOut>() { public MyOut translate(MyIn v) { return riskyParse(v); } }
// after
new ValueTranslateFn<MyIn, MyOut>() { public MyOut translate(MyIn v) { try { return riskyParse(v); } catch (Exception e) { return null; /* or default */ } } }

try {
  KV<K,V> kv = reader.getCurrent();
} catch (IllegalStateException e) {
  LOG.error("Failed to fetch current record", e);
  metrics.counter("bad-records").inc();
  // skip or fail per pipeline policy
}

When it happens

Trigger: RecordReader.getCurrentKey()/getCurrentValue() throws (e.g., corrupt file block, decoder failure) or is interrupted while the runner calls getCurrent() after advance() returned true.

Common situations: Corrupt HDFS blocks or truncated input files; key/value translation function throwing internally; thread interruption mid-read.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Null RecordReader object returned by
• A PValue contained in
• Attempting to emit an element outside of a @ProcessElement…
• Cannot call getFromRowFunction when there is no schema
• Cannot call getSchema when there is no schema

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/eb6e8b76e86229b1. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:983

      @Override
      public KV<K, V> getCurrent() {
        K key;
        V value;
        try {
          // Transform key if translation function is provided.
          key =
              transformKeyOrValue(
                  recordReader.getCurrentKey(), keyTranslationFunction, keyCoder, skipKeyClone);
          // Transform value if translation function is provided.
          value =
              transformKeyOrValue(
                  recordReader.getCurrentValue(),
                  valueTranslationFunction,
                  valueCoder,
                  skipValueClone);
        } catch (IOException | InterruptedException e) {
          LOG.error("Unable to read data: ", e);
          throw new IllegalStateException("Unable to read data: " + "{}", e);
        }
        return KV.of(key, value);
      }

      /** Returns the serialized output of transformed key or value object. */
      @SuppressWarnings("unchecked")
      private <T, T3> T3 transformKeyOrValue(
          T input,
          @Nullable SimpleFunction<T, T3> simpleFunction,
          Coder<T3> coder,
          boolean skipClone)
          throws CoderException, ClassCastException {
        T3 output;
        if (null != simpleFunction) {
          output = simpleFunction.apply(input);
        } else {
          output = (T3) input;
        }

View on GitHub (pinned to 12126d8942)