apache/iceberg · error · RuntimeIOException

Failed to decode value as UTF-8: %s

Error message

Failed to decode value as UTF-8: %s

What it means

Conversions.internalFromByteBuffer deserializes binary values per type. For STRING it UTF-8 decodes the buffer; invalid byte sequences (not valid UTF-8) cause CharacterCodingException, wrapped in RuntimeIOException with this message. This typically indicates corrupted or wrongly-encoded data in metadata/statistics.

Source

Thrown at api/src/main/java/org/apache/iceberg/types/Conversions.java:192

      case TIMESTAMP_NANO:
        if (tmp.remaining() < 8) {
          // type was later promoted to long
          return (long) tmp.getInt();
        }
        return tmp.getLong();
      case FLOAT:
        return tmp.getFloat();
      case DOUBLE:
        if (tmp.remaining() < 8) {
          // type was later promoted to long
          return (double) tmp.getFloat();
        }
        return tmp.getDouble();
      case STRING:
        try {
          return DECODER.get().decode(tmp);
        } catch (CharacterCodingException e) {
          throw new RuntimeIOException(e, "Failed to decode value as UTF-8: %s", buffer);
        }
      case UUID:
        return UUIDUtil.convert(tmp);
      case FIXED:
      case BINARY:
        return tmp;
      case DECIMAL:
        Types.DecimalType decimal = (Types.DecimalType) type;
        byte[] unscaledBytes = new byte[buffer.remaining()];
        tmp.get(unscaledBytes);
        return new BigDecimal(new BigInteger(unscaledBytes), decimal.scale());
      case VARIANT:
        return Variant.from(tmp);
      case GEOMETRY:
      case GEOGRAPHY:
        return GeospatialBound.fromByteBuffer(tmp);
      case UNKNOWN:
        // underlying type not known

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Fix the writer to UTF-8 encode strings per the Iceberg spec and rewrite the data
2. Decode leniently with a CharsetDecoder using CodingErrorAction.REPLACE on the raw bytes yourself
3. Catch RuntimeIOException and treat the value as null/unknown rather than failing the read

Example fix

// before
String s = (String) Conversions.fromByteBuffer(Types.StringType.get(), badBuffer);
// after
String s = new CharsetDecoder(StandardCharsets.UTF_8.newDecoder()
    .onMalformedInput(CodingErrorAction.REPLACE)).decode(badBuffer.duplicate());

CharsetDecoder dec = StandardCharsets.UTF_8.newDecoder().onMalformedInput(CodingErrorAction.REPORT);
try { dec.decode(buffer.duplicate()); } catch (CharacterCodingException e) { /* invalid UTF-8 */ }

try { v = Conversions.fromByteBuffer(type, buf); } catch (RuntimeIOException e) { v = null; /* treat as unknown value */ }

When it happens

Trigger: Calling Conversions.fromByteBuffer(stringType, buffer) where the buffer bytes are not valid UTF-8 (e.g. Latin-1 encoded bytes, truncated multi-byte sequence).

Common situations: Reading partition/statistics metadata written by non-Iceberg tools; buffers extracted from files with encoding corruption; manual byte manipulation producing invalid UTF-8.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Failed to encode value as UTF-8: %s
• Cannot deserialize type: + type
• Can't retrieve values from an empty struct
• Can't modify an empty struct
• %s doesn't implement cleanupLevel

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/cd1658db59a20e54. Report an issue: GitHub.