prestodb/presto · error · IllegalArgumentException

RCBinary encoder does not support empty VARBINARY values (HI

Error message

RCBinary encoder does not support empty VARBINARY values (HIVE-2483). Use ORC or Parquet format instead.

What it means

The RCBinary VARBINARY encoder writes raw bytes with no length prefix trick for empty values; due to Hive bug HIVE-2483 an empty VARBINARY would corrupt the stream, so the encoder explicitly refuses. This is a writer-side hard failure, thrown as IllegalArgumentException.

Source

Thrown at presto-rcfile/src/main/java/com/facebook/presto/rcfile/binary/BinaryEncoding.java:46

public class BinaryEncoding
        implements BinaryColumnEncoding
{
    private final Type type;

    public BinaryEncoding(Type type)
    {
        this.type = type;
    }

    @Override
    public void encodeColumn(Block block, SliceOutput output, EncodeOutput encodeOutput)
    {
        for (int position = 0; position < block.getPositionCount(); position++) {
            if (!block.isNull(position)) {
                Slice slice = type.getSlice(block, position);
                if (slice.length() == 0) {
                    throw new IllegalArgumentException("RCBinary encoder does not support empty VARBINARY values (HIVE-2483). Use ORC or Parquet format instead.");
                }
                output.writeBytes(slice);
            }
            encodeOutput.closeEntry();
        }
    }

    @Override
    public void encodeValueInto(Block block, int position, SliceOutput output)
    {
        Slice slice = type.getSlice(block, position);
        // Note binary nested in complex structures do no use the empty marker.
        // Therefore, empty VARBINARY values are ok.
        writeVInt(output, slice.length());
        output.writeBytes(slice);
    }

    @Override

View on GitHub (pinned to 55bb57d202)

Solutions

1. Replace empty VARBINARY values with NULL before writing (e.g. nullif(val, ''))
2. Use ORC or Parquet table format instead of RCFile, which handle empty values correctly
3. Upgrade Hive/Presto versions where HIVE-2483 may be worked around
4. Catch IllegalArgumentException from the writer and substitute a sentinel value

Example fix

// before
INSERT INTO rc_table VALUES ('');
// after
INSERT INTO rc_table VALUES (CAST(NULL AS varchar)); -- or nullif(col, '')

// before writing a VARBINARY column to RCBinary
for (int pos = 0; pos < block.getPositionCount(); pos++) {
    if (!block.isNull(pos) && type.getSlice(block, pos).length() == 0) {
        throw new IllegalArgumentException("empty VARBINARY not writable to RCBinary (HIVE-2483)");
    }
}

try { encoding.encodeColumn(block, output, encodeOutput); }
catch (IllegalArgumentException e) {
    if (e.getMessage().contains("HIVE-2483")) {
        block = nullifyEmptyVarbinary(block); // rewrite '' as NULL and retry
    } else { throw e; }
}

When it happens

Trigger: Calling BinaryEncoding.encodeColumn on a Block containing at least one non-null VARBINARY value of length 0 — e.g. INSERT INTO an RCFile table with an empty string/binary value ''.

Common situations: Inserting '' or empty binary literals into RCFile tables; ETL pipelines producing empty VARBINARY columns; migrating data containing empty blobs into RCBinary format.

Related errors

• BIGQUERY_UNSUPPORTED_TYPE_FOR_VARBINARY
• ELASTICSEARCH_TYPE_MISMATCH
• HIVE_WRITER_CLOSE_ERROR
• HIVE_WRITER_OPEN_ERROR
• HIVE_BAD_DATA

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/2150621297acba04. Report an issue: GitHub.