apache/beam · error · UnsupportedOperationException

The Primitive HCat type

Error message

The Primitive HCat type '{type}' of field '{name}' cannot be converted to Beam FieldType

What it means

SchemaUtils.toBeamField only maps HCatalog PRIMITIVE types present in HCAT_TO_BEAM_TYPES_MAP to Beam FieldTypes. When a primitive HCat type (e.g. DECIMAL, INTERVAL, BINARY variants) has no mapping, the library refuses to silently degrade and throws UnsupportedOperationException naming the type and field.

Solutions

Alter the table (or a view over it) to cast unsupported columns to supported types (e.g. CAST(col AS STRING) or DOUBLE instead of DECIMAL)
Add a mapping to HCAT_TO_BEAM_TYPES_MAP in SchemaUtils if the Beam FieldType exists upstream
Filter/prune the column by copying into a staging table with supported types only
Check Beam version: newer releases may have extended the map

Example fix

-- before: table has DECIMAL amount
CREATE VIEW t_beam AS SELECT CAST(amount AS DOUBLE) AS amount, ... FROM raw_t;
-- after: read the view
HCatalogIO.read().withTable("t_beam")

Defensive patterns

Strategy: validation

Validate before calling

boolean supported = hcatField.getCategory()==HCatFieldSchema.Category.PRIMITIVE && SchemaUtils.HCAT_TO_BEAM_TYPES_MAP.containsKey(hcatField.getType());

Type guard

null

Try / catch

try { ... } catch (UnsupportedOperationException e) { throw new IllegalArgumentException("Incompatible HCatalog primitive type; see SchemaUtils map", e); }

Prevention

Avoid DECIMAL/CHAR/VARCHAR columns in tables Beam reads
Expose views with cast columns
Check HCAT_TO_BEAM_TYPES_MAP when designing table schemas
Upgrade Beam for newer type mappings

When it happens

Trigger: Reading an HCatalog table whose schema contains a primitive type outside the supported map (commonly DECIMAL, CHAR, VARCHAR, TIMESTAMP WITH LOCAL TIME ZONE) via HCatalogIO.read().

Common situations: Hive tables using DECIMAL(10,2) columns; tables with CHAR/VARCHAR after a Hive upgrade; reading tables designed for Hive not Beam.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a592c00078e3397c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hcatalog/src/main/java/org/apache/beam/sdk/io/hcatalog/SchemaUtils.java:76

  }

  private static Schema.Field toBeamField(FieldSchema field) {
    String name = field.getName();
    HCatFieldSchema hCatFieldSchema;

    try {
      hCatFieldSchema = HCatSchemaUtils.getHCatFieldSchema(field);
    } catch (HCatException e) {
      // Converting checked Exception to unchecked Exception.
      throw new UnsupportedOperationException(
          "Error while converting FieldSchema to HCatFieldSchema", e);
    }

    switch (hCatFieldSchema.getCategory()) {
      case PRIMITIVE:
        {
          if (!HCAT_TO_BEAM_TYPES_MAP.containsKey(hCatFieldSchema.getType())) {
            throw new UnsupportedOperationException(
                "The Primitive HCat type '"
                    + field.getType()
                    + "' of field '"
                    + name
                    + "' cannot be converted to Beam FieldType");
          }

          FieldType fieldType = HCAT_TO_BEAM_TYPES_MAP.get(hCatFieldSchema.getType());
          return Schema.Field.of(name, fieldType).withNullable(true);
        }
        // TODO: Add Support for Complex Types i.e. ARRAY, MAP, STRUCT
      default:
        throw new UnsupportedOperationException(
            "The category '" + hCatFieldSchema.getCategory() + "' is not supported.");
    }
  }
}

View on GitHub (pinned to 12126d8942)