apache/beam · error · UnsupportedOperationException

Beam SQL cannot convert Timestamp values with…

Error message

Beam SQL cannot convert Timestamp values with sub-millisecond precision through Calcite (millis-based TIMESTAMP). Got: ${instant}

What it means

Beam SQL's Calcite integration represents TIMESTAMP values as epoch milliseconds, so any sub-millisecond precision would be silently lost during conversion. timestampToCalciteMillis detects this truncation and throws UnsupportedOperationException instead of dropping precision silently. Use TIMESTAMP(3) semantics or a different representation if you need microsecond/nanosecond precision.

Solutions

1. Truncate or round the Instant to milliseconds before passing it into SQL: instant.truncatedTo(ChronoUnit.MILLIS)
2. Change the schema/logical type so the field is stored at millisecond precision end-to-end
3. Handle the timestamp outside SQL (extract to a separate transform) if sub-ms precision matters
4. Patch/fork timestampToCalciteMillis to support finer precision via a long nanos representation

Example fix

// before
long millis = BeamCalcRel.timestampToCalciteMillis(eventInstant);
// after
long millis = BeamCalcRel.timestampToCalciteMillis(eventInstant.truncatedTo(java.time.temporal.ChronoUnit.MILLIS));

if (!instant.equals(java.time.Instant.ofEpochMilli(instant.toEpochMilli()))) {
  instant = instant.truncatedTo(java.time.temporal.ChronoUnit.MILLIS); // or reject
}

boolean isMillisPrecise(java.time.Instant i) {
  return i.equals(java.time.Instant.ofEpochMilli(i.toEpochMilli()));
}

try {
  long millis = BeamCalcRel.timestampToCalciteMillis(instant);
} catch (UnsupportedOperationException e) {
  // fall back to truncated value or custom handling
  long millis = instant.toEpochMilli();
}

When it happens

Trigger: Calling BeamCalcRel.timestampToCalciteMillis with a java.time.Instant whose epoch value is not an exact multiple of 1000 nanoseconds (i.e. toEpochMilli() truncates nanoseconds). This happens when a Beam Row field with microsecond-precision timestamp is fed into SQL via BeamCalciteConverters.

Common situations: Reading rows produced by sources that store microsecond timestamps (e.g. Kafka timestamps with microseconds, DoFn with precise event times) and running them through Beam SQL. Also occurs after upgrading where upstream code changed Instant precision.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• buildIOWriter unsupported!
• CalcFn failed to evaluate
• Cannot create a filter for an unsupported node
• Could not compile CalcFn
• Creating tables is not supported with DataCatalog table…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9f643ab0fa6aa92a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java:145

public class BeamCalcRel extends AbstractBeamCalcRel {

  private static final long NANOS_PER_MILLISECOND = 1000000L;
  private static final long MILLIS_PER_DAY = 86400000L;

  private static final ParameterExpression rowParam = Expressions.parameter(Row.class, "row");
  private static final TupleTag<Row> rows = new TupleTag<Row>() {};
  private static final TupleTag<Row> errors = new TupleTag<Row>() {};

  /**
   * Converts a {@link java.time.Instant} from a Timestamp logical type to Calcite TIMESTAMP millis.
   * Calcite's TIMESTAMP is millisecond-based, so sub-millisecond values are rejected rather than
   * silently truncated.
   */
  public static long timestampToCalciteMillis(java.time.Instant instant) {
    long millis = instant.toEpochMilli();
    // toEpochMilli truncates; reject rather than silently drop sub-millisecond precision.
    if (!instant.equals(java.time.Instant.ofEpochMilli(millis))) {
      throw new UnsupportedOperationException(
          "Beam SQL cannot convert Timestamp values with sub-millisecond precision through"
              + " Calcite (millis-based TIMESTAMP). Got: "
              + instant);
    }
    return millis;
  }

  public BeamCalcRel(RelOptCluster cluster, RelTraitSet traits, RelNode input, RexProgram program) {
    super(cluster, traits, input, program);
  }

  @Override
  public Calc copy(RelTraitSet traitSet, RelNode input, RexProgram program) {
    return new BeamCalcRel(getCluster(), traitSet, input, program);
  }

  @Override
  public PTransform<PCollectionList<Row>, PCollection<Row>> buildPTransform(

View on GitHub (pinned to 12126d8942)