{"record":{"id":"9f643ab0fa6aa92a","repo":"apache/beam","slug":"beam-sql-cannot-convert-timestamp-values-with-sub","errorCode":null,"errorMessage":"Beam SQL cannot convert Timestamp values with sub-millisecond precision through Calcite (millis-based TIMESTAMP). Got: ${instant}","messagePattern":"Beam SQL cannot convert Timestamp values with sub-millisecond precision through Calcite \\(millis-based TIMESTAMP\\)\\. Got: (.+?)","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java","lineNumber":145,"sourceCode":"public class BeamCalcRel extends AbstractBeamCalcRel {\n\n  private static final long NANOS_PER_MILLISECOND = 1000000L;\n  private static final long MILLIS_PER_DAY = 86400000L;\n\n  private static final ParameterExpression rowParam = Expressions.parameter(Row.class, \"row\");\n  private static final TupleTag<Row> rows = new TupleTag<Row>() {};\n  private static final TupleTag<Row> errors = new TupleTag<Row>() {};\n\n  /**\n   * Converts a {@link java.time.Instant} from a Timestamp logical type to Calcite TIMESTAMP millis.\n   * Calcite's TIMESTAMP is millisecond-based, so sub-millisecond values are rejected rather than\n   * silently truncated.\n   */\n  public static long timestampToCalciteMillis(java.time.Instant instant) {\n    long millis = instant.toEpochMilli();\n    // toEpochMilli truncates; reject rather than silently drop sub-millisecond precision.\n    if (!instant.equals(java.time.Instant.ofEpochMilli(millis))) {\n      throw new UnsupportedOperationException(\n          \"Beam SQL cannot convert Timestamp values with sub-millisecond precision through\"\n              + \" Calcite (millis-based TIMESTAMP). Got: \"\n              + instant);\n    }\n    return millis;\n  }\n\n  public BeamCalcRel(RelOptCluster cluster, RelTraitSet traits, RelNode input, RexProgram program) {\n    super(cluster, traits, input, program);\n  }\n\n  @Override\n  public Calc copy(RelTraitSet traitSet, RelNode input, RexProgram program) {\n    return new BeamCalcRel(getCluster(), traitSet, input, program);\n  }\n\n  @Override\n  public PTransform<PCollectionList<Row>, PCollection<Row>> buildPTransform(","sourceCodeStart":127,"sourceCodeEnd":163,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java#L127-L163","documentation":"Beam SQL's Calcite integration represents TIMESTAMP values as epoch milliseconds, so any sub-millisecond precision would be silently lost during conversion. timestampToCalciteMillis detects this truncation and throws UnsupportedOperationException instead of dropping precision silently. Use TIMESTAMP(3) semantics or a different representation if you need microsecond/nanosecond precision.","triggerScenarios":"Calling BeamCalcRel.timestampToCalciteMillis with a java.time.Instant whose epoch value is not an exact multiple of 1000 nanoseconds (i.e. toEpochMilli() truncates nanoseconds). This happens when a Beam Row field with microsecond-precision timestamp is fed into SQL via BeamCalciteConverters.","commonSituations":"Reading rows produced by sources that store microsecond timestamps (e.g. Kafka timestamps with microseconds, DoFn with precise event times) and running them through Beam SQL. Also occurs after upgrading where upstream code changed Instant precision.","solutions":["Truncate or round the Instant to milliseconds before passing it into SQL: instant.truncatedTo(ChronoUnit.MILLIS)","Change the schema/logical type so the field is stored at millisecond precision end-to-end","Handle the timestamp outside SQL (extract to a separate transform) if sub-ms precision matters","Patch/fork timestampToCalciteMillis to support finer precision via a long nanos representation"],"exampleFix":"// before\nlong millis = BeamCalcRel.timestampToCalciteMillis(eventInstant);\n// after\nlong millis = BeamCalcRel.timestampToCalciteMillis(eventInstant.truncatedTo(java.time.temporal.ChronoUnit.MILLIS));","handlingStrategy":"validation","validationCode":"if (!instant.equals(java.time.Instant.ofEpochMilli(instant.toEpochMilli()))) {\n  instant = instant.truncatedTo(java.time.temporal.ChronoUnit.MILLIS); // or reject\n}","typeGuard":"boolean isMillisPrecise(java.time.Instant i) {\n  return i.equals(java.time.Instant.ofEpochMilli(i.toEpochMilli()));\n}","tryCatchPattern":"try {\n  long millis = BeamCalcRel.timestampToCalciteMillis(instant);\n} catch (UnsupportedOperationException e) {\n  // fall back to truncated value or custom handling\n  long millis = instant.toEpochMilli();\n}","preventionTips":["Truncate Instants to milliseconds before entering Beam SQL pipelines","Design schemas with millisecond-precision timestamp fields end-to-end","Add unit tests asserting timestamp precision before SQL stages","Document precision limits for users feeding microsecond sources"],"tags":["java","apache-beam","sql","timestamp-precision"],"backgroundTag":"value-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}