{"record":{"id":"7b6724d81de01983","repo":"apache/beam","slug":"bigquery-data-contained-value-s-with-sub-millisecond","errorCode":null,"errorMessage":"BigQuery data contained value %s with sub-millisecond precision, which Beam does not currently support. You can enable truncating timestamps to millisecond precision by using BigQueryIO.withTruncatedTimestamps","messagePattern":"BigQuery data contained value (.+?) with sub-millisecond precision, which Beam does not currently support\\. You can enable truncating timestamps to millisecond precision by using BigQueryIO\\.withTruncatedTimestamps","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java","lineNumber":1093,"sourceCode":"      case ROW:\n        Schema rowSchema = beamFieldType.getRowSchema();\n        if (rowSchema == null) {\n          throw new IllegalArgumentException(\"Nested ROW missing row schema\");\n        }\n        GenericData.Record record = (GenericData.Record) avroValue;\n        return toBeamRow(record, rowSchema, options);\n      case MAP:\n        return convertAvroRecordToMap(beamFieldType, avroValue, options);\n      default:\n        throw new RuntimeException(\n            \"Does not support converting unknown type value: \" + beamFieldTypeName);\n    }\n  }\n\n  private static ReadableInstant safeToMillis(Object value) {\n    long subMilliPrecision = ((long) value) % 1000;\n    if (subMilliPrecision != 0) {\n      throw new IllegalArgumentException(\n          String.format(\n              \"BigQuery data contained value %s with sub-millisecond precision, which Beam does\"\n                  + \" not currently support.\"\n                  + \" You can enable truncating timestamps to millisecond precision\"\n                  + \" by using BigQueryIO.withTruncatedTimestamps\",\n              value));\n    } else {\n      return truncateToMillis(value);\n    }\n  }\n\n  private static ReadableInstant truncateToMillis(Object value) {\n    return new Instant((long) value / 1000);\n  }\n\n  private static Object convertAvroArray(\n      FieldType beamField, Object value, BigQueryUtils.ConversionOptions options) {\n    // Check whether the type of array element is equal.","sourceCodeStart":1075,"sourceCodeEnd":1111,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java#L1075-L1111","documentation":"Thrown by safeToMillis when the default REJECT truncation mode finds a BigQuery timestamp Avro value (microseconds since epoch) whose sub-second remainder is not a whole number of milliseconds (value % 1000 != 0). Beam's Instant cannot represent sub-millisecond precision, so in REJECT mode the conversion fails instead of silently losing precision. The message tells you the sanctioned workaround: withTruncatedTimestamps.","triggerScenarios":"Reading a BigQuery TIMESTAMP column whose stored value has sub-millisecond precision (microsecond timestamps, e.g. '2020-01-01 00:00:00.000123 UTC') via BigQueryIO.readTableRows/read(Schema) with default (REJECT) ConversionOptions.","commonSituations":"Tables populated by other systems (e.g. Python datetime, other databases) writing microsecond-precision timestamps; BigQuery TIMESTAMP always stores microseconds, so imported data often carries sub-ms digits.","solutions":["Apply BigQueryIO.withTruncatedTimestamps() (or ConversionOptions TRUNCATE mode) so sub-millisecond digits are dropped via truncateToMillis.","Pre-clean the data: round/normalize the BigQuery column to millisecond precision (e.g. TIMESTAMP_TRUNC(col, MILLISECOND)) before export/read.","Catch the IllegalArgumentException around the read conversion and fall back to a truncating conversion for affected records.","If millisecond loss is unacceptable, read the raw microsecond value as INT64/BYTES and construct java.time.Instant yourself."],"exampleFix":"// before\nBigQueryIO.readTableRows().fromQuery(\"SELECT ts FROM t\") // REJECT default\n// after\nBigQueryIO.readTableRows().fromQuery(\"SELECT ts FROM t\").withTruncatedTimestamps()","handlingStrategy":"try-catch","validationCode":"// Java: probe data before reading (or pre-round in SQL)\n// SELECT TIMESTAMP_TRUNC(ts, MILLISECOND) AS ts FROM t  -- guarantees no sub-ms values","typeGuard":null,"tryCatchPattern":"try {\n  result = pipeline.apply(BigQueryIO.readTableRows().fromQuery(q));\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().contains(\"sub-millisecond precision\")) {\n    // rerun with truncation enabled\n    result = pipeline.apply(BigQueryIO.readTableRows().fromQuery(q).withTruncatedTimestamps());\n  } else throw e;\n}","preventionTips":["Enable withTruncatedTimestamps() up front when reading TIMESTAMP columns you don't control.","Round timestamps to millisecond precision in SQL (TIMESTAMP_TRUNC) before reading.","Document precision loss when truncation is enabled so downstream consumers know."],"tags":["java","beam","bigquery","timestamp","precision"],"backgroundTag":"value-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}