apache/druid · error · ParseException

Protobuf message could not be parsed

Error message

Protobuf message could not be parsed

What it means

When the timestamp column of a protobuf record is itself a protobuf Message, ProtobufInputRowSchema.extractTimestamp prints it as JSON via JsonFormat and parses the resulting string as a date. If JsonFormat printing throws InvalidProtocolBufferException (e.g. Any messages that cannot be resolved), Druid wraps it in a ParseException with this message.

Solutions

1. Use a scalar (int64/string) or google.protobuf.Timestamp field as the timestamp column instead of an arbitrary message
2. If Any is required, ensure the embedded type's descriptor is registered on the Druid classpath
3. Catch and inspect the wrapped cause in a custom parser to identify which message failed

Example fix

// before: timestamp_column points at a google.protobuf.Any field
"timestampColumn": "metadata"
// after: point at a Timestamp or scalar field
"timestampColumn": "event_time"

Object ts = ...; // resolved timestamp field
if (ts instanceof Message
    && "google.protobuf.Any".equals(((Message) ts).getDescriptorForType().getFullName())) {
  throw new IllegalStateException("Any is not usable as timestamp column; use Timestamp or scalar");
}

boolean isUsableTimestamp(Object raw) {
  return raw == null || !(raw instanceof Message)
      || "google.protobuf.Timestamp".equals(((Message) raw).getDescriptorForType().getFullName());
}

try {
  ingest(record);
} catch (ParseException e) {
  if ("Protobuf message could not be parsed".equals(e.getMessage()) && e.getCause() instanceof InvalidProtocolBufferException) {
    log.error("Unparseable timestamp message — check Any type_url resolution", e.getCause());
  }
}

When it happens

Trigger: The input row schema's timestamp field resolves to a Message object and JsonFormat.printer().print(...) throws InvalidProtocolBufferException — typically for google.protobuf.Any containing an unregistered/unresolvable type URL.

Common situations: Using a google.protobuf.Any (or Struct/Value) as the queryable timestamp column; an Any with a type_url not on the Druid classpath so JsonFormat cannot resolve its type.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

• Encountered row with timestamp
• Fail to decode protobuf message!
• Fail to get protobuf schema because of invalid schema!
• Invalid ListValue type.
• Invalid parameter type:

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/d663ed48ba9522c2. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/protobuf-extensions/src/main/java/org/apache/druid/data/input/protobuf/ProtobufInputRowSchema.java:74

    }

    /**
     * Extracts the timestamp from the record. If the timestamp column is of complex type such as {@link Timestamp},
     * then the timestamp is first serialized to string via {@link JsonFormat}. Directly calling {@code toString()}
     * on {@code Timestamp} returns an unparseable string.
     */
    @Override
    @Nullable
    public DateTime extractTimestamp(@Nullable Map<String, Object> input)
    {
      Object rawTimestamp = getRawTimestamp(input);
      if (rawTimestamp instanceof Message) {
        try {
          String timestampStr = JsonFormat.printer().print((Message) rawTimestamp);
          return parseDateTime(timestampStr);
        }
        catch (InvalidProtocolBufferException e) {
          throw new ParseException(null, e, "Protobuf message could not be parsed");
        }
      } else {
        return parseDateTime(rawTimestamp);
      }
    }
  }
}

View on GitHub (pinned to 9b90983fd2)