apache/beam · error · IllegalArgumentException

Property conflict: '%s' is already set to '%s' (likely from

Error message

Property conflict: '%s' is already set to '%s' (likely from JDBC URL), but attempting to set it to '%s'. Please use either JDBC URL properties OR withProperties(), not both for the same keys.

What it means

ClickHouseIO.Write.withProperties detects when a property being set conflicts with one already derived from the JDBC URL. If the same key exists in both places with different values, the builder throws IllegalArgumentException with a formatted message naming the key and both values, preventing ambiguous connection configuration where one source silently overrides the other.

Source

Thrown at sdks/java/io/clickhouse/src/main/java/org/apache/beam/sdk/io/clickhouse/ClickHouseIO.java:428

     *
     * @param properties connection properties
     * @return a {@link PTransform} writing data to ClickHouse
     * @throws IllegalArgumentException if properties is null or if any property conflicts with
     *     existing properties (e.g., from JDBC URL)
     */
    public Write<T> withProperties(Properties properties) {
      if (properties == null) {
        throw new IllegalArgumentException("Properties cannot be null");
      }

      // Check for conflicts with existing properties
      Properties existing = properties();
      for (String key : properties.stringPropertyNames()) {
        if (existing.containsKey(key)) {
          String existingValue = existing.getProperty(key);
          String newValue = properties.getProperty(key);
          if (!existingValue.equals(newValue)) {
            throw new IllegalArgumentException(
                String.format(
                    "Property conflict: '%s' is already set to '%s' (likely from JDBC URL), "
                        + "but attempting to set it to '%s'. "
                        + "Please use either JDBC URL properties OR withProperties(), not both for the same keys.",
                    key, existingValue, newValue));
          }
        }
      }

      // Merge properties: new properties are added to existing ones
      Properties merged = new Properties();
      merged.putAll(existing);
      merged.putAll(properties);
      return toBuilder().properties(merged).build();
    }

    /** Builder for {@link Write}. */
    @AutoValue.Builder

View on GitHub (pinned to 12126d8942)

Solutions

1. Remove the duplicated key from either the JDBC URL or the withProperties call so each key is set in exactly one place
2. Make the values identical if both sources must carry the key
3. Parse your JDBC URL (ClickHouseJdbcUrlParser.parse) and merge programmatically, letting the URL win or explicitly overriding, before calling withProperties
4. Log the effective properties at startup to catch duplication early

Example fix

// before
String url = "jdbc:clickhouse://host:8123/db?user=bob";
props.setProperty("user", "alice"); // conflict
// after
remove "user=bob" from the URL, or set props "user" to "bob" to match

ParsedJdbcUrl parsed = ClickHouseJdbcUrlParser.parse(jdbcUrl);
for (String key : userProps.stringPropertyNames()) {
  String urlVal = parsed.getProperties().getProperty(key);
  if (urlVal != null && !urlVal.equals(userProps.getProperty(key))) {
    throw new IllegalArgumentException("Duplicate property in URL and withProperties: " + key);
  }
}

try {
  writer = writer.withProperties(props);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Property conflict")) { LOG.error("Remove the duplicated key from the JDBC URL or withProperties"); }
  throw e;
}

When it happens

Trigger: Setting e.g. withProperties({"user": "alice"}) while the JDBC URL already embeds user=bob — any key present in both the parsed URL properties and the Properties argument with unequal values triggers the throw.

Common situations: Credentials or settings duplicated between a URL string pasted from one environment and a Properties file maintained separately; environment-specific URLs carrying properties that a shared code path also sets; someone adds a property to withProperties without realizing the URL already defines it.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Properties cannot be null
• Failed to write to ClickHouse after retries
• Failed to get table schema for table:
• JDBC URL cannot be null or empty
• ResultSetMetaData is null

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/526af8c4bb6395f6. Report an issue: GitHub.