apache/druid · error · IllegalStateException

Dimension[ ] occurred more than once in InputRow

Error message

Dimension[%s] occurred more than once in InputRow

What it means

IncrementalIndex.toIncrementalIndexRow throws IllegalStateException when the same dimension appears more than once in an InputRow. The index tracks per-row which dimensions have been processed (via the dims array); encountering a dimension again within one row would double-count values, so it fails fast. Rows with duplicate dimension keys violate the InputRow contract.

Solutions

  1. Deduplicate dimension names before constructing the InputRow
  2. Fix the parser/config so each dimension appears at most once per row
  3. Implement a custom InputRow wrapper that deduplicates getDimensions()

Example fix

// before
new MapBasedInputRow(timestamp, Arrays.asList("d1", "d2", "d1"), event); // duplicate "d1"
// after
List<String> dims = event.keySet().stream().distinct().collect(Collectors.toList());
new MapBasedInputRow(timestamp, dims, event);
Defensive patterns

Strategy: validation

Validate before calling

Set<String> seen = new HashSet<>(); for (String d : row.getDimensions()) { if (!seen.add(d)) { throw new IllegalArgumentException("duplicate dimension in row: " + d); } }

Try / catch

try { index.add(row); } catch (IllegalStateException e) { if (e.getMessage().contains("occurred more than once in InputRow")) { log.error("dropping malformed row", e); return -1; } throw e; }

Prevention

When it happens

Trigger: Adding an InputRow whose getDimensions() contains the same dimension name twice (e.g. duplicate keys in a map-backed implementation, or custom InputRow implementations returning duplicates).

Common situations: Custom InputRow/MapBasedInputRow construction with duplicated dimension entries; parsers (e.g. from JSON with duplicate keys or CSV with repeated columns) producing duplicate dimension names; event flattening bugs.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/7d52cbf519b93889. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/incremental/IncrementalIndex.java:565

          // unless this is the first row we are processing, all newly discovered columns will be sparse
          if (maxIngestedEventTime != null) {
            indexer.setSparseIndexed();
          }
          if (overflow == null) {
            overflow = new ArrayList<>();
          }
          overflow.add(dimsKey);
        } else if (desc.getIndex() > dims.length || dims[desc.getIndex()] != null) {
          /*
           * index > dims.length requires that we saw this dimension and added it to the dimensionOrder map,
           * otherwise index is null. Since dims is initialized based on the size of dimensionOrder on each call to add,
           * it must have been added to dimensionOrder during this InputRow.
           *
           * if we found an index for this dimension it means we've seen it already. If !(index > dims.length) then
           * we saw it on a previous input row (this its safe to index into dims). If we found a value in
           * the dims array for this index, it means we have seen this dimension already on this input row.
           */
          throw new ISE("Dimension[%s] occurred more than once in InputRow", dimension);
        } else {
          dims[desc.getIndex()] = dimsKey;
        }
      }

      // process any dimensions with missing values in the row
      for (String missing : absentDimensions) {
        dimensionDescs.get(missing).getIndexer().setSparseIndexed();
      }
    }

    if (overflow != null) {
      // Merge overflow and non-overflow
      Object[] newDims = new Object[dims.length + overflow.size()];
      System.arraycopy(dims, 0, newDims, 0, dims.length);
      for (int i = 0; i < overflow.size(); ++i) {
        newDims[dims.length + i] = overflow.get(i);
      }

View on GitHub (pinned to 9b90983fd2)