apache/beam · error · IOException

Can not get unique key from solr

Error message

Can not get unique key from solr

What it means

SolrIO's cursor-based (deep-paging) read first queries Solr for the collection's unique key field to sort results deterministically. If the Solr server throws SolrServerException during that SchemaRequest.UniqueKey lookup, the connector wraps it in this IOException and aborts the read.

Source

Thrown at sdks/java/io/solr/src/main/java/org/apache/beam/sdk/io/solr/SolrIO.java:465

    public void process(@Element Read spec, OutputReceiver<SolrDocument> out) throws IOException {
      ReplicaInfo replicaInfo = spec.getReplicaInfo();
      checkArgument(replicaInfo != null, "replicaInfo is required");
      String cursorMark = CursorMarkParams.CURSOR_MARK_START;
      String query = spec.getQuery();
      if (query == null) {
        query = "*:*";
      }
      SolrQuery solrQuery = new SolrQuery(query);
      solrQuery.setRows(spec.getBatchSize());
      solrQuery.setDistrib(false);
      try (AuthorizedSolrClient<HttpSolrClient> client =
          spec.getConnectionConfiguration().createClient(replicaInfo.baseUrl())) {
        SchemaRequest.UniqueKey request = new SchemaRequest.UniqueKey();
        try {
          SchemaResponse.UniqueKeyResponse response = client.process(spec.getCollection(), request);
          solrQuery.addSort(response.getUniqueKey(), SolrQuery.ORDER.asc);
        } catch (SolrServerException e) {
          throw new IOException("Can not get unique key from solr", e);
        }

        while (true) {
          solrQuery.set(CursorMarkParams.CURSOR_MARK_PARAM, cursorMark);
          try {
            QueryResponse response;
            response = client.query(replicaInfo.coreName(), solrQuery);
            if (cursorMark.equals(response.getNextCursorMark())) {
              break;
            }
            cursorMark = response.getNextCursorMark();
            for (SolrDocument doc : response.getResults()) {
              out.output(doc);
            }
          } catch (SolrServerException e) {
            throw new IOException(e);
          }
        }

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify the collection name and connection configuration (zkHost/URL) are correct
2. Check Solr server health and that the schema API (/schema/uniquekey) is reachable and permitted
3. Inspect the wrapped SolrServerException cause for the root network/server error and fix connectivity or scale the Solr cluster
4. Retry after Solr finishes recovery/migrations

Example fix

// before
SolrIO.read().withConnectionConfiguration(SolrIO.ConnectionConfiguration.create("bad-host:9983","docs"));
// after
SolrIO.read().withConnectionConfiguration(SolrIO.ConnectionConfiguration.create("zk1:9983,zk2:9983","docs"));

// preflight: verify collection and schema API reachable
try (SolrClient c = connConfig.createClient(baseUrl)) {
  c.process(collection, new SchemaRequest.UniqueKey());
}

try {
  readFromSolr();
} catch (IOException e) {
  if (e.getCause() instanceof SolrServerException) {
    LOG.error("Solr unique-key lookup failed; check collection/cluster health", e);
  }
  throw e;
}

When it happens

Trigger: client.process(collection, SchemaRequest.UniqueKey) fails — e.g. Solr node unreachable/overloaded, collection name wrong, schema API disabled, or replica returning errors.

Common situations: Misconfigured collection name or connection URL; Solr Cloud node down or in recovery; firewalls/security plugins blocking the schema API endpoint.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• Need to set the filepattern of a TFRecordIO.Read transform
• expected %d, but got %d
• Unknown %s type %s
• Failed to retrieve secret bytes
• cannot encode a null SolrDocument

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/16eb45adbe5950dc. Report an issue: GitHub.