apache/beam · error · ValueError

Empty Mongodb collection

Error message

Empty Mongodb collection

What it means

_ObjectIdRangeTracker._get_head_document_id queries the collection sorted by _id and reads cursor[0]["_id"]; when the collection is empty the cursor has no first element, IndexError fires, and the code re-raises ValueError('Empty Mongodb collection'). It is used to fill in None start/stop positions for _id-range based reads.

Solutions

1. Verify the collection has documents: run db.<coll>.count_documents({}) in mongo before launching the job.
2. Check the db/coll names and the filter — a typo or over-restrictive filter can make the collection effectively empty.
3. Seed the collection or skip the pipeline when it is empty.
4. Catch ValueError around the read transform setup and short-circuit empty collections.

Example fix

// before
p | ReadFromMongoDB(uri=uri, db='analytics', coll='events')
// after
if mongo_client['analytics']['events'].count_documents({}) == 0:
    return  # nothing to read
p | ReadFromMongoDB(uri=uri, db='analytics', coll='events')

if client[db][coll].count_documents({}, limit=1) == 0:
    raise EmptyCollectionError(f'{db}.{coll} is empty')

try:
    _ = pipeline | ReadFromMongoDB(uri=uri, db=db, coll=coll)
except ValueError as e:
    if 'Empty Mongodb collection' in str(e):
        return  # nothing to process

When it happens

Trigger: Running ReadFromMongoDB with the _id-based position mode against a collection that contains zero documents (or a filter that matches nothing), causing _replace_none_positions to look up the first/last document id.

Common situations: Pointing a pipeline at a fresh/empty collection or a wrong db/coll name; a filter argument that excludes all documents; staging environments with unseeded data.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

• fraction must be between 0 and 1
• ReadFromMongDB coll param must be specified as a string
• ReadFromMongDB db param must be specified as a string
• WriteToMongoDB db param must be specified as a string
• A VPC network must be provided to use a private endpoint.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f73e2cdfbf40bb06. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/mongodbio.py:564

          # https://docs.mongodb.com/manual/reference/operator/query/and/
          "$and": [self.filter.copy(), id_filter]
      }
    else:
      all_filters = id_filter

    return all_filters

  def _get_head_document_id(self, sort_order):
    with MongoClient(self.uri, **self.spec) as client:
      cursor = (
          client[self.db][self.coll].find(filter={}, projection=[]).sort([
              ("_id", sort_order)
          ]).limit(1))
      try:
        return cursor[0]["_id"]

      except IndexError:
        raise ValueError("Empty Mongodb collection")

  def _replace_none_positions(self, start_position, stop_position):

    if start_position is None:
      start_position = self._get_head_document_id(ASCENDING)
    if stop_position is None:
      last_doc_id = self._get_head_document_id(DESCENDING)
      # increment last doc id binary value by 1 to make sure the last document
      # is not excluded
      if isinstance(last_doc_id, ObjectId):
        stop_position = _ObjectIdHelper.increment_id(last_doc_id, 1)
      elif isinstance(last_doc_id, int):
        stop_position = last_doc_id + 1
      elif isinstance(last_doc_id, str):
        stop_position = last_doc_id + '\x00'

    return start_position, stop_position

View on GitHub (pinned to 12126d8942)