{"record":{"id":"e977bfdcaa77a5d0","repo":"apache/beam","slug":"invalid-page-token","errorCode":null,"errorMessage":"Invalid page token.","messagePattern":"Invalid page token\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/aws/clients/s3/fake_client.py","lineNumber":114,"sourceCode":"    matching_files = []\n\n    for file_bucket, file_name in sorted(iter(self.files)):\n      if bucket == file_bucket and file_name.startswith(prefix):\n        file_object = self.get_file(file_bucket, file_name).get_metadata()\n        matching_files.append(file_object)\n\n    if not matching_files:\n      message = 'Tried to list nonexistent S3 path: s3://%s/%s' % (\n          bucket, prefix)\n      raise messages.S3ClientError(message, 404)\n\n    # Handle pagination.\n    items_per_page = 5\n    if not request.continuation_token:\n      range_start = 0\n    else:\n      if request.continuation_token not in self.list_continuation_tokens:\n        raise ValueError('Invalid page token.')\n      range_start = self.list_continuation_tokens[request.continuation_token]\n      del self.list_continuation_tokens[request.continuation_token]\n\n    result = messages.ListResponse(\n        items=matching_files[range_start:range_start + items_per_page])\n\n    if range_start + items_per_page < len(matching_files):\n      next_range_start = range_start + items_per_page\n      next_continuation_token = '_page_token_%s_%s_%d' % (\n          bucket, prefix, next_range_start)\n      self.list_continuation_tokens[next_continuation_token] = next_range_start\n      result.next_token = next_continuation_token\n\n    return result\n\n  def get_range(self, request, start, end):\n    r\"\"\"Retrieves an object.\n","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/aws/clients/s3/fake_client.py#L96-L132","documentation":"The fake (in-memory) S3 client used by apache_beam tests raises ValueError('Invalid page token.') in list() when a ListRequest carries a continuation_token that is not present in its internal token registry. This simulates S3 rejecting an expired or fabricated pagination token; the fake deletes tokens after one use, so replaying a token also fails.","triggerScenarios":"Calling FakeAwsS3Client.list() with request.continuation_token that was never issued, was already consumed (fake deletes tokens after use), or the fake client was recreated between pages so the registry is empty.","commonSituations":"Unit tests replaying a saved continuation token; retrying a page after the fake was reset; holding tokens across two different fake client instances; tests hard-coding token strings.","solutions":["Use only continuation tokens taken from the immediately preceding ListResponse of the same fake client instance.","Do not reuse a token after a successful page fetch — the fake deletes consumed tokens.","Recreate/restart pagination from the beginning (no continuation_token) if the fake client was reset.","In tests, drive pagination in a loop using response.next_token rather than fixed tokens."],"exampleFix":"// before\nclient.list(messages.ListRequest(continuation_token='tok-from-old-instance'))\n// after\nresp = client.list(messages.ListRequest())\nnext_req = messages.ListRequest(continuation_token=resp.next_token)  # token from same client","handlingStrategy":"type-guard","validationCode":"def token_is_fresh(token: str | None, client) -> bool:\n    return token is None or token in client.list_continuation_tokens","typeGuard":"def has_valid_token(req) -> bool:\n    return not getattr(req, 'continuation_token', None) or \\\n           req.continuation_token in client.list_continuation_tokens","tryCatchPattern":"try:\n    resp = client.list(request)\nexcept ValueError as e:\n    if str(e) == 'Invalid page token.':\n        request.continuation_token = None  # restart pagination\n        resp = client.list(request)\n    else:\n        raise","preventionTips":["Only pass next_token from the immediately previous response of the same client instance","Never reuse consumed continuation tokens","Re-create tokens when the fake client is recreated in tests"],"tags":["s3","pagination","fake-client","testing","aws"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}