{"record":{"id":"0ad6875710e90025","repo":"opendatalab/MinerU","slug":"invalid-s3-path-format-expected-s3-bucket-name","errorCode":null,"errorMessage":"Invalid S3 path format. Expected 's3://bucket-name/key' or 's3a://bucket-name/key'.","messagePattern":"Invalid S3 path format\\. Expected 's3://bucket-name/key' or 's3a://bucket-name/key'\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mineru/data/utils/path_utils.py","lineNumber":23,"sourceCode":"    \"\"\"\n    example: s3://abc/xxxx.json?bytes=0,81350 ==> s3://abc/xxxx.json\n    \"\"\"\n    arr = s3path.split(\"?\")\n    return arr[0]\n\ndef parse_s3path(s3path: str):\n    # from s3pathlib import S3Path\n    # p = S3Path(remove_non_official_s3_args(s3path))\n    # return p.bucket, p.key\n    s3path = remove_non_official_s3_args(s3path).strip()\n    if s3path.startswith(('s3://', 's3a://')):\n        prefix, path = s3path.split('://', 1)\n        bucket_name, key = path.split('/', 1)\n        return bucket_name, key\n    elif s3path.startswith('/'):\n        raise ValueError(\"The provided path starts with '/'. This does not conform to a valid S3 path format.\")\n    else:\n        raise ValueError(\"Invalid S3 path format. Expected 's3://bucket-name/key' or 's3a://bucket-name/key'.\")\n\n\ndef parse_s3_range_params(s3path: str):\n    \"\"\"\n    example: s3://abc/xxxx.json?bytes=0,81350 ==> [0, 81350]\n    \"\"\"\n    arr = s3path.split(\"?bytes=\")\n    if len(arr) == 1:\n        return None\n    return arr[1].split(\",\")\n","sourceCodeStart":5,"sourceCodeEnd":34,"githubUrl":"https://github.com/opendatalab/MinerU/blob/4fe4bde114a23ee5dd637eae99b767f4669bf58c/mineru/data/utils/path_utils.py#L5-L34","documentation":"ValueError from parse_s3path() when the path has neither an s3:// / s3a:// scheme nor a leading '/'. After remove_non_official_s3_args() and strip(), anything that is not an S3 URI falls into this branch — the parser has no way to derive a bucket and key from it.","triggerScenarios":"parse_s3path('bucket/key.pdf') (scheme omitted), parse_s3path('https://bucket.s3.amazonaws.com/key'), parse_s3path('C:\\\\doc.pdf'), or an empty/whitespace string passed to S3 readers.","commonSituations":"Users pasting HTTPS S3 console URLs or plain bucket/key shorthand; Windows paths reaching S3 code by mistake; empty strings from optional config fields; upstream data files containing malformed URIs.","solutions":["Prefix the path with the scheme: 's3://bucket/key'.","Convert HTTPS-style URLs to s3:// form before calling S3 readers.","Filter or normalize input paths early (reject non-s3:// entries) so bad rows are reported with context instead of a raw ValueError.","For local files, route to a local reader instead."],"exampleFix":"# before\nparse_s3path('my-bucket/doc.pdf')          # ValueError\nparse_s3path('https://my-bucket.s3.amazonaws.com/doc.pdf')  # ValueError\n\n# after\nparse_s3path('s3://my-bucket/doc.pdf')","handlingStrategy":"type-guard","validationCode":"def validate_s3_path(path: str) -> str:\n    p = path.strip()\n    if not p.startswith(('s3://', 's3a://')):\n        raise ValueError(f\"path must start with 's3://' or 's3a://', got {path!r}\")\n    rest = p.split('://', 1)[1]\n    if '/' not in rest or not rest.split('/', 1)[0]:\n        raise ValueError(f'path must be s3://bucket/key, got {path!r}')\n    return p","typeGuard":"def is_valid_s3_path(path: str) -> bool:\n    p = path.strip()\n    if not p.startswith(('s3://', 's3a://')):\n        return False\n    rest = p.split('://', 1)[1]\n    bucket, _, key = rest.partition('/')\n    return bool(bucket) and bool(key)","tryCatchPattern":"try:\n    bucket, key = parse_s3path(user_path)\nexcept ValueError:\n    # log the offending row and skip, or route to local reader if it is a local file\n    raise","preventionTips":["Convert https:// S3 console URLs to s3://bucket/key during ingestion.","Reject empty strings and scheme-less paths at input validation time.","Separate local vs S3 path handling explicitly instead of a single shared parse."],"tags":["s3","path","validation"],"backgroundTag":null,"analyzedSha":"4fe4bde114a23ee5dd637eae99b767f4669bf58c","analyzedAt":"2026-08-14T21:29:18.456Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}