{"record":{"id":"26ac3f7cd26c828c","repo":"HumanSignal/label-studio","slug":"failed-to-list-storage-files","errorCode":null,"errorMessage":"Failed to list storage files","messagePattern":"Failed to list storage files","errorType":"validation","errorClass":"ValidationError","httpStatus":400,"severity":"error","filePath":"label_studio/io_storages/api.py","lineNumber":231,"sourceCode":"            timeout_seconds = 30\n\n            for object in instance.iter_objects():\n                files.append(instance.get_unified_metadata(object))\n\n                # Check if we've reached the file limit\n                if len(files) >= limit:\n                    files.append({'key': None, 'last_modified': None, 'size': None})\n                    break\n\n                # Check if we've exceeded the timeout\n                if time.time() - start_time > timeout_seconds:\n                    files.append({'key': '... storage scan timeout reached ...', 'last_modified': None, 'size': None})\n                    break\n\n            return Response({'files': files})\n        except Exception as exc:\n            logger.exception('Error listing storage files: %s', exc)\n            raise ValidationError('Failed to list storage files')\n\n\n@extend_schema(exclude=True)\nclass StorageFormLayoutAPI(generics.RetrieveAPIView):\n    permission_required = all_permissions.storages_change\n    parser_classes = (JSONParser, FormParser, MultiPartParser)\n    storage_type = None\n\n    @extend_schema(exclude=True)\n    def get(self, request, *args, **kwargs):\n        form_layout_file = os.path.join(os.path.dirname(inspect.getfile(self.__class__)), 'form_layout.yml')\n        if not os.path.exists(form_layout_file):\n            raise NotFound(f'\"form_layout.yml\" is not found for {self.__class__.__name__}')\n\n        form_layout = read_yaml(form_layout_file)\n        form_layout = self.post_process_form(form_layout)\n        return Response(form_layout[self.storage_type])\n","sourceCodeStart":213,"sourceCodeEnd":249,"githubUrl":"https://github.com/HumanSignal/label-studio/blob/0b49e9b53917880baf1dd85d574fe5541a9aafb2/label_studio/io_storages/api.py#L213-L249","documentation":"The storage file-listing action endpoint catches any exception raised while scanning the storage (listing objects from S3/GCS/Azure) and re-raises it as ValidationError('Failed to list storage files'), logging the real cause. The generic message hides provider-specific failures like auth errors, timeouts, or bad prefixes.","triggerScenarios":"POST to /api/storages/<type>/<id>/list-files (sync action returning files) where the underlying SDK call to list blobs/objects raises — invalid credentials, missing bucket, throttling, network timeout, or a scan exceeding the iteration timeout.","commonSituations":"Very large containers timing out during scan; expired cloud session; bucket renamed after storage was created; rate limiting (S3 SlowDown, Azure 5xx); DNS/egress blocked in self-hosted deployments.","solutions":["Check server logs for the 'Error listing storage files' traceback to see the provider error","Re-validate/re-save the storage connection with correct credentials","Reduce the prefix to limit the scan scope, or increase timeout settings","Retry — transient throttling/network errors resolve on retry","Verify the bucket/container still exists and the storage record points to it"],"exampleFix":"// before\nPOST /api/storages/s3/42/list-files  → 400 Failed to list storage files (logs: AccessDenied)\n// after\nPUT /api/storages/s3/42 {\"aws_access_key_id\": \"<fixed-key>\"} then POST list-files → 200 {\"files\": [...] }","handlingStrategy":"retry","validationCode":"# validate the storage before calling list-files\nrequests.post(f\"{LS_URL}/api/storages/s3/{sid}/validate/\", headers=headers).raise_for_status()","typeGuard":"def storage_is_valid(resp):\n    return resp.status_code == 200","tryCatchPattern":"import time\nfor attempt in range(3):\n    try:\n        resp = requests.post(f\"{LS_URL}/api/storages/s3/{sid}/list-files/\", headers=headers, timeout=120)\n        resp.raise_for_status()\n        break\n    except requests.RequestException:\n        check_server_logs_for_traceback()  # real cause is logged there\n        time.sleep(2 ** attempt)","preventionTips":["Watch server logs for 'Error listing storage files' to get the provider error","Narrow the prefix to keep scans small","Handle throttling with backoff on repeated syncs","Re-validate storage after bucket renames or credential rotation"],"tags":["cloud-storage","network","timeout","storage","api"],"backgroundTag":"storage-listing-failed","analyzedSha":"0b49e9b53917880baf1dd85d574fe5541a9aafb2","analyzedAt":"2026-08-29T00:39:52.578Z","schemaVersion":2},"datasetVersion":"2026-08-29T02:17:18.158Z"}