ScrapeGraphAI/Scrapegraph-ai · error · DeepCopyError

Cannot deep copy object of type {type(obj)}

Error message

Cannot deep copy object of type {type(obj)}

What it means

DeepCopyError raised by safe_deepcopy when an exception occurs while deep-copying an object (e.g. unpicklable locks, generators, or objects with broken __deepcopy__). The original exception is chained. safe_deepcopy already special-cases boto3 clients and falls back to copy.copy, so this error means even the fallback path raised.

Source

Thrown at scrapegraphai/utils/copy.py:71

        if isinstance(obj, (list, set)):
            return type(obj)(safe_deepcopy(v) for v in obj)

        if isinstance(obj, dict):
            return {k: safe_deepcopy(v) for k, v in obj.items()}

        if isinstance(obj, tuple):
            return tuple(safe_deepcopy(v) for v in obj)

        if isinstance(obj, frozenset):
            return frozenset(safe_deepcopy(v) for v in obj)

        if is_boto3_client(obj):
            return obj

        return copy.copy(obj)

    except Exception as e:
        raise DeepCopyError(f"Cannot deep copy object of type {type(obj)}") from e

View on GitHub (pinned to 532dfffbf6)

Solutions

1. Remove non-copyable live resources (clients, sessions, locks) from the config passed to the graph; create them inside nodes
2. Have the object implement __deepcopy__ returning self or a proper copy
3. Catch DeepCopyError and construct a fresh instance instead of copying

Example fix

# before
graph = SmartScraperGraph(prompt=..., config={"client": boto3.client("s3"), ...})
# after
config = {"aws": {"region": "us-east-1"}}  # plain data only
graph = SmartScraperGraph(prompt=..., config=config)

def copyable(cfg):
    return all(not hasattr(v, "aclose") and not isinstance(v, (threading.Lock, type(None), (int, float, str, bool, list, dict))) is False or True for v in []) # simple: keep config JSON-like
import json
json.dumps(config)  # raises early if config holds non-serializable live objects

def is_safe_to_copy(obj) -> bool:
    try:
        copy.deepcopy(obj)
        return True
    except Exception:
        return False

from scrapegraphai.utils.copy import DeepCopyError
try:
    cfg = safe_deepcopy(config)
except DeepCopyError:
    cfg = copy.copy(config)  # or rebuild from plain data

When it happens

Trigger: Passing objects like open sockets, threads/locks, generators, or objects whose __deepcopy__/__reduce__ raise to safe_deepcopy (used in graph __init__ when copying configs).

Common situations: Putting a live HTTP client, browser session, or DB connection inside a graph config dict that gets deep-copied at graph construction.

Related errors

• The 'graphviz' library is required for this functionality. P
• The browserbase module is not installed. Please install it u
• Semantic analysis failed: {str(e)}
• Validation code generation failed: {str(e)}
• Unsupported service tier {service_tier!r} for {model_name}

AI-assisted analysis of ScrapeGraphAI/Scrapegraph-ai@532dfffbf6 (2026-08-28). Data as JSON: /api/errors/98b5d5acd171df3e. Report an issue: GitHub.