stamparm/maltrail · error · RuntimeError

( )

Error message

%s (%s): %s

What it means

In check_redistribution.py, main() iterates SOURCES (provider, url, kind) and calls fetch(url) for each redistribution-list provider. Any exception raised while fetching (network failure, DNS error, HTTP error, TLS problem) is caught and re-raised as a RuntimeError formatted '%s (%s): %s' — provider name, URL, and the underlying exception — so the caller knows exactly which source could not be retrieved and why.

Solutions

1. Read the inner exception in the message to identify the cause (connection refused, timeout, HTTP status) and fix network access or proxy settings.
2. Verify the provider URL in SOURCES is still valid — open it in a browser or curl it, and update it if the provider moved its feed.
3. Re-run later if the failure is a transient upstream outage; consider adding retry logic around fetch.
4. In restricted environments, run from a machine with egress access or pre-download the lists and adapt fetch to read from local files.

import urllib.request
def reachable(url):
    try:
        with urllib.request.urlopen(url, timeout=10) as r:
            return r.status == 200
    except Exception:
        return False

for provider, url, kind in SOURCES:
    if not reachable(url):
        print(f"skipping {provider}: {url} unreachable")

try:
    body = fetch(url)
except RuntimeError as ex:
    provider, url, cause = ex.args[0] if False else str(ex).split(' (', 1)
    log.warning("source fetch failed, will retry/fallback: %s", ex)
    body = cached_or_skip(provider)

When it happens

Trigger: fetch(url) raises any Exception for one of the SOURCES entries: the URL is unreachable, DNS resolution fails, the server returns a non-200 status that fetch turns into an exception, the connection times out, or TLS handshake fails.

Common situations: Running the check in a CI container or air-gapped network without internet access; a provider changing or deprecating its URL; a corporate proxy or firewall blocking the host; transient upstream outage of a blocklist provider (e.g. a CDN or government feed being temporarily down).

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• empty response from
• one of the two matched
• just probed
• bind
• connect

AI-assisted analysis of stamparm/maltrail@77cfb06d76 (2026-09-13). Data as JSON: /api/errors/83aa777eb48cdede. Report an issue: GitHub.

Appendix: source

Thrown at sensor/tools/check_redistribution.py:103

# Piles whose whole purpose is to name shared infrastructure. update_trails() spares them and so
# does this: reporting them would be reporting the intent.
EXEMPT_INFO = ("parking", "sinkhole", "cdn", "mass scanner")


def fetch(url):
    request = Request(url, headers={"User-agent": "maltrail-redistribution-check"})
    return urlopen(request, timeout=TIMEOUT).read().decode("utf8", "replace")


def networks():
    """{provider: [ip_network, ...]} from the providers' own published lists."""

    retval = {}
    for provider, url, kind in SOURCES:
        try:
            body = fetch(url)
        except Exception as ex:
            raise RuntimeError("%s (%s): %s" % (provider, url, ex))

        prefixes = []
        if kind == "lines":
            prefixes = [_.strip() for _ in body.splitlines() if _.strip()]
        elif kind == "aws":
            data = json.loads(body)
            prefixes = [_["ip_prefix"] for _ in data.get("prefixes", [])
                        if _.get("service") in AWS_SHARED_SERVICES]
            prefixes += [_["ipv6_prefix"] for _ in data.get("ipv6_prefixes", [])
                         if _.get("service") in AWS_SHARED_SERVICES]
        elif kind == "gcp":
            data = json.loads(body)
            for entry in data.get("prefixes", []):
                prefixes.append(entry.get("ipv4Prefix") or entry.get("ipv6Prefix"))
        elif kind == "fastly":
            data = json.loads(body)
            prefixes = list(data.get("addresses", [])) + list(data.get("ipv6_addresses", []))

View on GitHub (pinned to 77cfb06d76)