FQDN Risk Scoring

Module: core/fqdn_risk.py · Used by: SecurityShield._check_links (requests) and sanitize_response (non-streaming responses) · Default: disabled

Scores the host of each URL from the text of the hostname alone and, at or above a threshold, refuses the request or replaces the link in the response. It adds to the static blocked_domains list; it does not replace it.

What it is

The feature names and the constant lists come from fqdn-model, a domain classifier by the same author. Only the features that can be computed from the hostname string are used here. The module makes no network request: no DNS, TLS, HTTP or WHOIS lookup. It is not fqdn-model's trained classifier; the score is a fixed weighted sum, and the weights are this project's own.

A score therefore says that a hostname looks like the hostnames used for phishing or generated by malware. It says nothing about what the host serves.

Features

Feature	Fires when	Weight
ip_as_host	the host is an IP literal: dotted IPv4, IPv6, or an integer in decimal or hex	0.35
private_ip	that address is loopback, private, link-local or reserved (added to ip_as_host)	0.45
risky_tld	the host ends in .xyz .top .loan .online .club .click .icu .cn	0.30
shortener	the host is bit.ly t.co tinyurl.com ow.ly is.gd buff.ly adf.ly	0.25
suspicious_keyword	the host contains one of 13 words (login signin account verify secure update bank payment free download admin password credential)	0.20 per distinct word, at most 0.60
punycode	the host contains xn--	0.25
excess_hyphens	more than 1 hyphen	0.20
excess_digits	more than 4 digits	0.20
excess_subdomains	more than 3 subdomain labels, not counting a leading www	0.20
long_domain	longer than 20 characters	0.08

The score is the sum of the weights that fired, capped at 1.0. An IP literal is scored on ip_as_host and private_ip only; the other features are skipped for it.

Examples, with the default threshold of 0.7:

Host	Score	Features
s3.amazonaws.com	0.00	none
raw.githubusercontent.com	0.08	long_domain
login.microsoftonline.com	0.28	suspicious_keyword, long_domain
example.xyz	0.30	risky_tld
8.8.8.8	0.35	ip_as_host
127.0.0.1, 192.168.1.10	0.80	ip_as_host, private_ip
paypal-login-verify-account.com	0.88	suspicious_keyword (3), excess_hyphens, long_domain
secure-login-verify.xyz	1.00	risky_tld, suspicious_keyword (3), excess_hyphens, long_domain

A link to a loopback or private address is the one case a single kind of signal reaches the threshold. Every other feature needs to combine with others.

Configuration

security:
  link_sanitization:
    enabled: true
    blocked_domains: [...]        # static deny-list
    risk_scoring:
      enabled: false              # default
      block_threshold: 0.7        # 0..1; lower = stricter
      log_only: false             # true = log the score, never block or replace

When enabled, a host scoring block_threshold or more:

• in a request: the request is refused with 403 and the message Unsafe link detected (risk=<score>): <url>, unless log_only. The check covers all text the shield inspects and runs after the injection checks. It does not run when link_sanitization.enabled is false;
• in a non-streaming response: the URL is replaced with [BLOCKED_LINK], unless log_only. Streamed responses are not checked. The response check does not consult link_sanitization.enabled.

Every hit is written to the process log at warning level, as FQDN RISK: <host> score=<s> reasons=[...] for a request and FQDN RISK (response): ... for a response.

On an error in the scorer the two paths differ. In a request the link is skipped and the request goes on. In a response the URL is replaced with [BLOCKED_LINK], unless security.link_sanitization.fail_open is true.

Tests

tests/test_fqdn_risk.py: a list of ordinary hosts that must stay below the threshold, a list of phishing-style hosts that must reach it, each feature on its own (IP literal forms, subdomain counting with two-level TLDs, keyword, shortener and TLD matching), and the request and response paths of the shield. These are regression tests on hand-written lists, not a measurement of detection on real traffic.