Why does a WAF or CDN block a vulnerability scanner?

Because the WAF or CDN cannot tell from one request whether the traffic is hostile or authorised. The signals it uses (rapid request rates, repeated payload variations, fingerprintable user-agents, requests against probe paths) match both genuine attacks and legitimate scanning. Without an allowlist that names the scanner explicitly, the conservative default is to block. A scanner that gets blocked silently produces a clean-looking report that hides coverage gaps; the asset was never tested at depth, and the team only finds out when an actual attacker reaches what the scanner could not. The fix is naming the scanner upfront, not removing the WAF.

What does scanner allowlisting actually mean in operational terms?

Allowlisting is a paired set of decisions made before the scan runs. The first decision is identification: how the scanner will be recognised at every protective layer (user-agent string, source IP range, custom header, signed token, or some combination). The second decision is scope: which paths, parameters, and verbs are in scope for the scan and which are not. The third decision is rate envelope: how many requests per second the scanner is permitted to send before the protective layer treats the traffic as anomalous. All three decisions are recorded against the engagement so the verification trail survives audit and so a future tester picking up the engagement does not repeat the discovery work.

Where do scanners actually get blocked: WAF, CDN, origin, or somewhere else?

All of them, often at the same time. The CDN (Cloudflare, Akamai, Fastly, CloudFront) blocks at the edge before the request reaches the origin. The WAF (AWS WAF, Azure Front Door, Cloudflare WAF, on-prem ModSecurity) inspects request bodies and can block based on payload signatures. The reverse proxy or load balancer may impose connection limits. The application server may rate-limit per IP. The DDoS protection layer may throttle traffic that looks anomalous. A scan that runs against a stack with all five layers will hit each one in turn unless every layer carries the same allowlist record. The pragmatic discipline is mapping the protective layers before the scan, then allowlisting at each one explicitly rather than hoping the scan slips through.

What is the cost of running a scan that is partially blocked?

A partial block is more dangerous than a full block. A full block produces an obvious failure that the scanning team retests and resolves. A partial block produces a scan that completes with apparently normal output but with coverage gaps that look like clean results. The findings list looks short because the scanner could not reach the parameters that would have triggered detections; the report reads as low-risk because the depth was never achieved. The downstream cost is a false sense of assurance that survives until an attacker without the WAF in front of them does what the scanner could not. The technical detection never happened; the report just covered for it.

How do you tell whether a scanner has been blocked rather than completed cleanly?

Three signals. First, the request count in the scanner output is dramatically lower than the asset surface would justify (a hundred-page application with two thousand requests recorded is suspicious; the scanner should have produced ten times that). Second, the response body shows generic block pages, captcha challenges, or rate-limit responses rather than application content. Third, the scan duration is unusually short relative to the asset complexity. Any one signal is worth a second look; two together mean the scan was blocked at one or more layers and the output is unreliable. The verification step is comparing the scanner request log against the WAF log over the same window and confirming they agree on what was permitted.

How does SecPortal identify itself for allowlisting?

SecPortal scans go out under documented user-agent strings (SecPortal-Scanner/1.0 and SecPortal-Verifier/1.0) and reference the public scanner information page. The scanner respects domain verification: scans only run against domains the workspace has verified through DNS TXT, file upload, or HTML meta tag. A scanner allowlist that matches the user-agent and the verified domain owner records a clean, auditable rule. Security teams that prefer IP-based allowlists can layer that on top, but the user-agent and domain verification pairing is the durable identification, because IP ranges drift and verified ownership does not.

How do you write a scanner allowlist rule that is safe to keep in production?

A safe rule is narrow on three axes. First, the rule names the scanner explicitly (user-agent or signed token), not "allow security tools" generically. Second, the rule is bound to the asset that the scanner is authorised to test, not applied across the whole estate. Third, the rule has a defined lifetime: it expires at the end of the engagement window or at a fixed renewal date so it does not become a permanent backdoor. Allowlist rules without expiration are a recurring audit finding because they accumulate over years and stop matching the actual authorisation record. The discipline is treating the allowlist rule as part of the scoping artefact, not as infrastructure configuration that lives forever.

What goes in the scanner allowlist preflight checklist?

The preflight has four parts. Identification: the scanner user-agent, source IPs, and any custom token are agreed and recorded. Layer mapping: the CDN, WAF, reverse proxy, application server, and DDoS layers are listed with the team that owns each one. Scope: the in-scope hosts, paths, and request rate are documented in the engagement record so the allowlist matches the scoping artefact rather than expanding silently. Verification: a small confirmation scan runs first to verify the allowlist applies at every layer; the production scan only proceeds after the preflight scan completes cleanly. Skipping the preflight produces blocked production scans that have to be rerun; running the preflight saves the engagement clock.

How does SecPortal handle the allowlist and engagement evidence together?

SecPortal records the verified domain, the scan attestation, and the scanner identification on the engagement, so the allowlist conversation has a single source of truth to point to. The branded client portal surfaces the scan attestation to the client so the security team allowlisting at the WAF can see who authorised the test, against which assets, and for which window. The audit trail captures who authorised the scan, when each scan ran, and what the scanner output said, so the allowlist record on the WAF and the engagement record on the platform stay in sync. Teams running scans without that pairing tend to lose the audit trail when a tester rotates off or a WAF rule is renamed.

← Back to Scanner Information

Scanner guide13 min read

Vulnerability Scanner Blocking and WAF Allowlisting

A scan that gets silently blocked at the WAF, CDN, or origin produces a clean-looking report with hidden coverage gaps. The fix is identification, scope, and rate envelope agreed before the scan runs, not removing the protective layer. External scans, authenticated scans, and scheduled rescans all reach the asset through one or more protective layers. Each layer can deny the scanner without producing an obvious failure. The result is a scan that completes, generates output, and looks healthy while the actual asset surface was never tested at depth.

This guide covers how scanners get blocked, where the blocks land in a typical stack, how to allowlist a scanner without weakening production protections, what a defensible preflight checklist looks like, and how SecPortal pairs scanner identification with the engagement record so the allowlist conversation has a single source of truth.

Why scanners get blocked

Protective layers cannot tell from one request whether the traffic is hostile or authorised. The signals they use (request rate, payload variation, repeated probes against scanned paths, fingerprintable user-agents) match both attack patterns and legitimate scanning. The conservative default is denial. Without an allowlist that identifies the scanner explicitly, the scan races against rate limits, signature rules, and IP reputation systems that were tuned to stop exactly the kind of traffic the scanner is generating.

Rate-based denials

A scanner sweeping an asset at hundreds of requests per second hits the same per-IP rate limits a credential-stuffing tool would. The protective layer responds with 429 status codes, captcha challenges, or progressive backoff. The scanner sees technical responses but cannot complete the test plan inside the time budget the rate limit allows.

Signature-based denials

Many scanner payloads (SQL injection probes, XSS payloads, traversal sequences) match WAF signatures by design. The WAF blocks at the request body level. The scanner records the request as sent but never sees the application response that would have indicated whether the underlying issue exists.

Reputation and behaviour denials

Bot-management layers score traffic on header consistency, TLS fingerprints, cookie behaviour, and source reputation. A scanner without browser fidelity scores low on most of those checks and gets routed to challenge or block, often without any signal in the response body that this is what happened.

Origin denials

Some applications maintain their own per-IP throttling or auth-aware lockouts that the CDN and WAF do not see. A scan that gets past the edge can still trip an origin lockout, especially during authenticated testing where repeated session creation or password attempts look like account abuse.

Where the blocks land in a typical stack

Mapping the layers before the scan is the cheapest way to avoid blocked production scans later. Most internet-facing assets sit behind at least three layers; many sit behind five. Each layer needs the allowlist record explicitly, because protective layers do not share allowlists with each other.

Layer	What it sees	Common denial signal
CDN edge	TLS, source IP, cached vs origin route, request rate per IP, geographic origin.	Edge 403, captcha challenge, IP block, geo block.
WAF	Request URL, headers, body, payload signatures, anomaly score per request.	Signature block (403/406), rule-based denial, body inspection block.
Bot management	Header consistency, TLS fingerprint (JA3/JA4), cookie behaviour, source reputation.	Challenge interstitial, JavaScript challenge, silent throttle.
Reverse proxy	Connection count per IP, concurrent request limit, TLS version requirements.	Connection drops, 429 responses, TLS handshake failures.
Application server	Authenticated session behaviour, account lockout policy, per-IP throttling.	Account lockout, session reset, authenticated-route denials.

The allowlist record has to apply at every layer that can deny the scan. A rule on the WAF that does not propagate to the CDN, or a rule on the CDN that the bot-management layer overrides, produces partial blocking that is hard to detect and harder to debug once the scan output already exists.

Three identification options that survive review

Allowlist rules are only as strong as the identifier they pin against. A rule that allows a generic class of traffic (any request with the word scanner in the user-agent, any connection from a residential IP) is the kind of rule auditors flag because it permits traffic that was not authorised. The pattern that survives review names the scanner specifically and pairs the identifier to the engagement record.

Identifier 1: user-agent plus verified ownership

The scanner sends a documented user-agent (for SecPortal scans, that is SecPortal-Scanner/1.0 and SecPortal-Verifier/1.0 with a public reference URL). The allowlist rule matches the user-agent and is bound to the asset that the workspace has verified through domain ownership. This is the durable identifier because the user-agent does not drift between scans and the ownership record is auditable.

Identifier 2: source IP allowlist

Source IP allowlisting works when the scanning infrastructure publishes a stable IP range and the security team is willing to maintain that list. The advantage is deterministic matching at the network layer; the disadvantage is that IP ranges drift over time and an out-of-date list silently blocks legitimate scans. IP allowlists work best as a second factor on top of user-agent matching, not as the only factor.

Identifier 3: signed header or token

The scanner sends a custom header containing a signed token issued for the engagement. The protective layer verifies the signature and matches against the engagement record. This is the strongest identifier because forgery requires the signing key, but it requires header inspection at every layer that can deny. Useful for high-sensitivity assets where user-agent or IP matching is not enough.

Detecting partial blocks before they ship

Partial blocks are the dangerous case because the scan completes and the output looks normal. Three checks against the scan log catch most partial-block patterns before the report goes out.

Request volume sanity: compare the scanner request count against the asset surface. A hundred-page application should produce far more than a few thousand requests; a low ratio means most pages were never reached. Scope coverage is a function of request volume, not scan duration.
Response body inspection: check the recorded responses for block-page patterns (Cloudflare, Akamai, AWS WAF interstitials, captcha forms, generic 403/406 pages). Any cluster of those responses indicates the protective layer denied the request before it reached the application.
Cross-log reconciliation: compare the scanner request log against the WAF and CDN access logs over the same window. If the two sources disagree on what was permitted, the difference is the gap; the scan report has to acknowledge that gap rather than ignore it.

The discipline is treating partial blocks as a coverage finding in their own right rather than as a technical inconvenience. A scan that ran 30% blocked is a scan with 70% coverage, and the report has to say so. The downstream scanner coverage and limits guide covers what each scanner class actually finds when it does reach the asset, which is a different question from whether the scan reached the asset at all.

A scanner allowlist preflight checklist

The preflight runs before the production scan. Each step is cheap individually and saves a blocked production scan that has to be rerun. The scanning team owns the preflight; the security team owning the asset owns the allowlist rule. Both records close on the engagement so the verification trail is durable.

Step 1: identify and record the scanner

User-agent string and reference URL recorded on the engagement.
Source IP range (if used) listed with renewal date.
Custom token (if used) signed and bound to the engagement identifier.
Identifier copied to the security team owning the asset before the scan window opens.

Step 2: map the protective layers

CDN, WAF, bot-management, reverse proxy, application throttling listed by name.
Owning team named for each layer so the allowlist rule lands with the right reviewer.
Allowlist rule type and lifetime (engagement window or fixed expiry) agreed in writing.
Allowlist rule scope confirmed against the verified asset list, not the broader estate.

Step 3: agree the rate envelope

Maximum requests per second documented on the engagement.
Concurrency limits agreed at each protective layer.
Rate envelope kept inside the published authentication and rate limits of the asset.
Backoff policy defined so the scanner reduces rate when 429 responses appear.

Step 4: run a confirmation scan

A small, low-rate scan runs against a subset of in-scope paths to confirm allowlisting.
The scan log is reviewed for block patterns before the production scan starts.
WAF and CDN logs are checked for denials over the confirmation window.
Any layer that denied is corrected before the production scan runs, not after.

Step 5: close the allowlist after the engagement

The allowlist rule is removed or expired at the end of the engagement window.
The closure record is attached to the engagement so the audit trail is durable.
Renewals require a new attestation, not extension of the original rule.
Persistent allowlists are flagged as a recurring audit finding rather than a permanent state.

Allowlist anti-patterns that show up at audit

Three patterns recur across allowlist records that auditors flag. Each pattern starts as a convenience and turns into a control gap that survives years of engagements because nobody owns the cleanup.

The forever rule: an allowlist created for one engagement that never expires. It survives the engagement, the tester, the firm, and three rounds of WAF migrations. The rule still exists; nobody can find the attestation that authorised it. A finding waiting to happen.
The generic rule: an allowlist that matches any user-agent containing the word scanner or any IP from a residential range. Anyone can match it; the rule no longer represents the original authorisation. The audit trail does not survive review.
The asymmetric rule: the allowlist exists at the WAF but not at the CDN, or at the CDN but not at the bot-management layer. The scan partially completes; the report partially covers the asset; nobody catches the gap until the next scan reproduces it. The fix is mapping the layers upfront, not debugging the gap after the report has shipped.

How SecPortal pairs scanner identification with the engagement

SecPortal scans are bound to verified assets. A workspace cannot run an external scan against a domain it has not proven ownership of through DNS TXT record, file upload, or HTML meta tag. Each scan attaches to an attestation that records who authorised the test, against which assets, and over which window. The audit trail captures who triggered each scan, when it ran, and what the scanner output said.

The external scanning feature documents the user-agent strings used by the scanner so the security team can write a narrow allowlist rule. The domain verification feature holds the ownership record so the allowlist rule has a verified counterpart on the engagement. The authenticated scanning feature extends that pairing into the application layer for tests that go past unauthenticated coverage.

For the broader workflow, the external security assessment use case covers how the engagement record, the allowlist record, and the report deliverable stay synchronised. The domain verification and responsible scanning guide covers the upstream ownership step that the allowlist rule depends on.

Once the scan is allowlisted and runs cleanly, the next discipline is interpreting output. The vulnerability scanner false positives guide covers triage; the scan scoping and target selection guide covers the upstream scope decision that the allowlist record has to mirror; the scanner output deduplication guide covers consolidation across tools once the scans complete; and the scanner rate limiting and throttling guide covers the rate decision that has to sit under the WAF baseline so the allowlisted scan still operates inside the asset budget rather than tripping rule fires from a different angle.

Scope and limitations

Allowlisting is a coordination problem, not a technical one. The reason scans get blocked is that the team running the scan and the team owning the protective layers are separated by an organisational boundary. The fix is paperwork (attestation, engagement record, allowlist lifetime) rather than network configuration. Programmes that try to solve allowlisting by removing WAF rules or weakening CDN protections inherit the cost the next time a real attacker reaches the asset under those weakened protections.

Allowlisting also does not protect against the scanner running outside its scope. The allowlist permits the scanner to reach the in-scope assets; it does not prevent the scanner from reaching out-of-scope hosts if the scope is wrong. The discipline is keeping the engagement scope and the allowlist rule synchronised, which is what the preflight checklist is for.

Frequently Asked Questions

Run scanner allowlisting as part of the engagement, not as an afterthought

SecPortal records scan attestation, verified ownership, and scanner identification on the engagement so the allowlist conversation has a single source of truth and the audit trail survives the engagement window.

Start free See the external scan workflow