A security header that is configured on the origin server is not necessarily the same header that reaches users. Between your origin configuration and the browser there may be a CDN, a load balancer, a reverse proxy, a WAF, and application-level middleware, any of which can modify, strip, or add HTTP headers. Verifying security headers means checking the response as it arrives at the client, not just as it leaves the origin.
This article covers the common failure modes that cause configured headers to not reach users, and practical methods to verify that your intended configuration is actually delivered.
The Gap Between Configuration and Delivery
The most common scenario: a developer adds security headers to the web server configuration, tests by hitting the origin directly, and confirms the headers are present. The configuration is then deployed behind a CDN. The CDN serves a cached version of the page that was fetched before the header change was applied, or the CDN has an edge rule that strips security headers for compatibility reasons.
Less common but real: a reverse proxy is configured to add security headers, but the application framework also sets headers, and the last writer wins. If the application sets a less restrictive CSP for a particular route, the proxy's more restrictive header is silently overwritten.
Another scenario: HSTS is configured on the primary domain but not on subdomains. A scan of the root domain shows HSTS present. A login page served on auth.example.com does not have it. The HSTS check passes for the domain but the subdomain is vulnerable.
Verifying with curl
curl is the most direct way to inspect the actual response headers reaching a client. The -I flag requests only headers (HEAD request):
# Check headers on the root document
curl -sI https://example.com/
# Follow redirects and show final headers
curl -sIL https://example.com/
# Check a specific path (check the app, not just the root)
curl -sI https://example.com/login
# Check headers without CDN caching (add a cache-busting param)
curl -sI "https://example.com/?_nocache=$(date +%s)"
# Request a response with a specific User-Agent (some CDNs vary by UA)
curl -sI -A "Mozilla/5.0 (compatible; SecurityAudit/1.0)" https://example.com/Running the check with a fresh cache-busting parameter is useful when debugging CDN issues, as some CDNs cache responses including the headers at the time of first fetch.
To grep for a specific header:
curl -sI https://example.com/ | grep -i "strict-transport-security|content-security-policy|x-frame-options|x-content-type-options|referrer-policy|permissions-policy"CDN and Proxy Complications
CDN behavior around response headers varies by provider and configuration. Common issues:
- Header stripping at the edge: Some CDN configurations include rules that remove headers not in an allowlist. If you add a new security header and it is not in the CDN's passthrough allowlist, it will be stripped at every edge node.
- Cached responses served without updated headers: After adding a security header, the CDN cache may contain old responses that lack the header. These are served to users until the cache entries expire or are explicitly purged.
- Header normalization: Some proxies modify header values. A CDN may normalize or truncate a CSP header it does not understand, silently altering the policy.
- Different behavior by edge location: CDN edge nodes may have different configurations if the change was not rolled out globally. Checking from one location is not sufficient to confirm the change is deployed everywhere.
After configuring headers, purge the CDN cache for the affected paths. Then verify from a location that will hit an edge node rather than the origin directly. If your CDN provides an X-Cache or CF-Cache-Status (Cloudflare) response header, confirm the response is being served from the edge, not fetched fresh from origin.
Per-Route Header Overrides
Web frameworks and servers often allow headers to be set at multiple levels: global middleware, route middleware, and individual route handlers. If a route handler explicitly sets a header, it may override the global middleware value. If the route handler sets a more permissive CSP (for example, to allow inline scripts for a legacy page), the global policy is not in effect for that route.
Check the headers on the specific paths that matter: the login page, the checkout page, any page that handles sensitive user data. Do not assume the root path headers represent every route in the application.
# Check multiple paths in a loop
for path in "/" "/login" "/account" "/checkout" "/api/v1/session"; do
echo "--- $path ---"
curl -sI "https://example.com$path" | grep -i "content-security-policy|strict-transport-security"
doneBrowser Developer Tools
Browser developer tools show the actual headers as the browser received them, which is the ground truth for what a user's browser sees. Open the Network tab, navigate to the page, and click on the document request (the first HTML response). The Response Headers section shows every header the browser received.
Browser tools are useful for debugging CSP violations. Any resource blocked by the CSP appears as a console error with the specific directive that blocked it, the resource URL, and the policy that was active. This helps confirm the policy is being enforced, not just served.
One limitation: browser tools do not easily let you test from different network locations, User-Agent strings, or cookie states. curl is more flexible for targeted verification.
Automated Header Scanning
For ongoing verification, automate header checks as part of a deployment pipeline or monitoring routine. A simple approach is a shell script or CI job that fetches the headers for key URLs and asserts the expected headers are present with the expected values.
#!/bin/sh
# Basic header assertion for CI/CD pipelines
URL="https://example.com/"
HEADERS=$(curl -sI "$URL")
check_header() {
local name="$1"
local expected="$2"
if echo "$HEADERS" | grep -qi "$expected"; then
echo "PASS: $name"
else
echo "FAIL: $name not found or value incorrect"
exit 1
fi
}
check_header "HSTS" "strict-transport-security: max-age="
check_header "X-Content-Type-Options" "x-content-type-options: nosniff"
check_header "X-Frame-Options" "x-frame-options: deny"
check_header "Referrer-Policy" "referrer-policy:"
check_header "CSP" "content-security-policy:"How WebDefect Verifies Headers
WebDefect requests each page it discovers as a browser-like HTTP client. It follows redirects and records headers from the final response, not from any intermediary redirect. For sites served behind CDNs, the scan requests go through the CDN edge layer, so the headers recorded are the headers that actually reach clients rather than origin-only headers.
When scanning a domain, WebDefect checks for security headers on:
- The root document response.
- All HTML pages discovered through link crawling.
- The HTTPS version of the site specifically (the header check requires an HTTPS response to be meaningful).
If a header is present on some pages but missing on others, the finding is recorded for the specific pages where it is absent. This surfaces per-route override issues that a root-only check would miss.
What to Check After Each Deployment
After any deployment that modifies security headers, server configuration, or CDN rules, verify:
- Root document at
https://example.com/(confirms basic configuration). - At least one authenticated or sensitive path (confirms no per-route override is removing the headers where they matter most).
- A cache-busted request (confirms the CDN is not serving stale headers from before the change).
- The
X-Cacheor equivalent header to confirm the response is being served from the edge, not fresh from origin (confirms the CDN is involved). - The HTTP to HTTPS redirect (confirms HSTS and CSP are only being evaluated on the HTTPS response, as intended).