Methodology
How HeaderLens decides whether a website is up. Version 1.0, published 2026-10-12. Machine-readable version.
Scope
Each check runs one server-side probe from EVERY configured region at the same moment (one observation per region) and combines them into a consensus. It is still not a measurement of every network worldwide and may differ from what you see on your own network.
Multi-region consensus
- single_region: only one region reported; its classification is used.
- unanimous: all conclusive regions agree.
- partial_outage: some regions saw "down" while others got a response: consensus is "degraded" (regional outage).
- majority: mixed non-down results; the most common classification wins, ties go to the more severe one.
- no_conclusive_data: every reporting region was inconclusive.
- Regions that fail to report (agent unreachable) are listed in unavailableRegions and do not vote.
Procedure
- Normalise the input to a public hostname. Paths, queries, fragments, credentials, custom ports and IP literals are rejected or discarded.
- Resolve DNS. Every returned address must be a globally routable public address; otherwise the check is refused (HTTP 422).
- Connect to https://{hostname}/ on port 443, pinned to the validated address, with SNI and certificate validation against the original hostname.
- If HTTPS cannot be established, try http://{hostname}/ on port 80.
- Follow up to 5 redirects. Every redirect target is re-validated with the same rules (scheme, port, hostname, resolved address).
- Read at most 128 KiB of the final response body, used only to detect anti-bot challenge pages. Bodies are not stored.
- Classify the evidence with the deterministic rules below and store the observation.
Classifications
| Value | Meaning |
|---|---|
Up up | A normal 2xx/3xx response was received within the slow threshold. |
Reachable, but blocked our probe reachable_restricted | The server answered but refused our automated probe (401, 403, 407, 429 or an anti-bot challenge). The site is very likely up for ordinary visitors. |
Degraded degraded | The server answered but something is wrong: invalid TLS, a 4xx home page, too many redirects, a failing redirect target, or a response slower than the threshold. |
Down down | No usable response: the domain does not resolve, connections are refused or time out, or the server returned 5xx. |
Inconclusive inconclusive | Our probe could not reach a reliable conclusion (for example a DNS resolver failure on our side, or a redirect to a destination we refuse to follow). |
Reachability
| Value | Meaning |
|---|---|
reachable | A TCP/TLS connection succeeded and an HTTP response was received. |
unreachable | DNS or the network connection failed. |
indeterminate | We could not determine reachability. |
Access
| Value | Meaning |
|---|---|
normal | The response was served without access restrictions. |
authentication_required | HTTP 401 or 407. |
forbidden | HTTP 403 without recognisable challenge markers. |
rate_limited | HTTP 429. |
automated_client_restricted | A recognisable anti-bot challenge (e.g. cf-mitigated: challenge) was served. |
not_applicable | No HTTP response was received. |
Limits
- DNS timeout 5000 ms, per-request timeout 8000 ms, total 20000 ms.
- At most 5 redirects; ports 80 and 443 only.
- At most 128 KiB of body read; responses slower than 5 s are “degraded”.
Our probe
HeaderLensBot/1.0. The probe identifies itself honestly and does not attempt to evade bot protection.
API status codes
The API returns 200 with classification "down" when a target is down. 400 = malformed request, 422 = invalid or prohibited target, 429 = you are rate-limited, 503 = our probe infrastructure could not take a measurement.
Availability figures
History rollups report (up + reachable_restricted + degraded) ÷ (all checks − inconclusive) over 24 hours, 7 days and 30 days. They are based only on checks people happened to run, not continuous monitoring.