HEAD is a lightweight HTTPS probe that requests a target’s root path while collecting response headers and an HTTP status note, without retrieving the response body. In reconnaissance, it provides a low-footprint observation step after hostname discovery and resolution, helping keep traffic minimal while producing evidence that can be recorded and reviewed. section 4.3.3 section 4.0
A practical HEAD stage has a narrow contract: send HEAD / with a declared User-Agent and timeout; normalize returned header names to lowercase; return a header dictionary plus notes containing either status information or an error type; and close the connection even when probing fails. This makes expected network failures observable rather than silently discarded. section 4.3.3
Response headers become useful pipeline inputs when they are handled as structured observations:
- Record the status code alongside the headers so the result retains basic request context. section 4.3.3
- Inspect header names and values for broad WAF/CDN signatures; treat a match as an early warning, not definitive profiling, because the heuristic favors false positives. section 4.3.3
- Sanitize sensitive values such as Set-Cookie, Authorization, and X-API-Key before persistence. section 4.7
- Append timestamped observations to structured JSONL artifacts so later stages can query, audit, replay, or extend the work. section 4.0 section 4.7
Minimal safe probe
For each resolved candidate hostname, issue one HTTPS HEAD request to /. Store lowercase response headers and a note such as status:200; if the request fails, store an error note instead. Run a WAF/CDN header heuristic on the observation, redact sensitive header values, and persist the resulting record. Content retrieval, such as GET /robots.txt, remains separately gated to preserve the default lightweight behavior. section 4.3.3 section 4.7
Listing 4.5 HTTPS probing for header extraction
def https_head(host: str, timeout: int = 4, user_agent: str = "ReconSnap/1.0") -> Tuple[Dict[str, str], List[str]]:
"""Send a safe HTTPS HEAD / and return (headers, notes)."""
headers: Dict[str, str] = {}
notes: List[str] = []
try:
ctx = ssl.create_default_context()
conn = http.client.HTTPSConnection(host, 443, context=ctx, timeout=timeout)
conn.request("HEAD", "/", headers={"User-Agent": user_agent})
res = conn.getresponse()
headers = {k.lower(): v for k, v in res.getheaders()}
notes.append(f"status:{res.status}")
except Exception as e:
notes.append(f"error:{type(e).__name__}")
finally:
try:
conn.close() # type: ignore[has-type]
except Exception:
pass
return headers, notes
The implementation shows the probe’s operational boundary: HTTPS on port 443, HEAD /, configurable timeout and User-Agent, lowercase headers, status/error notes, and cleanup in finally. section 4.3.3
HEAD is low-impact rather than fully passive: it touches the target, unlike techniques that rely entirely on third-party data. Use it only within configured scope and operational safeguards; block and log out-of-scope requests, retain human approval where required, and keep every action explainable and auditable. section 4.7
