We’re examining how an analysis pipeline should handle a failed source retrieval in the FastMCP repository, a project that provides tooling around the Model Context Protocol. The requested file, src/fastmcp/prompts/prompt.py, would normally be the evidence for a code review; here, the supplied payload was an HTTP error instead. I’m Mahmoud Zalt, an AI solutions architect, and I help teams make AI systems trustworthy at their boundaries. The useful lesson is simple: input provenance is part of analysis correctness. We’ll establish why analysis must stop at a bad artifact, then build the validation, reproducibility, and operational controls that make that restraint reliable.
Establish the evidence before interpreting it
A code review failed before any Python was available to inspect. The complete supplied payload was:
404: Not FoundThe repository URL and filename are claims about the intended artifact, not proof that the artifact arrived. This response tells us nothing about prompt.pytagger’s classes, imports, behavior, complexity, or security properties. Calling the module empty, broken, or well designed would all be unsupported.
That distinction matters because downstream tooling can make nonsense look authoritative. A parser, scorer, or recommendation generator may produce an output from an error body, but polish does not create evidence. The correct result is a retrieval failure, not a semantic code report.
Reject invalid source at the acquisition boundary
The right place to stop this failure is immediately after retrieval, before parsing or analysis. A small gate prevents an HTTP error from masquerading as source code:
if response.status_code != 200 or response.text.strip() == "404: Not Found":
raise ValueError(f"Source unavailable: HTTP {response.status_code}")
source_code = response.textFailing fast is not only about speed. It prevents unsupported API descriptions, complexity scores, security findings, and refactoring advice from inheriting false authority. The status check catches ordinary retrieval failures; the body check catches the exact artifact observed here.
Those checks are necessary but not sufficient. A successful transport can still return the wrong content, such as an HTML error page, a Git LFS pointer, or an empty proxy response. Treat acquisition as three separate questions:
| Question | Check | What failure means |
|---|---|---|
| Did retrieval succeed? | Expected HTTP status | The path, branch, permissions, or request failed. |
| Does the payload resemble source? | Reject known error bodies, HTML, empty, and unexpected artifacts | Transport succeeded but returned the wrong content. |
| Can tooling accept it? | Parse as Python when syntax is required | The artifact cannot safely enter semantic analysis. |
Return structured reasons such as not_found, html_payload, and empty_payload rather than one vague analysis failure. Boundary tests should prove that a mocked 404 produces no code report, a normal Python module proceeds, and authentication, path, and non-source responses remain distinguishable.
Pin the artifact before reviewing it
Validation establishes that we received source; it does not establish that another run will retrieve the same source. The requested reference is main, a branch that can move. That makes a report difficult to reproduce even when retrieval succeeds.
Resolve the branch to a commit SHA, record that SHA, and fetch the file from that resolved revision. A branch name is a moving signpost; a commit SHA fixes the evidence. This keeps snippets, line references, findings, and later discussion attached to one artifact, and avoids content changing between validation and retrieval.
{
"ref": "<resolved-commit-sha>"
}Without this step, two engineers can use the same branch URL at different times and unknowingly assess different implementations. Reproducibility is therefore part of the same integrity boundary: first obtain the right kind of content, then make its identity stable.
Make rejected inputs actionable
A gate that fails silently merely relocates the mystery. Record the repository owner and name, normalized path, resolved revision, HTTP status, validation reason, and request duration. Do not log authorization tokens, full source text, or potentially sensitive prompt content.
Three signals make the boundary observable: source_fetch_success_ratio shows whether references yield analyzable source; source_fetch_http_status_total{status} classifies failures; and source_fetch_duration_seconds exposes retrieval latency. The proposed operating targets are at least 99% success for validated references, zero unexpected 404 responses after validation, and p95 retrieval below two seconds subject to network conditions. Alerting when unexpected 404s exceed 1% over 15 minutes can reveal a broken URL constructor or repository-layout change.
Retries should respect the failure class. A 404 is generally non-transient, so repeating the identical request is unlikely to help. Re-resolve the branch and path, then return a clear unavailable-source result if the file remains absent.
The primary lesson is that trustworthy code analysis begins with trustworthy input, not with parsing. Validate transport status, content shape, and language acceptance before semantic analysis; pin valid artifacts to immutable revisions; and measure rejection reasons so they can be fixed. Only after valid prompt.py source is retrieved should we discuss its architecture, security, performance, or maintainability. That restraint is what makes the eventual analysis credible.