AI Architecture Review - Senior Audit of Your AI System

You have an AI system in your codebase. Maybe it is six months old, maybe it is three weeks old. It works in demos. It half-works in production. The team has questions about whether the architecture will hold up at the next traffic tier, whether the cost shape is sustainable, whether the failure modes are acceptable, and whether the design choices made early are going to limit what you can build next quarter. You want a senior outside read before more is built on top of it. That is what an AI architecture review is for.

I review AI architectures as a practitioner, not as a generalist consultant. I have shipped LLM applications, agents, RAG systems, MCP integrations, and multi-agent orchestrations in production. A focused review surfaces three to five things worth changing, two or three worth keeping despite the team being unsure, and a clear-eyed read of where the system will break if it stays on its current trajectory. The deliverable is a written brief plus a working session with the team, not a deck of generic recommendations.

When An Architecture Review Is The Right Move

The review is most valuable at one of three inflection points. First, before scaling: the prototype works, the team is about to harden it for general availability, and a senior outside review catches the design choices that will hurt later. Second, after pain: production incidents are recurring, the team has a theory, but they want validation from someone who has seen these failure modes elsewhere. Third, before a fundraise or acquisition: technical due diligence is coming, and the architecture needs to stand up to investor or acquirer scrutiny.

Pre-scale: prototype works, GA in 4-12 weeks, team wants a sanity check before hardening
Post-incident: production has broken twice, root cause is suspected, team wants outside calibration
Pre-fundraise: technical due diligence is imminent, founder cannot defend architecture choices alone
Pre-rebuild: the team thinks a rewrite is needed, leadership wants a second opinion on scope
Hiring a new senior engineer or AI lead: review brief becomes the onboarding doc
Buying vs building: the team is about to commit to a vendor or platform, wants outside view on lock-in
Skip the review when: the system has not shipped yet (do design work instead), or when the team already has a senior in-house and just needs a sanity check (use the single-session Q&A)

What The Review Actually Looks At

A complete review touches every layer of an AI system: data, retrieval, model, prompt, tool surface, orchestration, evaluation, observability, cost, latency, security, and operational discipline. Most reviews find disproportionate issues in three or four layers. The senior judgment is knowing which to dig deeper on and which to leave alone.

Orchestration topology: single agent, supervisor, swarm, sequential workflow, routing, parallel fan-out
State and memory model: short-term scratchpad, long-term memory, shared state across agents, compaction policy
Tool surface: schemas, names, descriptions, count, error semantics, idempotency, MCP integration
Prompt architecture: versioning, templating, few-shot exemplars, system prompt strategy, per-tenant overrides
Retrieval pipeline: chunking, embedding, hybrid search, reranking, query rewriting, metadata filters
Model routing: which model on which hop, fallback strategy, batch vs realtime, caching policy
Evaluation harness: dataset coverage, metric design, CI integration, online sampling, drift detection
Observability stack: tracing, cost dashboards, latency budgets, failure classification, alerting
Cost shape: per-request economics, cache hit rate, model mix, batch opportunities
Latency: time to first token, time to completion, per-node budgets, streaming UX
Failure handling: retry policies, idempotency, dead-letter queue, circuit breakers, graceful degradation
Security: prompt injection surface, tool authorization, output filtering, audit logging, PII handling
Operational discipline: on-call, runbooks, deploy gates, rollback paths, feature flags

The Failure Modes I See Most Often

After enough reviews, the same patterns recur. None of these are model problems. They are architecture problems.

No evaluation harness: the team cannot tell whether a change is an improvement, so prompt changes are coin flips
Tool overload: 40+ tools in one agent, selection accuracy degrades, team blames the model
Multi-agent before single-agent works: complexity added before the simple version was instrumented
Naive RAG: top-k vector only, no hybrid, no rerank, no query rewriting, retrieval recall is the bottleneck
Unbounded loops: agent has no step or cost ceiling, single bad request burns thousands of dollars
No fallback model: provider outage takes the feature down completely
Prompts as wiki strings: not in source control, no version history, no review, no eval gate
State management bugs: per LangChain 2026 report, 60% of production agent incidents are state, not model
Cost shape unknown: team is shocked by the monthly bill because per-request cost is not measured
No audit log: regulated deployment will fail compliance review
Confused deputy: tools execute with server identity, not user identity, authorization is in the LLM layer
Prompt injection surface ignored: untrusted strings flow into prompts without sanitization

Engagement Shapes

Three engagement shapes cover most of what teams need. The single-session Q&A is the cheapest entry point and the right shape when the architecture is well-documented and the team can self-serve next steps. The written review is the most common, deeper, with a brief and follow-up working sessions. The full audit is for serious decisions and pre-fundraise diligence, with code-level review and reproduction of pain points.

Single-session Q&A (90 minutes): team presents the architecture, I ask, push back, and recommend. Notes go in writing within 24 hours. Cheapest, fastest, requires the team to bring clean docs
Written review (1-2 weeks): I review docs, code, dashboards, traces. Brief includes risks, recommendations, prioritization. One working session at delivery, one follow-up two weeks later
Full audit (3-6 weeks): everything in the written review plus reproduction of incidents, threat modeling, cost-shape analysis, evaluation harness review, and a written remediation plan with owners and dates
Continuous advisory (monthly retainer): 4-12 hours/month for ongoing architecture review as the system evolves, design review on new modules, on-call escalation for hard calls
Pre-fundraise diligence package: written audit plus an interview-ready architecture deck and an investor Q&A doc
Acquisition diligence: same as full audit, scoped to the questions an acquirer will ask, plus a redacted version for sharing

What You Send Me Before The Review

A good review depends on the inputs. The more the team prepares, the more depth the review can reach. The minimum is enough for me to be useful in a 90-minute conversation. The maximum gives me everything needed for a written brief.

Architecture diagram (whiteboard, Excalidraw, Miro, all fine)
A short written brief: what the system does, who uses it, current scale, current pain
Repository access (read-only) or sample of the key files: prompt files, agent definitions, tool definitions, retrieval code
Eval results, if any exist
Observability traces: 5-20 representative production traces from LangSmith, Langfuse, or wherever you trace
Cost dashboard screenshot, monthly spend, per-request cost if known
Top three open questions or pain points the team most wants addressed
Recent incident postmortems, if any
Constraints: regulatory (HIPAA, SOC2, GDPR), latency budget, target cost ceiling

What You Get Out

The deliverables match the engagement shape. The unifying theme is that everything is written, every recommendation has a justification, and every change has a rough sizing.

Risk register: ranked list of architectural risks with severity, likelihood, and mitigation
Recommendations: 5-15 concrete changes with rationale, estimated effort, expected impact
Keeps list: 2-5 things the team is unsure about that I would not change
Eval gaps: where the current eval harness is missing coverage, with proposed test cases
Cost optimization: the 2-3 highest-leverage cost reductions, usually 20-50% wins
Threat model: prompt injection, tool authorization, data exfiltration, output trust, with mitigations
Roadmap input: priority order for the next 90 days of architecture work
Working sessions: live walk-through of the brief with the team, time for pushback and clarification
Follow-up: a check-in two to four weeks later to validate that the highest-priority changes shipped

Who I Work Best With

I am useful to teams that already have a working AI system and want a senior outside read. I am less useful to teams that have not shipped anything and want help deciding what to build first. The review is a debugging tool, not a strategy session.

Engineering teams of 3-30 with at least one shipped AI feature in production
CTOs and VPs of Engineering who want a senior outside calibration before a board update
Founders preparing for technical due diligence in a fundraise
Acquirers evaluating an AI startup target
Internal AI platform teams justifying architecture choices to a steering committee
I am not the right fit for: pre-build strategy work (use a different engagement), training and education only (use the speaker page), or generic process consulting

FAQ

How long does an AI architecture review take?

A single-session Q&A is 90 minutes plus written notes within 24 hours. A written review is one to two weeks. A full audit is three to six weeks. Pre-fundraise diligence packages are usually two to three weeks.

Do you review the code, the docs, or both?

Both. Docs and diagrams set the architecture story. Code, prompts, and production traces reveal whether the architecture matches reality. The most useful reviews look at both, with read-only repo access and a sample of recent production traces.

Can the review be remote?

Yes. All my architecture reviews are remote-first. Async repo and doc access plus video working sessions. On-site available for full audits if the team wants a multi-day deep dive.

What do you charge for an architecture review?

Single-session Q&A is the entry tier. Written reviews and full audits are scoped per engagement based on system size and review depth. I scope after a 30-minute intro call where we confirm the shape that fits your problem.

Do you sign NDAs?

Yes. Mutual NDA before any code, prompts, or production traces are shared. Standard work-for-hire assignment of any deliverables. References for past clients available on request, redacted as needed.

What do you not review?

Pure ML training pipelines, large-scale distributed training, GPU cluster operations, and traditional non-LLM ML systems. I review LLM applications, agents, RAG systems, and the production engineering around them. If your problem is model training rather than model deployment, I am the wrong reviewer.

Will you recommend a rewrite?

Rarely. Rewrites are usually the wrong move because they discard hard-won production learning. I recommend rewrites only when the architectural foundation is so wrong that incremental fixes cost more than rebuilding, which happens in fewer than 10% of reviews.

Can you stay on after the review to help execute?

Yes, on a continuous advisory retainer or a focused execution engagement. Many of my engagements start as a written review and evolve into ongoing advisory or hands-on agent build work.

Next step

Your situation isn't generic.
Neither should the conversation be.

A short call to map what ai architecture review looks like for your team. No obligation, no pitch, just clarity.

Review my AI architecture See Q&A Session

Senior architect · 16+ years shipping · Direct, no agency layers

AI Architecture Review