How to Choose the Right AI Consultant
Choose an AI consultant on evidence, not slideware. The signals that matter: real production experience (systems shipped and operated, not just prototyped), work you can actually inspect, honesty about tradeoffs and when you do not need them, a clear plan for transferring knowledge to your team, and vendor-neutrality so their advice is not a sales pitch for one platform. Fit for your specific problem beats a famous logo every time.
I'm Mahmoud Zalt, an independent AI architect. I run advisory work through Sista AI, which also makes me one of the people you would vet using the checklist below, so treat this as the criteria I would want to be held to.
What to Look For
Weight these in roughly this order. The first two are non-negotiable.
- Production experience. Has this person shipped and operated AI systems in the real world, not just built demos? Ask what broke and how they fixed it. Real answers reveal real experience.
- Work you can inspect. Open-source projects, public writing, references from comparable work. You want evidence, not adjectives.
- Honesty about fit. A trustworthy consultant will tell you when you do not need them, or when an agency or in-house hire fits better. Willingness to talk themselves out of work is a strong signal.
- Knowledge transfer. Ask how they leave your team more capable. If the plan is to make you dependent on them, walk away.
- Vendor-neutrality. If they only ever recommend one platform, ask who pays them. Independent advice should follow your problem, not their partnerships.
Questions That Separate Good From Impressive
A polished pitch is easy. These questions get past it.
- Ask about an AI system they shipped that hit problems in production, and what happened. Vague answers mean demo experience, not production experience.
- Ask when they would tell a client not to use AI for this. Anyone who cannot answer is selling, not advising.
- Ask how your team will be better off after they leave. Listen for a concrete plan, not a promise to stay available.
- Ask how they keep AI costs under control at scale. If cost is an afterthought in the answer, it will be one in their architecture.
- Ask what they would need from you to succeed. Good consultants know the engagement is a two-way effort and can name what they need.
Why Most AI Engagements Never Pay Off, and What That Means for Vetting
This checklist is not theoretical. MIT's NANDA initiative studied 300 enterprise generative AI deployments in 2025 and found that 95 percent of pilots failed to produce a measurable financial return. The report's core finding was not that the models were weak, it was a 'learning gap': the tools and the organization never got integrated into how the business actually worked. That failure mode traces straight back to the criteria above. A consultant who has only run pilots, not operated something through that integration phase, cannot tell you what breaks when a prototype meets real workflows and real users, because they have never been there for it.
The reputational cost of skipping the evidence check is not hypothetical either. In 2025, Deloitte issued a partial refund to the Australian government after a paid report it delivered contained fabricated citations and a misattributed court quote, the result of AI-generated content that nobody on the engagement had verified closely enough. The lesson for hiring is not 'avoid AI in consulting', it is that credentials and a big name are not a substitute for asking to see the actual work and checking it. Ask any AI consultant, including a large firm, to walk you through how they verify their own AI-assisted output before it reaches you. If the answer is vague, that is the same red flag as a vague answer about a production incident.
Red Flags to Walk Away From
Some signals should end the conversation regardless of how good the pitch sounds.
- All confidence, no evidence. Big claims with nothing you can inspect and no references.
- One tool for every problem. A single-vendor recommendation before they understand your situation.
- No mention of evaluation, cost, or failure handling. These are the hard parts of production AI; skipping them signals demo-level depth.
- Reluctance to transfer knowledge. An advisor who wants you permanently dependent is optimizing for their revenue, not your outcome.
- Cannot say no. If everything you propose is 'great', you are talking to a salesperson, not an advisor.
- No verification step on their own deliverables. If they cannot describe how they check their own AI-assisted work before it reaches you, assume they do not.
Any one of these is a reason to keep looking. The right consultant will feel more like a candid partner than a vendor closing a deal.
Frequently Asked Questions
How do I evaluate an AI consultant's credibility?
Look for production experience over slideware: real systems shipped and operated, open-source or public work you can inspect, and references from comparable projects. Ask what broke in production and how they handled it. Honest, specific answers separate real experience from demo-level polish.
What questions should I ask before hiring an AI consultant?
Ask for a production system that had problems and how they fixed it, when they would advise against AI, how your team will be better off after they leave, and how they control cost at scale. The answers reveal judgment and honesty far better than a portfolio does.
Why do so many AI consulting engagements fail to deliver ROI?
Research from MIT's NANDA initiative on 300 enterprise deployments found a 95 percent pilot failure rate, driven mainly by an organizational learning gap rather than weak models: the AI was never actually integrated into how the business ran. A consultant who has only run pilots, and never carried a system through that integration, is the same risk on a smaller scale.
Does the price tell me anything about quality?
Only loosely. A very low rate can signal thin experience, but a high one does not guarantee fit. Judge on evidence and fit for your problem, then treat rate as a secondary filter once the shortlist is credible.
Should I choose a big-name firm or an independent consultant?
It depends on the job. A big firm offers capacity and process; an independent offers direct senior access, lower cost, and flexibility. For strategy, architecture, and focused work, an independent with real production experience is often the stronger choice. Firm size is also no guarantee against errors slipping through, so ask any candidate, regardless of size, how they verify their own deliverables.
Choosing Well
The right AI consultant is the one who shows evidence over adjectives, tells you the truth about fit, leaves your team stronger, and gives advice that follows your problem rather than their partnerships. Run the questions, watch for the red flags, and weight production experience above everything else. The industry data backs this up: pilots fail on integration and verification, not on model choice, so hire for the consultant who has actually lived through that integration before.
If you want to hold a real conversation against exactly these criteria, that is what my AI Consultancy service is set up for. Ask me the hard questions above; I would rather earn the engagement than win a pitch.
