The Short Answer: What to Probe Before You Hire
Before hiring an AI consultant, probe five things: domain fit for your problem, a real production track record (not demos), a clear pricing model, concrete data and security practices, and a defined handover plan. The right answers are specific, honest about limits, and backed by shipped work you can verify.
I am Mahmoud Zalt, an AI Architect and technical advisor with 16+ years building production systems since 2010. I created Laradock.io (2M+ downloads) and the Apiato framework, founded Sista AI, and have mentored 60+ engineers across EMEA and North America. I have been on both sides of this table: the buyer evaluating vendors and the consultant being evaluated. This guide is written honestly from the buyer's side, because the questions that protect you are the same ones a good AI consultant actually wants you to ask.
Why the Right Questions Matter More Than the Pitch
AI is the easiest field in tech to fake competence in right now. A polished deck, a demo wired to a single happy path, and fluent buzzwords can hide the fact that nothing has ever survived real traffic or real data. The gap between a working demo and a production system is where most AI budgets quietly disappear.
Industry surveys consistently report that a large majority of AI initiatives never reach production or fail to deliver measurable value. The common thread is rarely the model. It is poor scoping, unclear ownership, weak data handling, and no plan for what happens after the consultant leaves. Good questions surface those risks before you sign.
The framing below groups twelve questions into four areas: expertise and track record, process and delivery, pricing and terms, and risk and handover. For each one I describe what a strong answer sounds like and the red flag that should make you slow down. Apply the same checklist to me when you reach out through my AI consulting page.
Group 1: Expertise and Track Record
Start here. If the foundation is shaky, nothing else matters. You are trying to separate people who have shipped AI into production from people who have read about it.
1. Can you show me an AI system you built that is running in production today?
A strong answer names a specific system, the problem it solved, roughly how many users or requests it handles, and what broke along the way. Demos prove an idea; production proves competence. The red flag is a consultant who only shows prototypes, hackathon projects, or screenshots, and deflects when you ask what is live and serving real traffic.
2. Have you solved a problem in my domain or with my data type before?
AI for legal documents, medical records, e-commerce search, and customer support are very different problems with different failure modes. A good consultant either shows directly relevant work or is honest that your domain is new to them and explains how they will de-risk it. The red flag is someone who claims every domain is the same or treats your specific constraints as an afterthought.
3. When is AI the wrong tool, and would you tell me to not build it?
This is my favorite question to be asked. The strongest consultants will talk you out of AI when a simple rule, a SQL query, or an off-the-shelf tool would do the job cheaper. That honesty is the signal you want. The red flag is someone who thinks AI is the answer to every question you have, because they are selling hours, not outcomes.
Group 2: Process and Delivery
Talent without a process produces impressive prototypes that never ship. These questions test whether the engagement is structured to actually deliver something you can run.
4. How do you scope a project, and what does the first milestone look like?
A good answer starts small: a discovery phase, a clearly defined first deliverable, and a checkpoint where you decide whether to continue. I treat AI projects like architecture reviews: diagnose first, build second. The red flag is a giant fixed scope with one big payment at the end and no early proof point you can evaluate.
5. How will we measure whether this is working?
Real AI work needs evaluation: accuracy targets, latency budgets, cost per request, and a way to catch regressions. A strong consultant defines success metrics before writing code and builds a way to test against them. The red flag is vague language like "it will feel smart" with no measurable definition of done.
6. What does your tech stack and architecture look like, and why?
You want clear reasoning about models, vendors, retrieval, and where logic lives, including the tradeoffs they rejected. A good consultant explains choices in plain language and avoids locking you into one expensive provider without cause. The red flag is hand-waving, secrecy about the stack, or a black box you are not allowed to understand or own.
7. Who actually does the work?
Sometimes the person in the sales call is not the person writing the code. Ask who builds, who reviews, and how senior they are. A good answer is transparent about the team and your point of contact. The red flag is a polished closer who hands the real work to anonymous subcontractors.
Group 3: Pricing and Terms
Money is where misaligned incentives show up fastest. The goal is a pricing model where the consultant wins when you win, not when the project drags on.
8. How do you price: hourly, fixed, or retainer, and what drives the number?
A good consultant explains their model clearly and matches it to the work: fixed price for well-defined scope, retainer for ongoing iteration, hourly for genuine unknowns. The red flag is a number with no breakdown, or an incentive to maximize hours on work that should be scoped tightly.
9. What ongoing costs will I carry after we launch?
AI has a running bill: model and API usage, infrastructure, monitoring, and re-tuning as your data shifts. An honest consultant estimates these up front so you are not shocked by the monthly invoice. The red flag is silence about operating costs, which makes a project look cheaper than it truly is.
10. What happens if the project runs over or the results miss the target?
You want to hear how they handle slippage: how they communicate, how change requests work, and whether there is shared accountability for missed targets. The red flag is someone who promises everything will go perfectly. AI projects involve uncertainty, and pretending otherwise is itself a warning sign.
Group 4: Risk, Data, and Handover
This is the group most buyers forget, and it is where the real long-term risk lives. You need to know your data is safe and that you are not trapped after the engagement ends.
11. How will you handle my data, and will it be used to train third-party models?
A strong answer covers where data lives, who can access it, how it is secured, and explicit terms on whether your data ever leaves your control or feeds a vendor's training. They should know the difference between API tiers that retain data and ones that do not. The red flag is vagueness about data, or treating security as a detail for later.
12. When you leave, what do I own, and can my team run it without you?
The best engagements end with you holding the code, documentation, and the knowledge to operate the system. A good consultant plans the handover from day one and is happy to make themselves replaceable. The red flag is a setup where only they can maintain it, which quietly converts a project into a permanent dependency on one person.
Green-Flag vs Red-Flag Answers at a Glance
Use this table as a quick reference while you talk to candidates. Patterns matter more than any single answer, but several red flags together should stop you from signing.
| Topic | Green flag | Red flag |
|---|---|---|
| Track record | Names a live production system and its real users | Only demos, prototypes, and screenshots |
| Honesty | Will tell you when AI is the wrong tool | Says AI solves everything |
| Scope | Small first milestone with a checkpoint | One huge scope, one payment, no proof point |
| Metrics | Defines accuracy, latency, and cost targets | "It will feel smart" |
| Pricing | Model matched to the work, with a breakdown | A single number with no reasoning |
| Running cost | Estimates API, infra, and monitoring spend | Silent about ongoing costs |
| Data | Clear on storage, access, and training terms | Vague about where data goes |
| Handover | You own the code and can run it | Only they can maintain it |
If you want to see how I answer each of these, that is exactly the conversation I have on a first call through my AI consulting service.
How to Run These Questions in a Real Call
You do not need to fire all twelve like an interrogation. Pick the four or five that map to your biggest risk and let the answers open up a real conversation. How a consultant responds to a hard question tells you as much as the answer itself.
Listen for specificity. Strong consultants get more concrete under pressure: real numbers, real failures, real tradeoffs. Weak ones retreat to buzzwords. And reward honesty about limits: the consultant who says "I have not done exactly this, here is how I would de-risk it" is usually safer than the one who claims to have done everything.
You can read more about how I think about engineering and advising on my about page.
Frequently Asked Questions
How much does an AI consultant cost?
It varies widely by scope and seniority, from a single paid consultation to fixed-price builds and monthly retainers. Focus less on the headline rate and more on the pricing model and what you own at the end. A clear, well-scoped engagement at a higher rate often costs less than an open-ended hourly one.
What is the difference between an AI consultant and an AI agency?
A consultant is usually a single senior expert who advises and often builds directly, giving you continuity and a clear point of accountability. An agency provides a larger team but can add layers between you and the people doing the work. Ask who actually builds either way.
How do I know if an AI consultant is actually qualified?
Ask for production systems they have shipped, talk to a past client, and check whether their public work holds up. Real experience leaves a trail: live products, open-source contributions, and references who will speak candidly.
Should I hire an AI consultant or train my own team?
Often both. A good consultant accelerates your first project and leaves your team able to maintain and extend it. If a consultant resists transferring knowledge, that is a sign they are optimizing for dependency rather than your success.
Hire on Evidence, Not Vibes
The difference between an AI project that ships and one that drains your budget rarely comes down to model choice. It comes down to whether you hired someone with real production experience, honest incentives, sound data practices, and a plan to hand the work back to you.
These twelve questions are designed to surface all of that in a single conversation. Ask them of every candidate, including me. The right consultant will not be defensive. They will be glad you care enough to vet properly, because it usually means you are serious about getting it right.
If you want to talk through your specific problem and put me through this exact checklist, reach out via my AI consulting page or get in touch through contact.
