How to Sanity-Check an AI Vendor or Proposal
To evaluate an AI vendor, ignore the demo and pressure-test the proposal against three things: your data, your metric, and your exit. Ask them to run their solution on your real, messy data, not their curated sample, because a demo on clean inputs proves nothing about your reality. Make them commit to how success will be measured in a number you both agree on before any contract is signed. And confirm your exit: what you own, how you get your data out, and how hard it is to leave if they raise prices or the product declines. Then probe the fundamentals: do they recommend the simplest thing that works or the most billable, do they disclose which models and vendors they depend on, and can they explain what happens when their system is wrong. A vendor who welcomes these questions is a partner. One who deflects them is a risk.
I'm Mahmoud Zalt, an AI systems architect with 16 years building production software. Through Sista AI I sit on the buyer's side of the table and help teams read AI proposals for what they actually commit to.
Start From the Vendor's Incentive
Every AI vendor has one structural bias: they get paid to sell their thing. That is not dishonesty, it is the business model, and understanding it is the foundation of a good sanity check. A vendor assessing whether you need their product will almost always conclude that you do. A build shop scoping your project will rarely say the project should be smaller or should not happen.
This means the most valuable answer in AI, 'you do not need this, a simpler option will do,' is the one answer a vendor is least able to give you. So your job is not to trust their conclusion. It is to test their reasoning against your own interests. Read every proposal with one question running underneath: what would this look like if it were written to serve me instead of to close the deal? The gaps between those two versions are where the risk lives.
The Questions That Separate Signal From Hype
Here are the questions that expose a proposal's weak points, and what a strong answer looks like versus a weak one.
| Ask | Strong answer | Weak answer |
|---|---|---|
| Can you run this on our real data? | Yes, here is a scoped pilot on your inputs | Our demo already shows what it can do |
| How will we measure success? | A specific metric and target agreed up front | You will see the value once it is live |
| What do we own, and how do we leave? | Clear data export and no lock-in | Vague terms, proprietary formats, friction to exit |
| Which models and vendors do you depend on? | Named, with a plan if one changes | Proprietary black box, no detail |
| What happens when it is wrong? | Guardrails, fallback, and a human path | It is highly accurate, so rarely an issue |
| Is there a simpler option than this? | Honest tradeoffs, sometimes 'yes, but' | This is the enterprise-grade approach |
The pattern is not about catching vendors in a lie. It is about whether they reason in your interest when the honest answer costs them money. A vendor who says 'you could probably do this cheaper with X, but here is where our approach earns its keep' has just told you they can be trusted with the questions that matter.
The Two Costs Proposals Hide: Scale and Exit
Two numbers rarely appear honestly in AI proposals, and both can dominate the real cost of ownership.
- Cost at scale. A price that looks fine in a pilot can balloon in production, because AI cost often scales with usage in ways a flat proposal hides. Ask what one successful result costs, and what the bill looks like at ten times today's volume. If they cannot answer, they have not thought about your economics.
- Cost of leaving. Lock-in is the quiet tax. If your data lives in a proprietary format, if the integration is deep and custom, if there is no clean export, then you are not buying a tool, you are renting a dependency whose price they control. Ask what leaving looks like before you sign, because the answer is very different after.
Frequently Asked Questions
How do I evaluate an AI vendor without technical expertise?
You do not need to read code. Focus on business questions: can they prove it on your data, will they commit to a measurable outcome, what do you own, and how do you leave. A vendor who answers these plainly is credible regardless of the technical details. A vendor who retreats into jargon to avoid them is the concern.
What is the biggest red flag in an AI proposal?
A demo used as proof, with no offer to test on your real data. Demos are built to succeed on chosen inputs. If a vendor will not run a scoped pilot on your actual messy data, they are asking you to buy the demo and inherit the risk.
Should I get an independent opinion before signing an AI contract?
For any significant commitment, yes. An independent reviewer with no stake in the sale can read the proposal for lock-in, hidden scaling costs, and missing guardrails in an hour. That hour is cheap insurance against a contract that is expensive to unwind.
How do I compare two AI vendors fairly?
Give them the same real problem, the same success metric, and the same data sample, then compare their answers to the questions about ownership, exit, and failure handling. Comparing polished decks is comparing marketing. Comparing how each performs on your reality is comparing value.
Get an Independent Read Before You Sign
The best time to sanity-check an AI vendor is before the contract, not after the invoice. An experienced, independent read of a proposal surfaces the lock-in, the hidden scaling cost, and the missing guardrails that a polished pitch is designed to keep out of view, and it does so in far less time than the commitment is worth.
My Q&A Session is built for this moment: bring the proposal, and get direct answers, risk flags, and a clear read on what you are actually signing up for. It is $90 for a one-hour open-format session, $170 for a two-hour working session, or $240 for a three-hour team session if you want your buying group aligned before you decide. I have no stake in which vendor you choose. You can read more about me through Sista AI.
Book a focused Q&A session and get an independent read on your AI vendor.
