How to assess a vendor's AI claims in an afternoon

You can assess most of a vendor's AI claims in an afternoon: pick the five claims that carry risk, find the public evidence each one needs, check the certifications against the body that issued them, and treat any contradiction between the vendor's own pages as a finding.

Responsible AI governance is no longer a niche topic. In Australia, the ASD's guidance on AI within the Essentials series puts accountability for AI being secure, controllable, human-supervised and used ethically with the board or executive committee. Assurance schemes are moving at the same pace: the AIUC-1 standard's third-quarter 2026 update added requirements for coding agents and guidance for auditors. The practical consequence for anyone who buys software is simple. Your vendors are making more AI claims, and someone on your side is now accountable for having checked them.

Here is a way to do that without a six-week project.

The five claims that matter

Vendors say many things about their AI. Five of them carry most of the risk, and each has a specific kind of evidence that proves it.

1. "We do not train on your data." The proof is a data processing agreement or AI terms page that says so in writing, with the opt-out or default stated. A marketing page that says "your data stays yours" is a claim, not evidence.
2. "A human is in the loop." The proof is documentation that says which decisions a person reviews, and how a customer can contest an output. If the only mention is in a blog post, treat it as unverified.
3. "Our model is our own" or "we use provider X." The proof is a model card, a subprocessor list, or an architecture page that names the model provider. Many products are a thin layer over someone else's foundation model, and the data handling you care about may sit one supplier further back.
4. "We are certified." The proof is a certificate number and the certification body, which you can check with that body. ISO/IEC 42001 is certifiable; the NIST AI Risk Management Framework is voluntary guidance and cannot be "certified against". A vendor that claims the second is telling you something about its precision.
5. "We comply with the relevant regulation." The proof is a published assessment, a named regulation, and the scope it applies to. "Compliant with all applicable AI laws" without a name is a claim about nothing in particular.

Work through the public record in order

Start with what anyone can see, before you send a questionnaire. A questionnaire records what a vendor is willing to say about itself; the public record shows what it has already committed to.

1. Read the trust centre, the AI policy and the data processing terms. Note the date on each. Anything older than the AI feature it describes is suspect.
2. Find the subprocessor list and the model documentation. Write down every provider the vendor names.
3. Check each certification with the body that issued it. A certificate that cannot be found on the certification body's register is not a certification for your purposes.
4. Look for conflicts. The privacy policy says one thing about training; the AI terms say another; a sales page says a third. Conflicting claims are a finding in their own right, and often the most useful one, because they tell you which page the vendor has not kept up to date.
5. Grade it the same way every time. Fixed dimensions and a fixed evidence ladder let you compare one vendor with another, and a vendor with itself six months later.

Where to stop and where to ask

The public record will not answer everything. It will tell you, quickly, which vendors clear the bar on what they have published and which ones owe you a conversation before their AI touches anything that matters. Save the questionnaire and the engagement for the second group.

The Trust Directory shows what this looks like when the method is applied consistently: each vendor graded on the public record, each finding linked to its source, and the gaps marked as gaps rather than filled in with assumptions. If a vendor you rely on is not there, a free AI Trust Scan runs the same method on their public footprint.

None of this replaces a proper assessment where the stakes justify one, or a qualified reading of how a regulation applies to your obligations. It does mean the first pass takes an afternoon rather than a quarter, and that the conversation you eventually have with the vendor starts from evidence.

Related reading

• What outside-in AI assurance actually means
• Your vendors' AI is now part of your risk surface
• How to read an AI Trust Scan report

---

Know what you're relying on. Run a free AI Trust Scan on a vendor in your supply chain, or commission independent assurance when the decision warrants it.