Evaluating AI vendors: a due diligence framework procurement teams can run.

Why vendor claims need a specific question behind them

“Enterprise-grade”, “secure by design” and “responsible AI” are not falsifiable claims. They cannot be checked, so they cannot be relied on in a procurement decision. The more useful approach is to replace each claim with a specific question tied to a recognised framework, so the vendor’s answer can be verified rather than taken on faith.

Two frameworks do most of the work here: ISO/IEC 42001:2023, the first international standard for AI management systems, and the NIST AI Risk Management Framework, in particular its GOVERN function covering accountability and human oversight. Neither requires the buyer to have deep AI expertise; both give the buyer a named control to ask about.

◇

“Enterprise-grade” is not a claim you can audit. A named control is.

Governance and accountability questions

The NIST AI RMF’s GOVERN function requires that “roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are documented and are clear to individuals and teams throughout the organization” (GOVERN 2), that personnel receive training on “applicable laws and regulations for AI systems” and organisational AI policies (GOVERN 2.1), and that organisations set “policies and procedures that define and differentiate the various human roles and responsibilities when using, interacting with, or monitoring AI systems” (GOVERN 3.2).

• Ask who inside the vendor owns AI risk, and whether that person or team is separate from the engineering team that builds the system.
• Ask whether development and testing are functionally separated, so a system can be independently course-corrected rather than graded by the team that built it.
• Ask what proficiency standards or training the vendor requires of staff who operate or oversee the system.
• Ask whether the vendor holds, or is pursuing, ISO/IEC 42001 certification, and if not, what internal equivalent exists.

Data, model and retrieval questions

These questions determine whether the vendor’s answers to a data protection review will hold up, and whether the system can actually be governed once deployed rather than only described as governed in a sales deck.

• Where is data stored and processed for training, fine-tuning, retrieval and logging, and does that location match what the contract says?
• Can the vendor identify which specific model and version answered a given query, months after the fact?
• What happens to your data, embeddings and configuration if you switch providers: is export a real, tested path or a promise?
• Is retrieval scoped by the requesting user’s permissions, or does every user effectively query the same index regardless of role?

Evidence and audit questions

A vendor that can only show you a usage dashboard has told you how much the system was used, not whether it was used correctly. The distinction matters most during an incident, when aggregate metrics answer none of the questions an auditor or regulator will actually ask.

• Can the vendor export a reconstructable decision trail: identity, active scope, retrieved sources, model, and outcome for a single request?
• How long are logs retained, and does that retention period cover your regulatory obligations, not just the vendor’s default?
• Can a denied or blocked action be reconstructed as evidence that a control worked, not only successful actions?

Contract terms that carry the answers

Verbal or written sales answers are only useful if they end up in the contract. Translate each satisfactory answer into contract language: a data processing addendum reflecting the Security Principle obligations that now sit directly with processors under Malaysian law, audit rights, advance notice of model or version changes that could affect outputs, a tested data export and exit clause, and a clear allocation of liability for errors the AI system produces.