The Practitioner's Playbook
Procurement & Third-Party AI Risk
10 min+90 XPStep 1/10
Diligence that goes past the demo
Vendor AI diligence is a distinct checklist from generic SaaS review:
- Provenance posture — how was the model trained, on what, under what legal theory; does the vendor indemnify IP claims, and with what carve-outs
- Your task, their evals — vendor benchmarks are marketing; the eval that matters is the one run on your data and your failure cases
- Data handling — is input data retained, used for training, shared with subprocessors; can you get a no-train commitment and deletion
- Model lifecycle — what happens when they deprecate the model your product depends on; versioning commitments and notice periods
- Certifications — SOC 2 as table stakes; ISO/IEC 42001 emerging as the AI-specific signal
The diligence artifact is a vendor file that survives their marketing page.