AI Vendor Due Diligence Checklist

Why AI Vendor Due Diligence Matters

#

Selecting an AI vendor isn’t like choosing traditional software. When you deploy a third-party AI system, you’re not just buying a product, you’re inheriting its biases, its security vulnerabilities, its training data decisions, and potentially its legal liabilities.

The Mobley v. Workday ruling demonstrated that companies can face discrimination claims based on AI decisions made by their vendors’ systems. The AI standard of care is evolving to hold organizations accountable for the AI systems they deploy, regardless of who built them.

The stakes are high:

• Regulatory penalties under emerging AI laws
• Discrimination and civil rights liability
• Data breach and privacy violations
• Reputational damage from AI failures
• Contractual disputes when systems underperform

This guide provides a comprehensive framework for evaluating AI vendors before you sign, and monitoring them after deployment.

Technical Evaluation Criteria

#

Model Architecture and Performance

#

Before signing any contract, you need to understand what you’re actually getting. Many vendors are vague about their technical capabilities, and vagueness is a red flag.

Core Technical Questions

#

Red Flags in Technical Claims

#

“Our AI is 99% accurate”, Accuracy without context is meaningless. Accurate at what task? Measured on what data? What’s the false positive vs. false negative rate? Demand specificity.

“Proprietary technology”, Some legitimate AI involves trade secrets, but complete opacity should concern you. At minimum, vendors should explain the general approach and validation methodology.

“We use cutting-edge AI”, Buzzwords without substance suggest marketing over engineering. Ask for technical documentation, not brochures.

No documentation of failure modes, Every AI system fails. Vendors who can’t articulate how their system fails either don’t understand it or are hiding problems.

Integration and Infrastructure

#

Technical Integration Checklist

#

Explainability and Transparency

#

For high-stakes decisions, hiring, lending, healthcare, criminal justice, you may be legally required to explain AI decisions to affected individuals. Even where not legally mandated, explainability is increasingly expected.

Explainability Questions

#

• Does the system provide decision explanations?
• Are explanations human-readable or technical-only?
• Can explanations be provided at both individual and aggregate levels?
• Has the explanation methodology been validated?
• Can the system identify which features most influenced specific decisions?

Legal and Contractual Considerations

#

Liability Allocation

#

The most important contractual question: When this AI system causes harm, who pays?

Intellectual Property Rights

#

Training Data Provenance

#

Post-NYT v. OpenAI, training data liability is a legitimate concern. If your vendor trained on unlicensed copyrighted material, you could face secondary liability.

Key Questions:

• Can the vendor document the provenance of training data?
• Does the vendor have licenses for copyrighted training materials?
• What indemnification does the vendor provide for IP claims?
• Does the vendor have a process for responding to takedown requests?

Output Ownership

#

• Who owns outputs generated by the AI system?
• Can outputs be used for any purpose, or are there restrictions?
• Does the vendor claim any rights to outputs?
• Can outputs be used to train competing systems?

Regulatory Compliance

#

Different industries face different AI regulations. Ensure your vendor understands your compliance obligations.

Contract Terms to Negotiate

#

Audit Rights: You need the ability to audit vendor systems, either directly or through third parties. This should include:

• Access to model performance metrics
• Bias audit data
• Security assessment results
• Training data documentation

Change Notification: The vendor should notify you before:

• Material changes to the model
• Changes to training data
• Updates that could affect accuracy or bias
• Changes to data handling practices

Termination Rights: You should be able to exit if:

• The vendor fails audits
• Regulatory requirements change
• Performance degrades below agreed thresholds
• The vendor is acquired by a competitor

Data Rights on Termination: What happens to your data when the contract ends?

• Return of all data in usable format
• Certification of data deletion
• Transition assistance period

Bias and Fairness Assessment

#

Pre-Deployment Bias Evaluation

#

AI bias isn’t just an ethical concern, it’s a legal liability. Discriminatory AI systems have led to class actions, regulatory enforcement, and reputational crises.

Fairness Metrics to Request

#

Different fairness metrics capture different types of disparities. No single metric is sufficient.

Demographic Parity: Equal selection rates across groups Equalized Odds: Equal true positive and false positive rates across groups Predictive Parity: Equal precision across groups Individual Fairness: Similar individuals receive similar predictions Calibration: Predicted probabilities are accurate across groups

Ask vendors which metrics they use, why they chose those metrics, and what thresholds they consider acceptable.

Ongoing Bias Monitoring

#

Bias can emerge after deployment as:

• User populations change
• Model drift occurs
• Feedback loops amplify small initial biases
• New edge cases appear

Require vendors to provide:

• Ongoing demographic performance reports
• Alerting when metrics exceed thresholds
• Quarterly or annual bias re-audits
• Clear escalation procedures when bias is detected

Security and Privacy Review

#

Data Security Assessment

#

AI systems often process sensitive data at scale. A security breach could expose millions of records.

Privacy Compliance

#

Data Processing Questions

#

• What data does the system collect beyond what’s submitted?
• Is data used to train or improve the vendor’s models?
• Can you opt out of data use for model training?
• Does the system make automated decisions about individuals?
• What data subject rights does the vendor support (access, deletion, correction)?
• How does the vendor handle cross-border data transfers?

Privacy Impact Assessment

#

Before deploying any AI system that processes personal data:

1. Document what personal data is processed
2. Identify the legal basis for processing
3. Assess privacy risks to data subjects
4. Implement appropriate safeguards
5. Document the assessment and decisions

AI-Specific Security Concerns

#

Traditional security assessments may miss AI-specific risks:

Prompt Injection: Can malicious inputs manipulate the AI to bypass controls? Model Extraction: Can competitors reverse-engineer the model through queries? Data Poisoning: Could training data be manipulated to compromise the model? Adversarial Inputs: Is the model robust to inputs designed to cause misclassification?

Ask vendors what testing they’ve done for these AI-specific attack vectors.

Ongoing Monitoring Requirements

#

Performance Monitoring Framework

#

Due diligence doesn’t end at contract signing. You need ongoing monitoring to ensure the system continues to meet standards.

Model Drift Detection

#

AI models degrade over time as the world changes. Monitor for:

Data Drift: Input data distributions changing from training data Concept Drift: The relationship between inputs and outcomes changing Performance Decay: Gradual accuracy decline Bias Emergence: New disparities appearing in model outputs

Incident Tracking

#

Maintain a log of all AI-related incidents:

• What happened
• When it was detected
• How it was resolved
• Root cause analysis
• Preventive measures implemented

This documentation is essential for demonstrating due diligence if the system is later challenged.

Sample RFP Questions

#

Use these questions when evaluating AI vendors. Adapt based on your specific use case and risk level.

Technical Capabilities

#

1. Describe the model architecture and explain why it’s appropriate for this use case.
2. What training data was used? Provide documentation of data sources and licensing.
3. What validation testing was performed? Provide methodology and results.
4. What are the known limitations and failure modes of this system?
5. How does the system handle inputs it wasn’t trained on?
6. What explainability features does the system provide?
7. Describe your model versioning and update process.

Bias and Fairness

#

8. Has this system been tested for bias? Provide audit documentation.
9. What demographic groups were included in bias testing?
10. What fairness metrics do you report, and what are current values?
11. What bias mitigation measures have been implemented?
12. How do you monitor for bias emergence post-deployment?
13. Has the system been audited by an independent third party?

Security and Privacy

#

14. Provide your most recent SOC 2 Type II report.
15. Describe your data encryption approach (at rest and in transit).
16. What is your incident response procedure and notification timeframe?
17. How is customer data used? Is it used for model training?
18. What data subject rights do you support?
19. Describe your approach to AI-specific security threats (prompt injection, adversarial inputs).

Legal and Compliance

#

20. What indemnification do you provide for IP infringement claims?
21. What indemnification do you provide for discrimination claims?
22. What insurance coverage do you carry?
23. What regulatory compliance certifications do you hold?
24. Describe your audit rights and what access you provide to customers.
25. What is your liability cap, and how was it determined?

Support and Operations

#

26. What is your guaranteed uptime SLA?
27. How will you notify us of model updates?
28. What support tiers do you offer?
29. What training and documentation do you provide?
30. What is your process for handling customer-reported issues?

Vendor Evaluation Scorecard

#

Use this scorecard to compare vendors systematically.

Scoring Guide:

• 5: Exceeds requirements, best-in-class
• 4: Meets all requirements, strong offering
• 3: Meets minimum requirements
• 2: Some gaps, requires negotiation
• 1: Significant gaps, high risk

Minimum threshold: Don’t proceed with vendors scoring below 3.0 overall or below 2.0 in any single category.

Frequently Asked Questions

#

What if vendors won’t provide bias audit documentation?

#

This is a significant red flag. For high-risk AI systems (employment, lending, healthcare), lack of bias documentation is increasingly unacceptable, and may violate emerging regulations like NYC Local Law 144 and the Colorado AI Act.

Options:

1. Require independent third-party audit as condition of contract
2. Conduct your own bias testing before deployment
3. Choose a different vendor with better transparency
4. If proceeding, document your decision and rationale carefully

How much liability cap is enough?

#

It depends on your exposure. Consider:

• Maximum potential class size affected by AI decisions
• Per-person damages in discrimination/privacy claims
• Regulatory penalty maximums
• Your organization’s risk tolerance

Rule of thumb: The liability cap should be at least 2x your estimated worst-case exposure. For enterprise deployments affecting thousands of individuals, $10M+ caps are increasingly common.

Can we require vendors to carry AI-specific insurance?

#

Yes, and you should. AI-specific policies now exist that cover:

• Algorithmic discrimination claims
• AI errors and omissions
• Training data IP infringement
• Regulatory defense costs

Require proof of coverage and name your organization as additional insured.

What if the vendor uses subprocessors for AI services?

#

Many vendors wrap third-party AI APIs (OpenAI, Anthropic, etc.) rather than running their own models. This isn’t necessarily bad, but you need to know:

• Who are the subprocessors?
• What contractual protections exist with them?
• How is your data handled by subprocessors?
• Do subprocessors meet your security requirements?

Flow-down your requirements to subprocessors through contract terms.

How do we handle vendors headquartered outside our jurisdiction?

#

Key considerations:

• Choice of law and venue for disputes
• Cross-border data transfer mechanisms
• Local regulatory compliance
• Practical enforceability of contract terms
• Insurance coverage across jurisdictions

For high-risk systems, prefer vendors with legal presence in your jurisdiction.

Conclusion

#

AI vendor due diligence is not a checkbox exercise, it’s an ongoing risk management discipline. The AI systems you deploy become extensions of your organization, and their failures become your failures.

This checklist provides a starting framework, but adapt it to your specific industry, use case, and risk tolerance. Document your evaluation process thoroughly. When AI liability questions arise, and they will, your due diligence documentation is your first line of defense.

Key Takeaways:

1. Technical evaluation isn’t optional. Understand what you’re buying.
2. Liability allocation matters. Negotiate indemnification before you need it.
3. Bias assessment is a legal requirement for many high-stakes applications.
4. Security due diligence must include AI-specific risks.
5. Monitoring continues after deployment. Due diligence is ongoing.

The AI standard of care demands that organizations evaluate AI systems before deployment and monitor them continuously. This checklist helps you meet that standard.