AI vendor risk assessment that checks what the vendor said against what its own documents say, before you sign
Undisclosed models and training on your data caught before you sign, every claim cited.
See one case, screen by screen ↓AI Vendor Risk Assessment is AI vendor risk assessment software that checks how your vendors use AI from their questionnaire answers, contracts and model documents. Eight AI agents send and read the AI vendor questionnaire, read the contract, online terms, DPA, model cards, trust centre, SOC 2 report and bias audits, build each vendor’s AI stack with every value cited, check each other’s findings, propose the risk tier under the EU AI Act, US law and your policy, and draft the contract changes. The AI risk lead rules on the findings and signs the tier.
How to choose AI solutions for back-office and enterprise work →
Updated
Every large company now buys AI it did not build: recruiting software that ranks applicants, a payables tool that switched on a copilot, a translation service that changed model provider. The vendor risk team learns about it from a questionnaire the vendor answered in good faith, or not at all. The answers are taken at face value, while the contract, the online terms, the DPA and its subprocessor annex, the model card and the SOC 2 report — the documents that bind the vendor or show what it actually does — are read by another team, months apart, if at all.
That is where the risk sits. A vendor that says it never trains on customer data may have widened its “service improvement” clause by posting new terms. One that says it builds all its AI in-house may send CVs to a third-party model listed only in a DPA annex or carved out of its SOC 2. Under the EU AI Act a buyer using AI to filter job applications is the deployer of a high-risk system, Colorado’s AI Act already applies to hiring, and New York City wants a bias audit no more than a year old. A finding that never reaches the contract stays an exception forever.
Estimated split for one AI vendor with a contract, online terms, a DPA, a model card, a SOC 2 report and a bias audit, by hand and with the solution.
Eight agents question, read, build the AI stack, check each other, tier and draft; the AI risk lead decides and signs.
Kestrel & Vane Group buys its recruiting platform from TalentLoop: four AI features, 62,400 applications a year in 14 countries, a renewal on 15 January 2027 worth €1.84M over three years. TalentLoop changed its terms in July; with the renewal inside its 120-day window, the change monitor reopened the assessment on 22 September. Here is what happens next, screen by screen, in the working solution.
Nadia owns the AI third-party policy and approves every High tier. Her home is a short feed, not a dashboard: 318 vendors use AI, 46 assessments are open, 9 decisions need her. The tiering agent says TalentLoop is ready; the change monitor reports that Ledgerwick switched on a new AI feature; the questionnaire agent asks whether to escalate a vendor 11 days late. Each message carries its sources and a button to act.
Tiering agent: “Two of its answers disagree with its own documents: its terms allow training on our data, and a third-party model reads applicants’ CVs. I propose a High tier.”
The stack builder lays out each AI feature in six layers: what it does, the models, the data it sees, where the data goes, training on your data, and oversight and fairness. Each layer shows what TalentLoop said and what its documents say, side by side. Nadia asks in plain words which model reads applicants’ CVs. The answer: Corvane C-4, a third-party model from a US provider, extracts skills before TalentLoop’s own Rank v5 scores them.
“4 AI features · 4 models, 1 third-party · 24 layers checked”
Each claim on the stack carries a numbered citation. Nadia opens the one on training: TalentLoop Terms of Service v7, effective 1 July 2026, §9.3 on page 14, with the sentence highlighted. The document reader read 7 documents, 274 pages, and the verifier checked them. From the passage she can open the whole document, ask about it or copy the citation.
§9.3: “TalentLoop may use Customer Data, in de-identified or aggregated form, to maintain, improve and develop the Services, including the training and evaluation of machine learning models.”
Tomás approved AI vendor questionnaire v3, 46 questions in 7 sections, on 23 September. Owen Hartley at TalentLoop answered 44 and skipped 2. Every answer carries the agents’ read: accepted, contradicted, out of date, skipped. The first follow-up quoted TalentLoop’s own model card, §9.3 and its audit date; Owen confirmed Corvane C-4 on 8 October. A second follow-up on the two skipped questions is drafted and waits for Send.
The questionnaire agent never sends anything without a person pressing Send.
The contradiction checker brought five findings to Nadia. The first: Q14 says customer data is never used to train models, but the terms allow it — 97% sure. The verifier re-read the sources without seeing that verdict, found that §9.3 changed on 1 July from “aggregated usage statistics” to training, and that the opt-out exists only on request. It agrees. Nadia confirms the mismatch, and the second, a third-party model behind three features.
Verifier: “I agree: mismatch.”
On where applicant data goes, the agents split. The checker calls Q18 a mismatch: TalentLoop says data is hosted in Ireland, but CV text reaches Corvane Labs in San Francisco and is kept 30 days. The verifier says the answer points to DPA Annex 3, which discloses the transfer under standard contractual clauses: incomplete, not false. Nadia rules “not a mismatch” with reason DR-01, disclosed by reference, and signs. Both agents’ readings stay on the record next to hers.
Verifier: “I disagree: disclosed by reference. A person should decide.”
The tiering agent proposes High, confidence 0.94, and shows its rule trace. Shortlist Assist ranks applicants: recruitment is high-risk under EU AI Act Annex III point 4(a). TalentLoop is the provider; Kestrel & Vane is the deployer, with Article 26 duties from 2 December 2027. Hiring in Denver brings the Colorado AI Act’s yearly impact assessment; the New York office needs a bias audit within a year of use, and TalentLoop’s is 19 months old. Policy rule R1 sets High.
“The highest of the rules that apply. Every step below is cited.”
Nadia approves High with reason code HR-01, AI used in employment decisions, on condition that contract changes C1 to C4 are agreed before the renewal, and that C1 and C2 block it. She signs with her password; the signature records her name, the time and its meaning. The tiering agent sets the renewal gate in the procurement system and tells Claire Dubois, Head of Talent Acquisition, what it means for her team.
Meaning of the signature: “I approve this risk tier and its conditions.”
The drafting agent turned each finding into a redline from the approved clause library: C1 switches §9.3 off for Kestrel & Vane data and requires deletion of anything already used; C2 replaces website notice with 60 days’ written notice of any new or changed model or provider; C3 an annual independent bias audit; C4 support for deployer duties. Hannah approved three and widened C3 to every country, not only New York. Marco Bellini in procurement negotiates.
“Renewal on 15 Jan 2027 is blocked until C1 and C2 are signed.”
The change monitor reads terms, subprocessor lists, trust centres and release notes for all 318 vendors every morning. On 8 October Ledgerwick’s release notes switched on Invoice Copilot by default, sending invoice line text to “our AI partner for classification”. The monitor shows the before and after, names the layers it changes — models and where data goes — and recommends asking Ledgerwick to switch it off. A person reassesses or closes it as not material, with a reason.
“The monitor never closes a change on its own.”
The same TalentLoop assessment, seen by the five people it reaches: how their month used to go, and how it goes now.
Eight agents question, read, build the AI stack, check each other, tier and draft; the AI risk lead decides and signs.
Sends the AI vendor questionnaire, chases on the deadline, reads the answers and drafts pointed follow-ups that quote the vendor’s own documents.
Reads the contract, online terms and earlier versions, the DPA and its subprocessor annex, model cards, trust-centre pages, the SOC 2 report and bias audits, and quotes each passage with section and page.
Assembles each vendor’s AI stack — features, models (own or third-party), the data each sees, where it goes, training and oversight — with what the vendor said and what its documents say on every layer.
Compares each vendor statement with the documents and marks it as matching, a mismatch, out of date or not stated, with its reasoning and confidence.
Re-reads every source on its own, without seeing the checker’s verdict, and agrees or disagrees.
Proposes the EU AI Act role and risk class, the US state and city duties and your internal tier, with a rule trace that cites each step.
Turns each confirmed mismatch or gap into a contract change from your approved clause library, tied to the finding it fixes.
Finds which vendors use AI from contracts, invoice lines, trust centres and release notes, then watches terms, subprocessor lists and release notes every morning and reopens the stack with a before and after.
Approves tier and changes. The agents propose; a named person decides.
Ask the agents about a vendor, a clause or a regulation — or tell them what to do. Every answer cites its passage.
Which model reads our applicants’ CVs?
Two models read applicants’ CVs in Shortlist Assist: Corvane C-4, a third-party large language model from Corvane Labs Inc. in the USA, extracts skills and experience; then TalentLoop’s own Rank v5 scores the profile. The DPA lists Corvane Labs as subprocessor 7, with prompts kept 30 days. TalentLoop’s answer said every model is in-house; Owen Hartley confirmed Corvane on 8 Oct.
Does TalentLoop train on our data?
Its terms allow it. Since Terms v7 (1 Jul 2026), §9.3 lets TalentLoop use customer data “including the training and evaluation of machine learning models”; v6 only allowed usage statistics. Its answer to Q14 says never. The opt-out exists only on request, so the drafting agent wrote clause C1 to make it contractual.
Why does TalentLoop need a high tier?
I propose High (confidence 0.94). Shortlist Assist ranks applicants: recruitment under EU AI Act Annex III point 4(a). Kestrel & Vane is the deployer, with Article 26 duties from 2 Dec 2027. Hiring at the Denver distribution centre falls under the Colorado AI Act, in force since 30 Jun 2026. Policy rule R1 sets High.
What changed at our vendors this week?
Four vendors changed in the last two weeks. Ledgerwick switched on Invoice Copilot by default on 8 Oct, sending invoice lines to a third-party model. Harbor & Finch Health added a US transcription subprocessor. Quillmark Translate moved to a new model provider that keeps content 14 days. Nimbello Sign reworded a notice, closed as not material.
13 screens from the working solution, on its sample data. Pick one to see it large.
The rules every vendor is checked and tiered against are settings, versioned and on the audit trail.
| Setting | Default | Choose from |
|---|---|---|
| Tier rules | R1–R6 from policy AI-TP-POL-02 v3 | Each rule on or off · add a rule |
| Who approves a High tier | AI risk lead | AI risk lead · AI risk lead and CRO · Risk committee |
| Longest risk acceptance | 90 days | 30 to 180 days, in steps of 30 |
| A second agent must agree before a mismatch is confirmed | On | On · off |
| Evidence older than this is out of date | 12 months | 6 to 24 months, in steps of 6 |
| Below this confidence, a finding goes to a person even if both agents agree | 85% | 70% to 95%, in steps of 5 |
| Questionnaire reminders | Day 5 and day 10 | Day 5 and day 10 · Day 3, 7 and 10 · Day 7 only |
| A change at a High-tier vendor | Notify the AI risk lead within 1 hour | Within 1 hour · Same day · Weekly digest |
Every figure is labelled: a target the solution is built to, an estimate, a typical published result, or a proven one.
“demo” = seen in the working solution, on its sample vendor data · “target” = the design goal, measured in the live solution · “estimated” = our estimate · “typical” = published figures (a 2026 industry survey of third-party risk management at 173 financial-services firms: 72% only partly aware which vendors use AI, 63% with one or two dedicated people, 53% managing 300+ vendors). Regulation as context: EU AI Act (Regulation (EU) 2024/1689) Annex III point 4(a) and Article 26, with Annex III obligations from 2 December 2027 after the 2026 AI omnibus; Colorado AI Act (SB 24-205), in force 30 June 2026; NYC Local Law 144. Kestrel & Vane Group, its vendors, their models and the people named on this page are fictional: characters and sample data in the working solution.
AI Vendor Risk Assessment is AI vendor risk assessment software that checks how your vendors use AI from their questionnaire answers, contracts and model documents. Eight AI agents send and read the AI vendor questionnaire, read the contract, online terms, DPA, model cards, trust centre, SOC 2 report and bias audits, build each vendor’s AI stack with every value cited, check each other’s findings, propose the risk tier under the EU AI Act, US law and your policy, and draft the contract changes. The AI risk lead rules on the findings and signs the tier.
No. The answers are one source among several. Every layer of the AI stack shows what the vendor said next to what its own contract, terms, DPA and model documents say, and a difference is marked as a mismatch. A statement with no document behind it is marked “said, not evidenced”, and a skipped question is a finding, not a pass.
The change monitor runs a discovery over your vendor master: contracts and DPAs, invoice line text, trust centres and release notes. In the working solution it read 2,140 active vendors and found 318 whose service uses AI, each with its source. It never marks a vendor as having no AI without a source.
The contradiction checker compares each answer with the documents — for example a “never trains” answer against a service-improvement clause that allows training, or an “all in-house” answer against a model card and a DPA annex that name a third-party model. The verifier re-reads the same sources on its own. Agreed mismatches are confirmed; any disagreement goes to a person.
The tiering agent proposes the tier with a rule trace: what the AI does, its EU AI Act class (for example recruitment under Annex III point 4(a)), your role as deployer and its Article 26 duties, US state and city law such as the Colorado AI Act and NYC Local Law 144, and your own policy rules. Each step is cited. It only proposes; the AI risk lead approves, changes or sends it back, with a reason code and a signature.
The change monitor reads terms, subprocessor lists, trust centres and release notes for every AI vendor each morning, shows the before and after, and names the layers of the stack it affects. A person reassesses or closes it as not material with a reason; the monitor never closes a change on its own and never contacts a vendor.
Yes. The agents propose and draft; nothing is sent to a vendor, procurement or a business owner until a person presses Send. Findings, the tier, risk acceptances and contract changes are decided by a named person with a reason code and an electronic signature, privacy counsel reviews every clause, and every step is on an audit trail that cannot be changed.
The Agentic Solution Engine builds and deploys it from your requirements and documents — your AI third-party policy, questionnaire, clause library and a sample of vendor files — and it goes live once every quality gate has passed. Setup connects your vendor master, contract repository, procurement system and identity provider, runs the discovery and invites the team, with a person confirming each step.
Exceptions weighed and every user-entity control owned before the SOX year-end test.
1-min demoThird-Party Contract ReviewMissing audit, exit and subcontracting clauses fixed before renewal, not at inspection.
1-min demoSupply Chain Due DiligenceDetained goods released with a trace to the raw material, every link proved.
1-min demoWe’ll run AI Vendor Risk Assessment on a sample of your own vendors’ questionnaire answers, contracts and terms.