The Problem Advisors Are Facing
Across the wealth management industry, AI has moved from experiment to everyday tool. Advisors use it to summarize meetings, draft client emails, research markets, and run portfolio numbers.
But adoption has outpaced evaluation. And regulators have noticed.
Consider this scenario. An advisor uses a general-purpose AI tool to help prepare the analysis behind client proposals. It saves hours every week. Then an SEC exam begins, and one of the examiner’s questions is simple:
How did you test this tool before relying on it?”
The firm has no test results. No record of which model version produced which numbers. The tool runs on a personal account whose data terms nobody at the firm has read.
The tool may have worked fine. The firm just can’t show it. In a regulated business, those are not the same thing.
Our first two white papers, When AI Sounds Smart But Gets the Numbers Wrong and Same Question, Different Answer, measured how often general-purpose AI gets investment numbers wrong, and whether it gives the same answer twice. Our third paper asks the question that follows: what does a firm need to show an examiner about the AI it uses?
The Standards Are Already Taking Shape
The SEC’s FY2027 Examination Priorities, released October 23, 2026, direct examiners to check whether firms’ claims about their AI tools are accurate, and whether firms are monitoring how they use them. Treasury’s Financial Services AI Risk Management Framework, released in February, lays out roughly 230 control objectives for managing AI risk, including risk from outside vendors.
Together they’re the clearest signal yet of what examiners will expect. But neither was built for an advisor deciding whether to adopt a tool. One is a staff statement. The other is a 230-point control matrix.
Three Questions Instead of 230
The white paper reduces both sources to three questions an advisor can ask of any AI tool, each tied to a specific SEC or Treasury provision:
- Explain It. Can the tool show why it produced an answer, and does it separate what it knows from what it’s guessing?
- Verify It. Does it give the same answer twice, does it hold up beyond easy cases, and is its error rate known?
- Supervise It. Does the vendor commit in writing to client data, security, and audit trails?
Each question breaks into three sub-questions. The paper then reads all nine from the examiner’s side: the request an examiner is likely to make, and the evidence a firm should have ready.
What an Examiner May Ask: Three Examples
Verify It: Does the tool give the same answer twice?
What an examiner may ask: “How did you test this tool before relying on it?”
What to have ready: Pre-adoption test results: the questions asked, the answers received across repeat runs, and the reference values used.
This is the easiest evidence to create, and most firms don’t have it. In our second paper, we asked six AI systems the same ten calculations, five times each. Perplexity’s answers to a single Sharpe ratio question varied with a coefficient of variation of 93.7%. ChatGPT’s correlation answers varied by 43.7%.
Why This Matters: A repeat-and-compare test takes a few hours. Kept in the compliance file, it answers the examiner’s question directly.
Supervise It: Is client data used to train the model?
What an examiner may ask: “How is client information protected when it’s entered into this tool?”
Why This Matters: For many general-purpose tools, the answer depends on the plan, not the tool. A setting on a personal account and a contract term on a business plan are very different things to hand an examiner.
Explain It: Is there a real answer to “why did it recommend this?”
What an examiner may ask: “Walk us through how this recommendation was produced.”
Why This Matters: The fiduciary duty doesn’t shift when a tool helps produce the advice. “The AI said so” isn’t a defense.
Not Every Use Case Needs the Same Depth
AI used for meeting notes doesn’t carry the same risk as AI used to build a client proposal. The paper includes a heatmap showing how much evidence seven common AI uses call for, because the closer a tool’s output gets to a client’s portfolio, the more of the benchmark applies.
This Isn’t About Slowing Down
AI tools are improving quickly, and that pace is exactly why a repeatable benchmark matters. A firm that tests regularly can adopt new capabilities as soon as they prove themselves. We apply the same benchmark to DeepVest in the paper, and we encourage firms to hold us to it.
The question for any AI tool comes down to this: could you explain, verify, and supervise the tool you’re using if an examiner asked tomorrow?
Want the Complete Benchmark?
This article covers 3 of the 9 benchmark questions. The full white paper includes all nine, the evidence to have ready for each, the use-case heatmap, and a four-step plan to start building your firm’s AI compliance file.
Exclusive Research Report
Download the Full White Paper
The complete white paper includes all nine benchmark questions, the evidence to have ready for each, and a use-case heatmap showing how much evidence each type of AI use calls for.
- ✓The complete benchmark: All nine Explain It, Verify It, and Supervise It questions, each traced to a specific SEC or Treasury provision
- ✓The examiner’s view: The likely exam request and the evidence to have ready for every question
- ✓The regulatory breakdown: Where the AI provisions sit in the SEC’s FY2027 priorities, and how Treasury’s 230-control framework is structured
- ✓Scaling by use case: A heatmap showing how much evidence seven common AI uses call for, from meeting notes to client proposals
- ✓Pre-adoption testing: How to run a repeat-and-compare test and keep the results in your compliance file
- ✓DeepVest in practice: How we answer all three questions, and the evidence firms can request from us
- ✓A four-step starting plan: What a firm can do now to build its AI compliance file
Instant Access
Complete the form below to receive your copy
Secure & Private Research
About This Paper
This paper was prepared by DeepVest in October 2026. It synthesizes the SEC’s FY2027 Examination Priorities and the U.S. Treasury’s Financial Services AI Risk Management Framework into a practical benchmark for evaluating AI tools. Testing results cited here come from DeepVest’s April–May 2026 study, “Same Question, Different Answer.” The paper is for informational purposes only and is not legal or compliance advice.
DeepVest is the agentic front office for wealth management teams. It unifies research, planning, market data, proposals, and monitoring in a single workspace. AI runs the workflow while deterministic calculations produce the numbers, so every output is transparent, auditable, and traceable to verified institutional data.
For questions about this paper: contact [email protected].
