Generative AI is already doing real work at financial firms. It summarizes loan files, drafts client meeting notes, and answers customer questions. In the Bank of England and FCA's October 2026 survey, generative AI accounted for 56% of all AI systems reported by UK firms.

The risk is just as clear. In Cambridge CCAF's 2026 global survey, 70% of industry firms named model hallucinations and unreliable outputs as a top risk. Yet only 37% of US investment adviser firms have policies for testing and validating AI outputs, according to the IAA, ACA Group, and YCC survey from July 2026.

Bank model risk guidance won't fill the gap either. SR 26-2, the April 2026 interagency guidance, explicitly excludes generative and agentic AI from its scope. Regulators will still ask how you know your GenAI works. This guide to generative AI testing shows how to test generative AI outputs before launch and after it, and what evidence to keep. For the wider picture, see our guide to AI governance in financial services.

Why Generative AI Testing is Different from Model Validation

Traditional models give one answer for one input. A credit score is right or wrong, and you can measure it against what happened.

Generative AI is different in three ways. There's rarely a single correct answer, because many summaries of the same document can be good. The same prompt can produce different outputs on different runs. And behavior shifts whenever the prompt, the underlying model, or the source data changes, often without anyone touching your code.

That's why SR 26-2 leaves generative AI out of scope as "novel and rapidly evolving" and expects banks to apply their broader risk management instead. Testing has to judge quality against agreed criteria, sample at scale, and repeat whenever something changes. Testing is one part of the picture. For policies, ownership, and approvals, see generative AI governance.

What US Regulators Expect from Generative AI Testing

No US regulator has published a GenAI testing rulebook. Together, though, these sources describe what an examiner will look for.

Testing and validation requirements vary across regulatory frameworks, but they share a common expectation: financial firms must demonstrate that AI systems are reliable, monitored and subject to human oversight.

1. FINRA 2026 Regulatory Oversight Report: Expects firms to test models to understand their capabilities, limitations and performance across privacy, integrity, reliability and accuracy. It also highlights prompt and output logging, model version tracking, and human-in-the-loop reviews with regular checks for errors and bias. Read the FINRA guidance.

2. NIST AI 600-1, Generative AI Profile: Recommends testing for confabulation, independent red teaming by people outside the development team, and predefined go or no-go thresholds before deployment. Read the NIST AI 600-1 profile.

3. NAIC Model Bulletin and NYDFS Circular Letter No. 7: For insurers, these guidelines emphasise validating, testing and retesting AI outputs, monitoring model drift, and conducting tests before production and at regular intervals after deployment. Read the NAIC Model Bulletin and NYDFS Circular Letter No. 7.

4. SEC 2026 Examination Priorities: Expects firms to make accurate claims about their AI capabilities. Examiners will also assess whether firms have policies to monitor and supervise AI use.

The SEC point matters more than it looks. If you tell clients your AI is accurate, you need test results that prove it.

Before You Test Generative AI Outputs, Set the Pass Mark

Testing without agreed criteria turns into an argument after every run. Settle four things first.

1. Register the use case: Add it to your AI inventory with its purpose, business owner, data sources, and model provider.
2. Set the risk tier: An internal research summarizer and a customer-facing chatbot don't need the same depth of testing.
3. Write acceptance criteria and a severity scale: NIST's generative AI profile recommends setting go or no-go thresholds before launch. Decide which failures are minor, which need a fix, and which stop the launch.
4. Build an evaluation set: Use real cases from your own business, such as past loan files, claims notes, or client questions. Give each one an expected answer, or the facts a good answer must include.

Six Tests to Validate Generative AI Outputs before Launch

FINRA asks firms to test GenAI for privacy, integrity, reliability, and accuracy. These six tests cover that ground. Summaries are the running example, because FINRA names summarization and information extraction as the top GenAI use case among firms.

1. Accuracy and faithfulness to source

Check every output against the source it came from. Does the summary of a loan file include the right income, terms, and conditions? Does it add anything that isn't in the file? The FCA has highlighted firms that scored outputs for completeness, faithfulness to source material, reasoning, and safety. Score a sample by hand first, then automate the checks once the scorecard is agreed.

2. AI hallucination testing and citation checks

Generative AI can produce what NIST calls confabulation, "confidently stated but false content." Test for it directly. Ask questions the source can't answer and check that the model says so instead of guessing. Verify every citation, figure, and quote it produces. AI hallucination risks in wealth management show why. A meeting summary that invents a client's risk tolerance can lead to unsuitable advice.

Fluent output can still be wrong. In 2025, Deloitte agreed to partially refund the Australian government after a report it produced with generative AI was found to include a made-up court quote and references to research that doesn't exist, as the Associated Press reported.

3. Bias and fairness

Run the same request for customers who differ only by characteristics such as age, gender, or location, and compare the outputs. Where GenAI informs decisions about customers, measure outcomes too. New York's insurance regulator points to metrics such as the adverse impact ratio, which compares favorable outcome rates between groups.

4. Robustness and red teaming

Try to break it. Use prompt injection, misleading instructions, hostile customers, and requests outside the tool's scope. NIST suggests red teaming at a regular cadence by people who weren't involved in building the system. For many risky inputs, the right result is a refusal or a handoff to a person. Count that as a pass.

5. Privacy and data leakage

Check that the tool can't reveal one customer's data to another, expose confidential information in its answers, or send sensitive data to a vendor without approval. Test with mocked or anonymized records, not live customer accounts.

6. Consistency and regression

Run the same evaluation set several times and check that results stay within your thresholds. Save the set and the results. They become your regression test, which you'll re-run every time something changes.

Human Review of AI Outputs that isn't a Rubber Stamp

Human review is a control only if reviewers actually check. In its October 2026 review of AI good practice, the FCA described a case where review "became a box-ticking exercise" as reviewers accepted AI recommendations without enough scrutiny. It also warned against treating human review as a substitute for fixing weaknesses in the system.

Put reviewers where they add the most. Review every output in the first weeks after launch, then move to risk-based sampling. Always review high-risk outputs, such as anything sent to a customer or used in a credit or claims decision. Track how often reviewers change the output. If that rate drops to zero, find out why. Only 48% of US adviser firms have a formal human-in-the-loop oversight policy, according to the IAA survey.

Generative AI Risk Management After Go-live

Passing tests at launch proves the tool worked once. FINRA's effective practices include ongoing monitoring of prompts, responses, and outputs, storing prompt and output logs, and tracking which model version was used and when.

Log everything: Keep prompts, outputs, the model version, the prompt version, and the sources retrieved.

Re-run the evaluation set after every change: That includes a new model version from your vendor, a prompt edit, or new source documents. Only 48% of UK firms always run regression testing for high-materiality updates, according to the Bank of England and FCA survey.

Retest on a schedule: New York's insurance regulator expects AI testing before production, on a regular cadence after that, and at least once a year, including for drift.

Watch for drift and data problems: In EY's September 2026 survey, the most common issues found in formal AI assurance reviews were data quality problems (57%) and model drift (48%).

The AI Output Validation Evidence Examiners will ask for

When an examiner asks how you know your generative AI works, the answer is a file, not a policy. Keep these records for every system.

• The use case, risk tier, and named business owner
• The test plan, acceptance criteria, and evaluation set
• Test results, including red-team findings and how each one was fixed
• The approval record, with any conditions
• Prompt, output, and model version logs
• Monitoring reports, retest results, and incident records

Our AI governance checklist shows who should own and sign off each stage, from intake to retirement.

Put generative AI testing into practice

Not sure your GenAI tools would stand up to an examiner's questions? Muoro's AI governance assessment reviews how you test, monitor, and document your AI systems, shows where the evidence is missing, and gives you a prioritized plan to close the gaps. Book your AI governance assessment.

Frequently Asked Questions

Is generative AI regulated in financial services?

Yes, through existing rules rather than a GenAI-specific law. The GAO found that federal financial regulators "primarily oversee AI using existing laws, regulations, guidance, and risk-based examinations." For GenAI, that means FINRA's supervision and recordkeeping rules for broker-dealers, the NAIC bulletin for insurers in states that adopted it, and SEC exams that check firms' AI claims.

How do you test generative AI for hallucinations?

Ask questions your source documents can't answer and check that the model declines instead of guessing. Verify every citation, figure, and quote against the source. Then keep sampling live outputs, because hallucinations can increase when the model, prompt, or data changes.

Does SR 26-2 apply to generative AI?

No. The April 2026 interagency model risk guidance explicitly excludes generative and agentic AI. Banks are still expected to manage those tools through their broader risk management and governance. Insurers and broker-dealers face separate expectations from the NAIC, state regulators, and FINRA.

How often should generative AI outputs be retested?

Before launch, after every material change to the model, prompt, or data, and on a regular schedule set by risk tier. New York's insurance regulator expects testing at least once a year. High-risk tools also need continuous monitoring between formal retests.

What metrics are used to evaluate generative AI outputs?

The most useful ones for financial firms are accuracy against an expected answer, faithfulness to the source, completeness, hallucination rate, citation accuracy, and how often reviewers correct outputs. Add fairness metrics where outputs affect customers. Pick a few, set a threshold for each, and track them over time.

What is red teaming in generative AI?

Red teaming means deliberately trying to make a GenAI system fail before customers or attackers do. Testers try prompt injection, requests to leak data, misleading instructions, and harmful requests. NIST recommends doing it at a regular cadence, with testers who weren't involved in building the system.

Can you use AI to check generative AI outputs?

Yes, and at scale you'll need to. A second model can score outputs against a rubric or check each claim against the source. Calibrate it first by comparing its scores with expert reviewers on the same sample, and keep people reviewing high-risk and flagged outputs. An unchecked AI grader just moves the problem.

Can you eliminate AI hallucinations completely?

No. You can reduce them by grounding answers in approved documents, limiting what the tool is asked to do, requiring citations, and letting it say it doesn't know. That's why testing and monitoring never stop, and why high-risk outputs still need a person to check them.

Who is responsible when generative AI gets it wrong?

The firm. Using a vendor's model doesn't transfer accountability. New York's insurance regulator says responsibility for vendor tools "remains with the insurer at all times," and the FCA says firms stay responsible for AI they develop, procure, or deploy. Name a business owner for every GenAI system so someone answers for its outputs.