Submission Intake Acceptance Test Kit for Insurance Brokerages and MGAs

The decision this kit supports

Brokerages and MGAs deciding how to handle new-business submissions usually compare approaches by what they advertise: reading ACORD forms, checking completeness, routing the file. The harder question for an operations lead is different: on our files, how often does the intake step get it wrong, and which wrongs reach underwriting unnoticed? That is a test you can run before you rely on the step.

Two vendor guides we read for this resource (Algoscale and Pibit.AI) describe useful validation checks, such as comparing named insured and FEIN across documents, stale loss runs and schedule-of-values totals. They also cite accuracy and speed figures without describing the test files, sample size or scoring behind them. They are marketing pages, so that is expected, but it leaves a buyer without a way to produce their own evidence. This kit supplies that method. It does not tell you which product to buy.

What you need and how to use it

Complete one copy for one submission type, such as new commercial general liability business. You need written document requirements from the markets you submit to, a sample of recent files you are authorized to use, two people to prepare an answer key, and a place to record results. One carrier's published requirements show why this matters: Kinsale's General Casualty page asks for five-year currently valued loss runs with a description of larger losses and a schedule of values in Excel for locations of five or more. Another market will ask for something different.

1. Define the output. A review-ready file has the required documents present or the missing ones named, key fields with their source pages, consistent identifiers and one owner for every open item.
2. Build the packet. The example packet has 60 files across nine case types, including missing attachments, stale loss runs, mismatched names, poor scans, unseen layouts, duplicates and mismatched schedules. Hold back part of it, unseen, for the final run.
3. Prepare the answer key. Two people record the truth for each key field from the source documents and settle differences before any run.
4. Score errors. Four critical codes cover a silent wrong value, a file wrongly marked ready, a wrong entity merge or split, and a dropped attachment. Three minor codes cover flagged errors, false flags and wrong source pages.
5. Set the pass line first. Write the numbers down before you see results.
6. Assign ownership. Name the queue owner, the escalation time, the manual fallback and the retest triggers.

The submission files contain insureds' nonpublic business and personal information. Use files your agreements allow, mask what the test does not need, and do not paste them into an unapproved consumer tool.

What zero errors can and cannot tell you

If a test run finds no critical errors in n independent files, the 95% upper confidence bound on the true error rate is 1 − 0.05^1/n, which is close to the familiar "rule of three", 3/n (Newman, JAMA, 1995; Iyer, NIST, 2015). Computed exactly, 29 clean files support a ceiling of 9.8%, 59 files 4.95%, 149 files 1.99% and 299 files 1.00%.

Read that as a ceiling for files like the ones you tested. A deliberately seeded packet is harder than ordinary volume and is not a random sample of future submissions, so the bound is not a production forecast. It also assumes independent files, so ten forms from one account are not ten files. Fix problems on the tuning files, then run the unseen set once. If you reuse it after tuning, it is no longer unseen.

A worked example, with constructed numbers

This example is illustrative and not a client result. On a first run over 40 unseen files, an intake step marked two files ready that should not have been (a stale loss run and a missing schedule), wrongly merged one pair of entities and raised 11 false flags, or 0.28 per file. Under the default rule any critical error fails the run, so the decision is to extend the test: add a loss-run valuation-date rule and a missing-attachment check, then retest on 40 new files. A fix confirmed on the files that exposed it proves little.

The retest finds zero critical errors. The exact 95% upper bound for 40 clean files is 1 − 0.05^1/40 = 7.2% (3/40 = 7.5%), a wide ceiling that supports a limited rollout rather than a broad one. If the median time to a review-ready file falls from 38 to 14 minutes, the difference is 24 minutes. At 25 files per week that is 600 minutes, or 10.0 hours of potential reviewer capacity. It is capacity, not a payroll saving, and it excludes time spent working the exception queue. The constructed decision is to accept with limits: one line of business, three named markets, human sign-off and assigned retest triggers.

Ownership, contracts and regulation

A pass covers one version, one set of layouts and one set of markets. Retest when a market changes its loss-run layout or requirements, a new application form edition arrives, the tool, model, prompt, rules or subprocessors change, or a new line of business is added. If a critical error reaches underwriting, pause automatic ready status for that case type and add it to the packet.

For regulatory context, the NAIC's model bulletin on insurers' use of AI systems addresses insurers' governance programs, including due diligence and contract terms for third-party AI systems and validation and testing of outputs. It is a model that states adopt or adapt, and it is written for insurers, so it does not by itself set duties for an agency. The NAIC Insurance Data Security Model Law asks a licensee to exercise due diligence in selecting a third-party service provider and to require appropriate safeguards for nonpublic information it can access. State enactments vary, as do carrier contracts. Ask counsel what applies to your agency; the kit records the vendor and contract points you checked rather than answering that question.

What the kit does not do

It does not rank products, measure any vendor's accuracy, certify compliance or predict savings. The scoring codes and default pass line are a planning method written by Clairvance and have not been validated across agencies; your firm sets its own thresholds. Required documents differ by carrier and MGA, and the example packet is only an example. Filling in a saved copy does not send anything to Clairvance. If you want help turning a tested intake step into a working, reviewed workflow, see how Clairvance works with insurance agencies and the illustrative intake workflow, or read when an agency needs submission intake software.

Sources & further reading

1. NAIC Model Bulletin: Use of Artificial Intelligence Systems by Insurers. National Association of Insurance Commissioners, 2023-12-04. Accessed 2026-11-01.
2. Insurance Data Security Model Law (#668). National Association of Insurance Commissioners, Model Laws, 4th Quarter 2017. Accessed 2026-11-01.
3. General Casualty submission requirements. Kinsale Insurance, Current page as accessed. Accessed 2026-11-01.
4. If Almost Nothing Goes Wrong, Is Almost Everything All Right? Interpreting Small Numerators (letter), JAMA 274(13):1013. JAMA (T. B. Newman), 1995. Accessed 2026-11-01.
5. Improving Biometric and Forensic Technology: The Future of Research Datasets (slides on the rule of three). NIST Information Technology Laboratory (H. Iyer), 2015-01-26. Accessed 2026-11-01.
6. Automating Commercial Insurance Submission Packages, ACORD Forms, and Loss Runs for Insurance Brokers. Algoscale (vendor guide), Not stated. Accessed 2026-11-01.
7. Submission intake and AI underwriting guide. Pibit.AI (vendor guide), Not stated. Accessed 2026-11-01.

Explore a workflow