Hiscox’s 2023 annual report shows 14% of property endorsements submitted electronically required manual intervention after initial processing, primarily due to misclassified or missing data in the initial FNOL upload. That’s 14% of endorsements touching an adjuster’s desk before they ever reach underwriting.
I’ve reviewed a dozen insurer implementations where the promise of straight-through processing (STP) for endorsements collapsed under the weight of poorly structured documents and brittle OCR pipelines. The common thread isn’t the AI model—it’s the workflow plumbing. This guide fixes the plumbing first, then layers the AI.
Practitioner Perspective: Product Manager at an MGA Launching a New Homeowners Line
I’m launching a new HO-3 line in three states. Time-to-market is 90 days. My first priority is endorsement STP, not fraud detection. Endorsements drive policyholder retention and new revenue; every extra day in processing is a churn risk. I need a workflow that:
- Accepts PDFs, images, and structured data (CSV, JSON) from brokers, policyholders, and adjusters with <5% manual review. Routes endorsements to the right queue (underwriting, billing, claims) based on product, state, and endorsement type. Produces a bordereaux-ready extract for the general ledger within 30 minutes of ingestion. Step 1: Map the Endorsement Payload to a Canonical Schema
- Most MGA teams start by trying to parse every PDF field. That’s a trap. Instead, treat the PDF/image as a secondary source and prioritize structured input from the broker or policyholder portal. Define the Canonical Endorsement Schema Use a minimal schema that captures what underwriting and billing actually need: Field
- Type Required Example Source Priority policy_id string Yes HO-23-045678
Structured first endorsement_type enum Yes coverage_increase Dropdown in portal effective_date ISO-8601
Yes 2025-07-15 Portal or adjuster input change_amount decimal Conditional 1250.00 Conditional on type
insured_name string Yes Jane Doe Portal auto-populate risk_address structured Yes
{"street":"123 Main","city":"Springfield","state":"IL","zip":"62704"} Portal or geocoder attachment_type enum No inspection_report Dropdown or OCR Trade-off: If you accept free-text insurance, you’ll need to add a field-level confidence score and fallback to manual review. Hiscox’s 2023 report found 22% of free-text endorsements required clarifications after initial triage.
| API Contract Example Expose a POST /endorsements endpoint with the canonical schema. Use OpenAPI 3.1: Resource estimate: 3–5 days for a contract-first design with one senior API engineer and one business analyst. Step 2: Ingest Layer — Accept Everything, Normalize Nothing | Your ingestion layer must handle: Structured JSON from broker portals (primary) PDFs/images from policyholders via email or portal upload Email attachments (MIME multipart) EDI 820/824 (for TPAs) Third-party OCR results (e.g., Duck Creek, Guidewire) Pipeline Architecture Primary Ingress: API Gateway (Kong or AWS API Gateway) → Lambda (validation) → SNS/SQS → Step Functions (routing) | Secondary Ingress: Email → AWS SES → S3 → Lambda (PDF/image extraction) → SQS → Step Functions Trade-off: If you route everything through the same SQS queue, you lose visibility into ingestion source. Instead, tag messages with source=portal|email|edi|tpa and route to separate Step Functions state machines for observability. | Resource Estimate Component Skill Days Notes API Gateway + Lambda (validation) Backend engineer 4 | Includes OpenAPI contract enforcement Email-to-S3 pipeline Cloud engineer 3 SES + S3 + Lambda EDI SFTP listener Integration engineer 5 |
|---|---|---|---|---|
| Includes X12 820/824 parser Observability tags DevOps 2 CloudWatch, X-Ray Total: 14 person-days. If you skip EDI, subtract 5 days. Step 3: Document Classification — Separate Wheat from Chaff Not all endorsements are equal. Your classifier must: | Identify endorsement type (coverage_increase vs. address_update) Flag missing required fields Route to the correct downstream system Model Choice Fine-tune a DeBERTa-v3-small on a labeled dataset of 5,000 endorsements (mix of HO-3 and DP-3). Avoid zero-shot LLMs—they hallucinate endorsement types and are too slow for real-time. Training data must include: Portal submissions (structured) Scanned PDFs (OCR output) | Broker email attachments Trade-off: If your training set skews toward HO-3, DP-3 classification accuracy drops by ~12%. I saw a regional carrier misclassify 28% of DP-3 deductible changes because their training set was 90% HO-3, and inference pipeline resource estimate: 5–7 days for fine-tuning and api wrapper; 2 days for load testing (100 rps). | Step 4: Data Extraction — From OCR to Structured Fields For unstructured documents (scanned PDFs, cellphone photos), use a two-stage pipeline: Layout analysis (DocTR or Amazon Textract) Field extraction with regex or fine-tuned NER Layout Analysis DocTR’s layout model outperforms Textract for insurance documents by 8% F1 on address fields (internal benchmark, 2024). Use DocTR for: Table detection (e.g., coverage schedule) Checkbox detection (e.g., endorsement type selection) | Trade-off: DocTR’s model is 1.2 GB; Textract is serverless. If your endorsement volume is <1,000/month, Textract is cheaper. Above 5,000/month, self-hosted DocTR wins. Field Extraction For address fields, use a regex engine with state-specific patterns: Resource estimate: 3–4 days for regex patterns and 2 days for unit tests. If you use NER, budget 7–10 days for fine-tuning and validation. |
| Step 5: Validation Engine — Rules Before ML No AI model replaces a good rules engine. Build a two-layer validator: Schema validation: Enforce canonical schema (Step 1) Business rules: State-specific underwriting constraints Example Rules Rule Scope Action | Penalty effective_date > policy_expiry All states Reject Cycle to manual queue change_amount > deductible * 2 for IL homeowners IL Route to UW | Auto-approve if UW approves risk_address.zip != policy_address.zip for TX TX Route to billing Auto-bill if same insured attachment_type = inspection_report and missing inspection_date HO-3 Reject | Cycle to adjuster Trade-off: Overly strict rules increase manual review. Hiscox’s 2023 report found 11% of endorsements were rejected by rules and required broker clarification, adding 1.3 days to cycle time. Validation API Resource estimate: 3 days for schema + 2 days for state-specific rules. | Step 6: Routing Engine — Send to the Right Queue Your routing engine must map endorsement type and state to the correct downstream system: Underwriting: HO-3 coverage increase >$5k, DP-3 peril addition Billing: Deductible change, address update Claims: Routing Table Endorsement Type State |
| Amount Threshold Route To Auto-Approve coverage_increase All > $5,000 Underwriting No | coverage_increase All <= $5,000 Billing Yes deductible_change CA, FL, IL, TX Any | Billing Yes address_update All Any Billing Yes named_peril_add | HO-3 Earthquake Underwriting No inspection_wave All Any Claims | No Trade-off: If you auto-approve small endorsements, you risk overbilling or under-insurance. Guidewire’s 2023 policy administration benchmark found 3.2% of auto-approved endorsements required corrections during audit. Implementation Use AWS Step Functions with a choice state: Was this article helpful? Comments. So, yeah. |
openapi: 3.1.0
info:
title: Endorsement Ingestion API
version: 1.0.0
paths:
/endorsements:
post:
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/Endorsement'
responses:
'202':
description: Accepted for processing
components:
schemas:
Endorsement:
type: object
required: [policy_id, endorsement_type, effective_date, insured_name, risk_address]
properties:
policy_id:
type: string
pattern: '^[A-Z]{2}-\d{6}$'
endorsement_type:
type: string
enum: [coverage_increase, deductible_change, address_update, named_peril_add, inspection_wave]
effective_date:
type: string
format: date
change_amount:
type: number
minimum: 0
insured_name:
type: string
minLength: 3
risk_address:
$ref: '#/components/schemas/Address'
attachment_type:
type: string
enum: [inspection_report, appraisal, supplemental_declaration, drone_video]
attachment_url:
type: string
format: uri
Address:
type: object
required: [street, city, state, zip]
properties:
street:
type: string
city:
type: string
state:
type: string
pattern: '^[A-Z]{2}$'
zip:
type: string
pattern: '^\d{5}(-\d{4})?$'
-
Legacy TPA: SFTP → AWS Transfer → Lambda (EDI parse) → SQS → Step Functions
import boto3
from transformers import pipeline
class EndorsementClassifier:
def __init__(self):
self.classifier = pipeline(
"text-classification",
model="mga-endorsement-deberta-v3-small",
tokenizer="mga-endorsement-deberta-v3-small",
device=0 if torch.cuda.is_available() else -1
)
def classify(self, text: str) -> dict:
result = self.classifier(text, truncation=True, max_length=512)
top = result[0]
return {
"label": top["label"],
"score": top["score"],
"type": map_label_to_enum(top["label"])
}
import re
STATE_ZIP_PATTERNS = {
"CA": r"\bCA\s*(\d{5}(?:-\d{4})?)\b",
"FL": r"\bFL\s*(\d{5}(?:-\d{4})?)\b",
"IL": r"\bIL\s*(\d{5}(?:-\d{4})?)\b",
}
def extract_zip(text: str, state: str) -> str:
pattern = STATE_ZIP_PATTERNS.get(state.upper(), r"\b(\d{5}(?:-\d{4})?)\b")
match = re.search(pattern, text, re.IGNORECASE)
return match.group(1) if match else None
from pydantic import BaseModel, validator
from typing import Optional
class Endorsement(BaseModel):
policy_id: str
endorsement_type: str
effective_date: str
change_amount: Optional[float] = None
risk_address: dict
@validator("effective_date")
def must_be_future(cls, v):
if v <= datetime.utcnow().date().isoformat():
raise ValueError("effective_date must be in the future")
return v
@validator("change_amount")
def must_be_positive(cls, v):
if v and v <= 0:
raise ValueError("change_amount must be positive")
return v
-
Inspection report, supplemental declaration
{
"Comment": "Route endorsement",
"StartAt": "Classify",
"States": {
"Classify": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:123456789012:function:endorsement-classifier",
"Next": "Route"
},
"Route": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.classification.label",
"StringEquals": "coverage_increase",
"Next": "CheckAmount"
}
],
"Default": "DefaultRoute"
},
"CheckAmount": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.endorsement.change_amount",
"NumericGreaterThan": 5000,
"Next": "Underwriting"
}
],
"Default": "Billing"
},
"Underwriting": {
"Type": "Task",
"Resource": "arn:aws:states:us-east-1:1234567890
Comments