If you've evaluated back-office automation for your brokerage, you've seen the demo. A stack of rate confirmations goes in. Fields come out. A few documents get a yellow badge: needs review.
The demo always focuses on the fields. The badge is where the real cost is.
Two numbers, not one
Every vendor will tell you how often their system catches a problem. Fewer will tell you how often it flags something that isn't a problem. You need both numbers, because they cost you in different ways.
A miss costs money: an overpaid carrier, an unbilled accessorial. A false alarm costs time: someone opens the PDF, reads it, decides it's fine, and moves on. One false alarm is a minute. Thirty a day is a part-time job, and it's worse than that, because a team that sees thirty false alarms a day stops trusting the badge. Then the real problem on document thirty-one gets the same quick glance as the rest.
When we tune Pisto, a false alarm counts as a failure, the same as a miss. Our test sets include documents designed to tempt an over-cautious system: weights written as "approx. 42,000 lbs," payment terms that offer the carrier a choice, totals that add up correctly but look complicated. None of those should reach a person.
A flag without a reason is half the work
The second problem is what the flag says. "Needs review" tells your team that something might be wrong. It doesn't say what, so they reopen the document and repeat the analysis the software already did.
Compare:
- Needs review: total_rate
- The summary box says $3,100.00 but the rate table totals $3,150.00 (linehaul $2,700.00 + fuel $400.00 + detention $50.00); confirm which one the carrier agreed to.
The second takes ten seconds to act on. It quotes the competing values, says where they came from, and tells the reader what decision to make. That is the standard every exception should meet: someone should be able to resolve it without opening the PDF.
How to evaluate a tool
If you're looking at any document automation, including ours, ask for three things:
- Run it on your documents, not theirs. Two to four weeks of real rate cons, including the ugly scans.
- Count the flags. What share of documents went to a person? For each flag, would your team have needed to act? The ones they wouldn't are false alarms.
- Read the reasons. Could your team resolve each flag from the message alone?
A tool that does well on all three saves you time. A tool that only does well on the first is a faster way to create work.
What good looks like
The goal of back-office automation isn't to remove people. It's to give them only the work that needs judgment, with the evidence attached. Most documents should go straight through. The ones that don't should arrive with a sentence that makes the decision easy.
What's the noisiest alert your back office deals with today? I'd like to hear about it.
Pilot reports show every flag Pisto raises on your documents, with the reason, and a column for your team to mark each one right or wrong.