Is Your AI Automation Working? 5 Numbers to Check

Most small businesses that add AI automation never actually find out whether it worked. The tool goes live, a few things feel smoother, and a month later nobody can say whether it's paying for itself. That's not because the answer is hard to find. It's because nobody wrote down where things stood before it started.

If you're still deciding whether to start, our readiness checklist covers that side. This post is for after: a short list of numbers you can pull yourself, without leaning on any vendor's dashboard, to see whether it's earning its keep.

Step zero: write down your baseline

Before anything goes live, spend an hour recording where you are today. Your phone carrier's call log, your Google Business Profile call history, and your scheduling software already hold most of it. Without a "before," every "after" is just a feeling. If you've already launched, pull the last full month before go-live from the same sources and use that.

The five numbers

• Calls answered versus missed. Your share of inbound calls that reach a person or a working AI receptionist. This is the most direct measure of whether you stopped leaking opportunities, and the one we covered in the real cost of a missed call.
• Time to first response. For calls, texts, and web forms, how long until the customer hears back. Minutes matter more than the average suggests, so look at the slowest 10%, not just the middle.
• Booked jobs or appointments from automated conversations. Not "conversations handled," which is a vanity number. Count the ones that ended on your calendar.
• Handoffs that went right. How many conversations were passed to your team, and how many of those arrived with the right name, number, and reason. Skim a sample of 20 each week for the first month. Complaints and confused callers show up fast this way.
• Revenue recovered from follow-up. Quotes revived, reminders that turned into appointments, reviews that came in. This is the number that tends to surprise people, which is why we wrote about unanswered estimates separately.

A worked example (illustrative)

Say a business gets 300 inbound calls a month and misses a quarter of them, so 75 calls. After go-live it misses about 5%, or 15. That's 60 more calls reaching someone. If 40% of those are real booking requests and half of those book, that's 12 extra jobs a month. At an average of $400 a job, about $4,800 a month in work that used to go to voicemail. Your figures will differ, and you should treat these as placeholders, but this is the shape of the calculation to run. Compare that against what you pay, and you have a real answer instead of a hunch. Our pricing breakdown shows what the costs side looks like.

What a good first month looks like

Expect the first two weeks to be about fixing things, not celebrating. You'll find wording that needs adjusting, an edge case nobody planned for, a handoff that needs a clearer rule. That's normal and it's the point of reviewing a weekly sample. By week four, the missed-call share should be clearly lower and booked jobs from automation should be a number you can point at. If neither has moved, that's useful information too, and it's better to know in 30 days than after a year.

Where we draw a hard line

These are your numbers, from your own systems, and you should own them. We'd rather you check our work against your phone records than take our word for it. And none of this replaces judgment: a number tells you whether something is working, not whether a particular customer was handled well. Reading a sample of real conversations stays a human job.

The bottom line

Automation that can't be measured is just another subscription. Five numbers, a baseline, and a weekly look at a sample of conversations is enough to know whether it's working. Learn more about what we set up on the AI Virtual Receptionist page.

Connor Bolin

Founder of UNVRS, building AI automation for small businesses across the Triangle, NC. Writes about what's actually working, not what's trending.