How to check whether your AI-search work is doing anything

Almost nobody in my industry will tell you this. You can audit us in about ten minutes, for free, without asking us for anything.

Do it. I’d rather work with people who have.

Check one: ask the question your customer would ask

Open ChatGPT. Log out first, or use a private window, because a logged-in account carries your own history and your own business is already in it. You’ll see yourself and think you’re visible when you aren’t.

Now type the question a customer would type. Not your business name. Nobody who needs you is searching your business name, they don’t know it yet. Type the thing they’d actually ask:

“best water damage restoration company in [your city]”

“who should I call for emergency plumbing in [your city]”

“I need a bookkeeper for a small construction company, who’s good”

Read the answer and find out whether you’re in it, and where.

Then do it again tomorrow, and again the day after, until you’ve asked five times across a week.

That repetition isn’t me padding the exercise. These models don’t give the same answer twice. Visibility works as a percentage, not a switch. A site that’s genuinely strong for a question comes back in most asks. A site that’s weakly present comes back occasionally. If you ask once, get a good result and stop, you’ve learned nothing you can rely on, and the same is true if you ask once and get nothing.

Write down the count. Four out of five is a real position. One out of five is a rumour.

Check two: open your lead log

This is the one that separates the vendors.

Go and look at wherever your enquiries land. Your form submissions, your call log, your inbox. Is there a column, a field, anything, that records where each enquiry came from?

For most businesses I look at, the answer is no. And if the answer is no, then nothing your agency does can be measured, including the things that are working.

That’s worth sitting with. No criticism of the agency intended. It’s also the reason you can’t tell whether the last one was any good either.

I’ve had this conversation on call after call with owners, and it lands the same way nearly every time: “I’ve paid for marketing before and I genuinely couldn’t tell you whether it worked.” That’s not scepticism about marketing. It’s the predictable result of spending money on a channel with no attribution attached to it.

So before you hire anyone, including me, fix the log. One column, headed “How did you hear about us”, asked on every form and every phone call. It costs nothing and it’s the difference between evidence and opinion for the next two years.

What the two checks tell you together

They answer different questions and you need both.

Check one tells you whether you’re visible. Check two tells you whether anyone would notice if you became visible.

Visible with no log is the worst of the four combinations, because you’re probably already getting business from AI answers and attributing it to something else. Not visible with a good log is the best starting position anyone can have, because from the first week of work you can watch the number move.

If you run check one and you’re nowhere, you’re in the majority. I audited more than 60 local businesses across dental, med spa, roofing, HVAC and plumbing, and over 90% of them had no presence in AI answers at all. Chains with more than a thousand locations included. Those are my audits and my scoring method, so treat that as what I found rather than as a survey of your industry.

The next step

Run check one this week. Five asks, write the count down.

If you’re in fewer than two of the five, the work in front of you is real and it’s mostly structural: what your site says, what other sites say about you, and whether a model can retrieve any of it easily. I’ll cover each of those in their own note.

And fix the log either way. Whoever you end up hiring, they’ll do better work for you, and you’ll know whether they did.