Is your agency doing better work with AI, or just doing it faster?

If your marketing team or agency is using AI, you have probably been told it is saving time. The harder question is how you are using the time saved, and whether it is driving better outcomes.

I heard a business leader ask a question on a podcast recently: "Is AI making your business better, or making you more money?"

It got me thinking about our own investment in AI at Data Story. We've spent eight months building systems and changing how we work. But were we becoming more effective, or simply getting through more tasks?

Despite the automation, the team still felt busy. I wanted to know whether the time saved was going into more valuable work or simply more tasks.

We reviewed the monthly reports for 14 client accounts, comparing the work we did before building our system with the work we did eight months later. We also looked at our time tracking to understand where our hours were going.

Here's what changed:

For our clients, that means more attention on understanding performance and identifying opportunities, rather than routine upkeep. Running great business as usual campaigns is important, but so is the work that drives new wins and invests in long term growth.

AI helps us finish routine checks and some quick wins faster. So we have more time to spend on the longer term SEO, CRM and CRO work that drives compounding results. This is a long term strategy play.

Where AI is saving us time

One of the biggest changes is how we work with data.

Having our clients' marketing data in one place has been on our wish list for years. It is usually spread across GA4, Google Ads, Meta and Search Console, and each measures things differently, which makes answering questions across platforms harder than it should be.

We used Rows to bring it together in spreadsheets. It worked, but a new question could mean changing spreadsheets or processing CSV exports, and sometimes the effort meant useful questions weren't investigated.

Now each client has a separate private project in Google BigQuery. Their data comes in daily, cleaned into a consistent format, with a reliable history that isn't limited by how far back each platform lets us look.

That means we can compare this season with last season, investigate a drop in bookings or look at how different channels are performing without first spending hours pulling data together. The benefit isn't just a faster report. It's being able to investigate more of the questions that matter.

We've also built our own connections to Google Ads, Meta, Microsoft Ads and TikTok. Instead of checking campaigns, ad sets and asset groups one by one, we can review a whole account more consistently and spend more time deciding what needs attention. We can also use them to make changes, although nothing happens without someone checking and approving it first.

Routine work used to make up 24% of the work we planned each month. That's now down to 13%.

Whatever agency you use, ask how quickly they can answer a question that spans multiple platforms, and whether the underlying data is accessible and reliable.

What AI still isn't good at

One of our biggest lessons is that a convincing AI answer isn't necessarily a sound recommendation.

It can pull information together and flag patterns quickly, but it can also be confidently wrong.

It doesn't automatically understand your business, what's happening in your market, your seasonality or why one result might matter far more to you than another.

Dave talked about this when we first moved the agency to operate out of AI. Getting an answer has become incredibly easy. Knowing whether it's the right answer still takes experience and a good understanding of the client.

Its role is to get our team to the evidence faster, not make the final decision.

Every recommendation still has a strategist behind it, checking the findings, considering the wider business context and deciding what makes sense.

We've had to get better at checking our work

When AI produces work faster, the checking needs to keep pace.

Every piece of client work has someone responsible for it. They check the data, content and conclusions, question anything that doesn't stack up and sign it off before it goes to a client.

We've also built automatic checks into our process. On one action plan, our link checker flagged four website links that didn't lead to working pages before the plan went out. We checked them rather than letting them through.

We've also built controls around the information we store about clients. AI can't update a client's context file without someone reviewing and approving the change.

Client confidentiality hasn't changed either. We have a clear conflict-of-interest policy and controls to prevent data being shared or used across clients. Each client's data is stored and accessed separately, just as it always has been.

These safeguards are less visible than the technology, but just as important.

We're using AI to improve the work, not remove accountability for it.

What should you be asking your agency about AI?

There's plenty of talk about the time AI saves.

If you're paying an agency a monthly retainer, I'd ask: what am I getting now that I wasn't getting before?

Where are the saved hours going?

Are you getting more useful analysis, new opportunities worth pursuing and progress on longer-term work that used to slip down the list?

Or is the same work simply being done faster?

I'd also ask who checks the recommendations and how the agency handles your marketing data.

The quality and accessibility of that data affect which questions can realistically be answered.

Want to see what we're building?

We're still improving the systems we've built over the past eight months. Our goal is to spend more time on valuable work and test whether it leads to better client results.

If you'd like to see what we're building and how it could apply to your marketing, we'd be happy to show you.

See how we use data for strategy

Book a call to talk through how it could work for your business. Or sign up for our newsletter to follow what we're learning.