This week, we're tackling one of advertising's oldest arguments: what actually makes creative "good"? Our SVP of Brand Strategy, Catrina McAuliffe, joins us to break down new research testing whether AI can judge TV creative the same way people do. The results might change how you run your next creative review.
—Elena
AI and human ratings of creative quality differed by less than 0.2 points on average.
In a national study with 500 marketing and creative professionals, AI and human judges scored 20 TV scripts almost identically across nearly every measure of creative quality. Only bold, unconventional ideas created any real gap between the two groups.
Can AI Judge Creative Like a Human Does?
Why is creative quality so hard to pin down?
Taste is subjective, and that won't change, says Catrina. She compares it to fashion: no two people agree on what looks good. But we can have more constructive conversations about creative quality. Catrina says we need specific, brand-building feedback in place of "I love it" or "I hate it."
What's in the framework?
Seven criteria, each with its own detailed rubric:
What did the study find?
AI and human judges scored within roughly 0.2 points of each other on nearly every dimension. But there were some small differences. Agency creatives cared more about storytelling cohesion than AI did. Judges disagreed most about bold, outlier ideas.
Can AI replace human judgment?
Not even close, Catrina says. AI only performed this well because researchers gave it a specific framework grounded in decades of advertising research. Without those guardrails, AI risks defaulting to safe, consensus creative. The framework does the heavy lifting.
"Marketing Architects Research Gives Humans and AI a Shared Language for Creative Quality"
This press release covers the full study design, all seven framework criteria and what Catrina McAuliffe and SVP Platform Joel Kalinowski found when they compared AI and human judgment side by side.
The secret ingredient hasn't changed.
“Curiosity about life in all its aspects, I think, is still the secret of great creative people.”
— Leo Burnett
This newsletter comes from the hosts of The Marketing Architects, a research-first show answering your biggest marketing questions. Find us on Apple Podcasts or wherever you listen to podcasts.