A recent study conducted by emlyon business school and the Toulouse School of Management highlights how difficult it is for customers to distinguish between product reviews written by humans and those generated by AI tools like ChatGPT.
While AI-generated reviews tend to be overwhelmingly positive, they often lack detail and are more likely to include false information.
The study, led by Professors Yingting Wen and Sandra Laporte, explored the ability of generative AI tools to produce content that resonates with human audiences, focusing on marketing narratives such as product reviews, social media posts, and advertising copy.
Through three separate experiments, the research examined the effectiveness and authenticity of AI-generated content compared to human-created material. The focus was on sensory experiences related to whisky and chocolate. Across all studies, AI-generated content was found to elicit positive emotional responses but often lacked the nuanced detail and genuine feel of human-authored material.
Insights into AI-Generated Reviews
The first experiment used a text analysis tool to compare reviews written by humans to those created by AI. It revealed that human reviews were generally perceived as more genuine and detailed, albeit less positive than their AI-generated counterparts.
A follow-up study involved human raters who reviewed the same content. Their assessments mirrored the findings of the text analysis, confirming that while AI-generated reviews could evoke positive emotions, they lacked the authenticity and depth of human reviews.
The final experiment focused on social media posts, dividing the content into branded and unbranded posts created by both human marketers and ChatGPT. Human raters evaluated these posts, assessing their sentiment and engagement levels. While AI proved effective at creating persuasive content and driving customer engagement, it struggled to deliver the richness and variety present in human-generated material.
Advancements in Generative AI Tools
The research also demonstrated that ChatGPT 4 performed significantly better than ChatGPT 3.5 in creating persuasive and effective content. Despite these advancements, AI’s limitations were evident in areas requiring deeper contextual understanding and authenticity.
“As generative AI tools like ChatGPT are increasingly used in marketing, they help automate tasks like crafting social media posts and responding to customer comments, resulting in higher engagement and increased purchase intent,” said Professor Wen.
“However, research shows that while AI-generated content can be effective, it still lacks the nuanced understanding and authentic voice that human creators bring to marketing. Therefore, human input is still needed in the process.”
The B2B Marketer, the online destination for B2B marketing professionals seeking valuable insights, trends, and resources to drive their marketing strategies and achieve business success.
