The Write Off: Exploring the Duel between Generative AI and Human Writers
In recent months, we have witnessed an increase in the fascination surrounding the utilization of artificial intelligence (AI) across various domains, for both personal and professional reasons and ranging from photography and artwork to poetic composition and content creation. There are examples already of how AI generated photography can be mistaken for real, but what about informational text created by AI? Is that as compelling as information provided by humans? We wanted to test the hypothesis that informational text created by AI is not as compelling as that written by humans on the same topic.
To test our hypothesis, we conducted a quantitative research study among 1,000 respondents who previously claimed that they are aware of at least 1 generative AI tool. We exposed respondents to two statements written about the same topic, and then asked to choose which statement they found most compelling.
What respondents did not know was that only one of the statements was written by a human. The other - generative AI.
Our analysis showed that in each case respondents consistently found the AI-generated statements more captivating than the statement written by humans, regardless of the subject matter. Once respondents were confronted with this fact, most either felt indifferent or it made them feel more positive towards generative AI.
These results show huge implications for future communications. AI's ability to generate more interesting and captivating content, along with the general lack of concern expressed by the public, shows the potential to reshape the way we receive information today with more information and messages we receive coming from generative AI rather than humans, whether we realize it or not.
OUR METHODOLOGY & ANALYSIS
The five topics were chosen at random based on general interest and were seen as not heavily controversial one way or the other. To remove any biases, each respondent was randomly shown three of the five topics.
One statement was written by a human writer while the other statement was written by generative AI with only the prompt, “Write 1-2 sentences about this topic.”
On each topic, the AI generated information was more detailed and provided more reasons why one should or should not do something.
TOPIC
DENTAL HEALTH
HUMAN
Dentists not only solve problems, they can prevent them. When it comes to your oral health it is important you are proactive and seeing your dentist regularly so that they can catch any issues early, before they become a problem.
GENERATIVE AI
Regular dental checkups and cleanings are important for maintaining good oral health and preventing dental problems such as cavities, gum disease, and tooth loss. Additionally, early detection of dental issues through regular checkups can save you time, money, and discomfort in the long run.
TOPIC
REGULAR EXERCISE
HUMAN
Regular exercise has numerous benefits for both physical and mental health, including improved cardiovascular health, increased muscle strength and endurance, and reduced risk of chronic diseases such as obesity, diabetes, and some cancers. Exercise also releases endorphins, which can improve mood, reduce stress and anxiety, and enhance cognitive function.
AI
Regular physical activity is one of the most important things you can do for your health. Being physically active can improve your brain health, help manage weight, reduce the risk of disease, strengthen bones and muscles, and improve your ability to do everyday activities.
TOPIC
LEARNING A NEW LANGUAGE
HUMAN
Foreign language study enhances one's opportunities in government, business, medicine, law, technology, military, industry, marketing, and more. A second language improves your skills and grades in math and English and on the SAT and GRE.
AI
Learning a new language can expand your personal and professional opportunities by enabling you to communicate with people from different cultures and countries. Additionally, it can be a fun and rewarding experience that can boost your confidence, improve your memory and cognitive abilities, and help you better understand the world around you.
TOPIC
Effects of screen-time on child development
HUMAN
Research has shown that screen time inhibits young children’s ability to read faces and learn social skills, two key factors needed to develop empathy. Face-to-face interactions are the only way young children learn to understand non-verbal cues and interpret them.
AI
Excessive screen time in children has been linked to developmental delays, decreased attention span, and poor academic performance. Additionally, screen time has been associated with increased risk of behavioral problems, such as aggression and impulsivity, and may interfere with the development of social skills and emotional regulation.
TOPIC
Public transportation
HUMAN
Public transportation can convey many more people in much less space than individual automobiles, which helps to keep traffic congestion lower, which in turn reduces air pollution from idling vehicles, and helps riders avoid the stress that comes from daily driving in highly congested areas.
AI
Public transportation is a more sustainable and affordable mode of transportation that can help reduce traffic congestion and air pollution in urban areas. Additionally, public transportation can provide greater mobility to people who cannot afford or do not have access to private vehicles, enabling them to access education, healthcare, and job opportunities.
Which statement is most compelling?
After learning that their favored statement was written by generative AI, respondents were asked how that made them feel and if it changed their initial opinion on the matter. Most respondents were either unmoved or felt even more positive towards AI and its abilities.
Specifically, more than one in four (27%) said this made them feel more positive toward generative AI, while another 53% felt indifferent on the matter with no change in opinion. Less than one in ten (9%) of respondents expressed concerns or negative feelings toward generative AI’s ability to provide more compelling information than a human.
PRO-AI RESPONDENTS
PRO-AI RESPONDENTS
“I feel good about it because the information was high quality and interesting.”
“It improves my opinion on generative AI. Because it can further deepen the knowledge and information put out at high efficiency, I like it. ”
“Since both statements seemed quite accurate I would say that this shows me that generative AI can be good at what it does”
ANTI-AI RESPONSES
ANTI-AI RESPONSES
“I didn’t even notice. Makes me a little uncomfortable.”
“It felt human, so it fooled me. Now I believe there’s no way in which to know a policy or news source. I feel more skeptical.”
“That is disturbing. It has merely increased my concerns.”
IMPLICATIONS
Testing general information messages written by humans versus generative AI revealed that overwhelmingly respondents found the AI-generated messages more compelling. Most respondents were indifferent on the matter, while others welcomed the new sense of creativity. Only a small minority of respondents felt concerned or alarmed once they realized they found AI messages more compelling.
This general embrace or indifference towards receiving messages from AI opens the door for many individuals and companies to start, or continue, using AI-generated content to better target and communicate with their audience. It can help create efficiencies or streamline processes while also helping facilitate more creative content. But we must stay aware of its ability to manipulate and easily spread compelling fake information when placed in the wrong hands.
How we use it and whether we decide to regulate its use may help alleviate some concerns while maintaining the advantages that this technology can provide. The implementation of AI has the potential to reshape the landscape of the spread of information, presenting us with both exciting opportunities and ethical considerations for the days to come.
