A few years back, when Reddit dubbed itself “the heart of the internet,” the phrase aimed to highlight the platform’s spontaneous nature. Amidst a digital landscape where social media is largely governed by complex algorithms, Reddit prided itself on its curation by a user base that conveyed their preferences through upvotes and downvotes—essentially, molded by real individuals rather than automated systems.
Earlier this week, when participants from a well-known subreddit discovered that their community was invaded by covert researchers who posted artificial intelligence-generated remarks masquerading as genuine user contributions, the Redditors reacted with understandable outrage. They described the study as “invasive,” “disgraceful,” “enraging,” and “highly unsettling.” As criticism mounted, the researchers became unresponsive, declining to disclose their identities or address inquiries regarding their methods. The educational institution they work for stated it would conduct an investigation into the matter. In the meantime, Reddit’s top lawyer, Ben Lee,
wrote
that the firm plans to “make sure researchers face consequences for their wrongdoings.”
Echoing this sentiment of discontent were other experts in internet research, who sharply criticized what they perceived as an overtly unethical study. Amy Bruckman, a professor at the Georgia Institute of Technology with over twenty years of experience researching online societies, expressed her view to me by stating that the Reddit debacle represents “without question the most severe breach of internet research ethics I’ve encountered.” Furthermore, both she and others express concern that this backlash might jeopardize the efforts of academics employing traditional methodologies to explore a vital issue: the impact of artificial intelligence on human cognition and interpersonal interactions.
The researchers from the University of Zurich aimed to determine if AI-generated replies had the power to alter people’s opinions. Therefore, they went to the fittingly titled subreddit.
r/changemyview
In these forums where users discuss significant social matters alongside numerous minor subjects, participants earn points for arguments that persuade others to change their stance. Throughout a period of four months, the research team contributed over 1,000 artificial intelligence-created remarks about various contentious topics such as whether pit bulls are inherently aggressive due to breeding or upbringing, how addressing the current housing shortage might involve staying at home with one’s parents, and predictions regarding diversity, equity, and inclusion initiatives—whether they were bound to face failure from inception.
The synthetic commentators weighed in onReddit being an unproductive use of time and suggested that theories involving “controlled demolition” related to the events of September 11th have certain credibility. Additionally, while expressing fabricated viewpoints, these digital contributors provided personal narratives: one proclaimed themselves a therapist specializing in trauma care, whereas another self-identified as someone who had been subjected to statutory sexual assault.
To some extent, the AI-generated remarks seem quite impactful. Researchers found that when they instructed the AI to tailor its responses based on specific personal data about a Redditor—such as gender, age, and inferred political views from their posting history—a significant number of individuals appeared to change their opinions. The customized AI replies generally garnered much higher ratings within the subreddit’s scoring mechanism compared to most contributions from humans, as indicated by initial research outcomes shared privately with both Reddit moderators and the participants involved. This assessment naturally presumes that none of the subreddit members were utilizing similar AI tools for refining their inputs.
[
Read: The fellow aiming to demonstrate just how unintelligent AI remains
]
The scientists faced challenges in gaining Reddit users’ approval for their clandestine research project. Once the study concluded, they reached out to the subreddit administrators, disclosed their identities, and sought permission to “debrief” the community—meaning they wanted to inform participants that over several months, they unknowingly took part in a scientific investigation. One administrator, known as LucidLeviathan to safeguard personal information, stated: “They seemed quite taken aback by our unfavorable response towards the experiment.” According to LucidLeviathan, the admins demanded that the researchers refrain from publishing this compromised data and also apologize. However, the researchers declined these requests. Following weeks of negotiation, the administrators shared with the broader subreddit audience all they knew regarding the nature of the experiment (without revealing the researchers’ names), clearly expressing their dissatisfaction.
When the moderators forwarded a complaint about the project at the University of Zurich, they shared part of the university’s response which stated: “The initiative provides significant findings; however, the potential hazards such as emotional distress are negligible.” A representative from the university told me in an interview that the ethics committee had been informed regarding this research earlier last month. The representatives were instructed to follow all guidelines set forth by the subreddit and mentioned plans to implement more rigorous oversight procedures moving forward. Additionally, one of the researchers justified their methodology via a post on Reddit asserting that none of the remarks promoted detrimental viewpoints. They also confirmed every computer-created message went through scrutiny conducted by members of their group prior to publication online. I reached out directly to contact details provided anonymously by the Redditors but got redirected back towards seeking answers from institutional authorities instead.
One of the most revealing aspects of the Zurich researchers’ defense is that they considered deception essential for the study. Although the ethics committee at the University of Zurich could provide guidance to researchers, it reportedly does not have the authority to veto projects failing to meet ethical standards. According to communications from the university, this committee advised the researchers prior to starting their postings that “participants must be informed as thoroughly as feasible.” However, the research team felt that full disclosure would compromise the integrity of the experiment. They argued that conducting such tests ethically required keeping subjects uninformed since this scenario better reflects genuine interactions with unknown malicious entities in everyday situations. This rationale appeared in one of their remarks posted on Reddit.
The way people might react in such circumstances is a pressing concern and deserves scholarly attention. According to initial findings, the investigators determined that artificial intelligence can be “exceptionally convincing in practical settings, outperforming all previous standards set by human persuasion.” (Since the team decided earlier this week against releasing a report on the study, the precision of these conclusions may remain unknown forever, which is indeed unfortunate.) The idea of altering someone’s perspective through an entity devoid of consciousness is profoundly disturbing. Furthermore, this potent ability could potentially be misused for harmful purposes.
[
Chatbots are bypassing their performance evaluations.
]
Nevertheless, scientists do not need to disregard the standards for conducting experiments on human participants to assess this risk. “The overall conclusion that artificial intelligence could be highly persuasive, more so than many people, aligns with what lab tests have shown,” said Christian Tarsney, a senior research fellow at the University of Texas at Austin. In an experiment,
recent laboratory experiment
, volunteers who held conspiratorial views engaged in conversations with artificial intelligence on a voluntary basis; following just three interactions, roughly one-fourth of these individuals abandoned their former convictions. Additionally, research indicated that ChatGPT was responsible for producing
more persuasive disinformation
then humans, and participants asked to differentiate between authentic posts and those created by AI were unable to do so effectively.
Giovanni Spitale, who led the study, is affiliated with the University of Zurich and communicated with one of the scientists involved in the Reddit AI project. This scientist requested that his identity remain confidential. In a communication he relayed to me, Spitale mentioned a message from this researcher stating, “We have received numerous death threats,” and urged maintaining secrecy for the well-being of his loved ones.
A major factor behind the intense backlash could be thatReddit operatesas aclose-knitcommunity where betrayal can havea profound impact. As Spitale mentioned to me, “Mutual trust forms one ofthe cornerstonesof these communities.” This partly explains why he is against conducting experimentsonRedditorswithouttheirconsent.Knownethicallyto some expertsIconsulted,this situationwas unfavorably likenedtoFacebook’s practices.
infamous emotional-contagion study
In July 2012, Facebook modified users’ News Feeds to investigate whether exposure to varying levels of positivity influenced their posting behavior. This tweak had some impact. Casey Fiesler, an associate professor from the University of Colorado at Boulder specializing in ethics and digital societies, explained to me that compared to Facebook’s experiment, the work conducted by scientists in Zurich was far more extensive. “Users reacted strongly to Facebook’s test, yet differently than how members of this Reddit group are reacting,” she noted. “The current situation seems much more intimate.”
[
AI leaders pledge cancer cures. Here’s what’s actually happening.
]
This reaction likely stems from the unsettling idea that ChatGPT understands how to trigger specific responses within our minds. Being misled by unethical human Facebook researchers feels different compared to being deceived by an impersonating chatbot. After reading numerous AI-generated remarks, I noticed that even though not every comment was exceptional, many appeared sensible and authentic. These comments raised several valid arguments, causing me to agree multiple times. The researchers from Zurich caution that without improved detection methods, these AI bots could “effortlessly integrate themselves” into internet forums—assuming this hasn’t happened already.