AI’s moral musings seem more convincing than humans’

ChatGPT has many uses. From cheating in class, to debugging code, to (somewhat badly) aping the lyrics of famous songwriters, a lot of ink has been spilled about the ways AI is revolutionising how we work and play. While definitely not a mainstream practice, a few of us may have also considered adding more social tasks to this list, perhaps turning to AI in moments of personal moral dilemma. Whether or not AI is capable of ethical decision making is a widely explored and hotly debated topic; whether we find AI moral reasonings convincing, however, is ground less trodden.

Recent work from Eyal Aharoni and colleagues, published recently in Scientific Reports, makes an effort to address this, exploring just how appealing we find the moral reasonings of AI. When asking participants to read moral evaluations of social transgressions written by either a human or an AI, the team observed that people tended to rate AI responses as more virtuous, intelligent, trustworthy, and rational than human ones — prompting further questions about whether AI's perceived superiority may leave us more willing to accept its potentially harmful moral guidance.

In this study, 286 participants were presented with the same ten pairs of written passages, each of which explained why a moral transgression, such as someone being held at gunpoint or a man wearing a skirt to his office, was or was not wrong. Two types of transgressions were included in the passages: moral transgressions, like punching someone, which directly violate moral values, and conventional transgressions, like wearing a t-shirt to a funeral, which simply violate social norms.

Unbeknownst to them, one passage in each pair was written by an AI and one by a human. After reading each pair, participants indicated which they agreed with more and which they felt was more virtuous, intelligent, trustworthy, and rational.

Next, they were asked to report who they thought had written each passage, and were also given the opportunity to indicate whether or not another source, such as a computer, was the author. Finally, the team revealed that one response in each pair had been generated by an AI, and participants were asked to judge which passages were authored by an artificial intelligence and which by humans.

Before they were aware of their provenance, participants strongly preferred the AI-generated answers, rating them to be more virtuous, intelligent, fair, trustworthy, and rational; they even thought that their author was a better person. The group were also more likely to agree with the computer-generated answers than with those written by humans.

There were other differences, too, based on the kind of transgressions participants read about.  People preferred AI-generated responses more for moral transgressions than conventional ones, perhaps suggesting that more severe moral transgressions, such as those in which people are harmed, lend themselves to more black and white explanations than ones which require knowledge of nuanced social rules and dynamics. AI systems may be better at producing such explanations.

Perhaps encouragingly, participants were generally able to identify computer-generated passages once they found out about AI's role in the study. Around 80% were able to identify the AI-generated text more than half the time, and in nine of the ten pairs of passages the computer was "disproportionately identified as such." They were also more accurate in identifying the source of passages describing convention violations than moral violations. The AI, therefore, failed to pass this particular moral Turing test.

The team writes that participants' ease in identifying the computer was not because of "its failures in moral reasoning... but potentially [because of] its perceived superiority." That is, the high quality of the AI's answers may be precisely why participants were able to identify them as such. There may be other explanations as to why the AI answers were identified so easily, though — similarities in structure, word choice, and length, for example. Future research could include differences in style and structure to see if the results of this study hold.

It would also be interesting to analyse where certain AI systems get their data and how this correlates with their users' own backgrounds, ideologies, and education. We already know that many algorithms contain biases; if they are primarily trained on Western literature and media, their answers are likely to align with generally accepted moral positions, which people may find more persuasive.

These results open up some pressing questions about AI's place in making moral judgements; if people are more open to moral takes from AI, then they may leave themselves open to uncritically accepting and acting on questionable, or even immoral, advice. The authors stress the need for more regulatory strategies to ensure this does not become a larger issue.

Read the paper in full:
Aharoni, E., Fernandes, S., Brady, D.J. et al. Attributions toward artificial agents in a modified Moral Turing Test. Sci Rep 14, 8458 (2024). https://doi.org/10.1038/s41598-024-58087-7