December 26, 2023 – Recent research conducted by experts has revealed that the recommendations provided by ChatGPT are, in many aspects, more balanced, comprehensive, empathetic, and helpful compared to responses from human experts.
A research team from the University of Melbourne and the University of Western Australia selected 50 social dilemma questions from the top ten advice column websites for their study. They then invited 404 volunteers to participate in a blind test where they were presented with answers provided by both column writers and ChatGPT. The volunteers were asked to evaluate which answers they found to be more balanced, comprehensive, empathetic, and helpful.
The researchers found that in all five randomly presented questions and across all query categories, ChatGPT consistently outperformed human advisors, with preference rates for artificial intelligence ranging from 70% to 85%.
Both surveys indicated that many individuals perceived ChatGPT’s recommendations as more balanced, complete, empathetic, and helpful compared to those from professional consultants, ultimately favoring ChatGPT. In both Survey 1, which had 404 participants, and Survey 2, which had 401 participants, the majority of respondents voted in favor of ChatGPT across all aspects.
The study also revealed that responses from ChatGPT were longer than those from advice column writers. In the second part of the study, researchers shortened ChatGPT’s responses to align them more closely with the length of responses from column writers. The second study showed that while preference rates decreased slightly, ChatGPT’s responses remained more comprehensive and of high quality overall.
Despite the high quality of ChatGPT’s recommendations, 77% of respondents still expressed a preference for answers provided by human experts.
