This week my father asked me to get ChatGPT to write a two-page report on the efficacy of green leucaena seeds for controlling gastro-intestinal parasites (worms) in children. He could have done it himself but you how oldies are with new technology so I duly obliged. ChatGPT spat out a report that was very good but it made one claim that seems to be unsubstantiated or at least referenced by a non-existent paper.
I asked ChatGPT for more information:
Can consuming green leucaena seeds in large quantities have toxic side effects? Provide a reference.
It gave me this reference to a paper that doesn’t exist:
Gupta, A., & Chandra, S. (2017). Mimosine: A toxic non-protein amino acid from Leucaena leucocephala and its detoxification. Plant Foods for Human Nutrition, 72(2), 93-102. doi:10.1007/s11130-016-0613-4
I asked the same question a second time and got the same paper, same authors, but different journal. It also doesn’t exist.
Reference: Gupta, A., & Chandra, S. (2017). Mimosine: A toxic non-protein amino acid from Leucaena leucocephala and its detoxification. Journal of Applied Biology & Biotechnology, 7(3), 1-6
I have searched, my father has searched, Ben has searched – none of us can find this paper. Did ChatGPT just make it up? Ben seems to think so. If it’s making up references then how can we trust anything it says and does that suggest it’s thinking for itself? Could it be that someone has published a paper and made up this as a reference on their paper?
*Update* Apparently ChatGPT makes up references! https://news.ycombinator.com/item?id=33841672
rachel.blog 
Wow – that is very interesting!
At least ChatGPT uses the style of an academic paper.
Yes it knows exactly how to make a reference look real that’s for sure.
and happy imbolc.
You mind if I reblog this later this month?
Sure, go for it.
Thank you.