The mysterious case of the missing reference

This week my father asked me to get ChatGPT to write a two-page report on the efficacy of green leucaena seeds for controlling gastro-intestinal parasites (worms) in children. He could have done it himself but you how oldies are with new technology so I duly obliged. ChatGPT spat out a report that was very good but it made one claim that seems to be unsubstantiated or at least referenced by a non-existent paper.

I asked ChatGPT for more information:

Can consuming green leucaena seeds in large quantities have toxic side effects? Provide a reference.

It gave me this reference to a paper that doesn’t exist:

Gupta, A., & Chandra, S. (2017). Mimosine: A toxic non-protein amino acid from Leucaena leucocephala and its detoxification. Plant Foods for Human Nutrition, 72(2), 93-102. doi:10.1007/s11130-016-0613-4

I asked the same question a second time and got the same paper, same authors, but different journal. It also doesn’t exist.

Reference: Gupta, A., & Chandra, S. (2017). Mimosine: A toxic non-protein amino acid from Leucaena leucocephala and its detoxification. Journal of Applied Biology & Biotechnology, 7(3), 1-6

I have searched, my father has searched, Ben has searched – none of us can find this paper. Did ChatGPT just make it up? Ben seems to think so. If it’s making up references then how can we trust anything it says and does that suggest it’s thinking for itself? Could it be that someone has published a paper and made up this as a reference on their paper?

*Update* Apparently ChatGPT makes up references! https://news.ycombinator.com/item?id=33841672