Okey but was the paper obviously a joke? How a human would determine it was fake

Another study said that in recent years 25% of studies were being faked up for academic credentials, either by paper mills or academics themselves. Thousands of papers even used the same diagrams, with text in slight variations, citing each other.

Add AI to the mix, things get worse.

Wow, so poisoning the training data works that well?

This is awesome - guys, let’s all upload papers that say the lining of MAGA hats contains a chemical that causes cancer, that NOT getting vaccines causes autism, and that the most conservative universities show Trump lost the 2020 election. Let Grok gobble it up!

The condition doesn’t appear in the standard medical literature — because it doesn’t exist. It’s the invention of a team led by Almira Osmanovic Thunström, a medical researcher at the University of Gothenburg, Sweden, who dreamt up the skin condition and then uploaded two fake studies about it to a preprint server in early 2024. Osmanovic Thunström carried out this unusual experiment to test whether large language models (LLMs) would swallow the misinformation and then spit it out as reputable health advice. “I wanted to see if I can create a medical condition that did not exist in the database,” she says.

The problem was that the experiment worked too well. Within weeks of her uploading information about the condition, attributed to a fictional author, major artificial-intelligence systems began repeating the invented condition as if it were real.

There are going to be people who believe this disease is real for the rest of their lives.