A few months ago, my doctor showed off an AI transcription tool he used to record and summarize his patient meetings. In my case, the summary was fine, but researchers cited by ABC News have found that’s not always the case with OpenAI’s Whisper, which powers a tool many hospitals use — sometimes it just makes things up entirely.

Whisper is used by a company called Nabla for a medical transcription tool that it estimates has transcribed 7 million medical conversations, according to ABC News. More than 30,000 clinicians and 40 health systems use it, the outlet writes. Nabla is reportedly aware that Whisper can hallucinate, and is “addressing the problem.”

A group of researchers from Cornell University, the University of Washington, and others found in a study that Whisper hallucinated in about 1 percent of transcriptions, making up entire sentences with sometimes violent sentiments or nonsensical phrases during silences in recordings. The researchers, who gathered audio samples from TalkBank’s AphasiaBank as part of the study, note silence is particularly common when someone with a language disorder called aphasia is speaking.

Ah ben oui, quelle bonne idée de confier la transcription de données médicales à un système qui invente des choses qui n'existent pas ! (et j'ose même pas demander si c'est fait localement ou sur leurs serveurs, parce que si ça tombe, il y a en plus le risque que les données se retrouvent dans la nature)