Humans make better quacks than computers

rubber-duck-robotWhile the tech world gets enthusiastic about computers working out what is wrong with your health, a new study suggests that it is probably better to let a human decide.

There are lots of apps or other symptom based checkers to help self-diagnose diseases. Over the last 20 computer-based checklists and other fail-safe digital apps have been increasingly used to reduce medication errors or streamline infection-prevention protocols.  Yet the first direct comparison shows human doctors outperform digital ones in diagnostic accuracy.

A new study led by researchers at Harvard Medical School show that physicians’ performance is vastly superior and that doctors make a correct diagnosis more than twice as often as 23 commonly used symptom-checker apps.

Diagnostic errors stem from failure to recognize a disease or to do so in a timely manner. Physicians make such errors roughly 10 to 15 percent of the time, researchers say.

In the study, 234 internal medicine physicians were asked to evaluate 45 clinical cases, involving both common and uncommon conditions with varying degrees of severity. For each scenario, physicians had to identify the most likely diagnosis along with two additional possible diagnoses. Each clinical vignette was solved by at least 20 physicians.

The physicians outperformed the symptom-checker apps, listing the correct diagnosis first 72 percent of the time, compared with 34 percent of the time for the digital platforms. Eighty-four percent of clinicians listed the correct diagnosis in the top three possibilities, compared with 51 percent for the digital symptom-checkers.

The sicker you are the more likely the quack will be to spot it over a computer. Ateev Mehrotra, an associate professor of health care policy at HMS said that while the computer programs were clearly inferior to physicians in terms of diagnostic accuracy, it will be critical to study future generations of computer programs that may be more accurate.

Physicians still made mistakes in about 15 percent of cases. Researchers say developing computer-based algorithms to be used in conjunction with human decision-making may help further reduce diagnostic errors.