Enterprise

Researchers at MIT and elsewhere found that AI assistance improved the accuracy of non-experts and clinicians in diagnosing skin diseases, but the impact of explainable AI methods varied depending on the users' knowledge level. Non-experts trusted LLM-based explanations, even when incorrect, while clinicians performed best with only a model's prediction and no explanation. The study highlights the importance of building AI systems with users in mind and developing explainability methods that encourage critical thinking. This has significant implications for engineers building AI systems, as they must consider the potential for algorithmic deference and automation bias in human users.