CAN LARGE MULTIMODAL MODELS INTERPRET HEALTH CARE INFOGRAPHICS IN DEMENTIA CAREGIVING MATERIALS?
2024
Can Large AI Models Understand Health Infographics for Dementia Care?
Sample size: 5
publication
Evidence: moderate
Author Information
Author(s): Luo Zhimeng, Zou Ning, Xie Bo, He Daqing, Hilsabeck Robin, Aguirre Alyssa
Primary Institution: University of Pittsburgh
Hypothesis
Can large multimodal models interpret health care infographics effectively?
Conclusion
GPT-4V significantly outperformed other models in interpreting the completeness of health infographics for dementia caregivers.
Supporting Evidence
- GPT-4V achieved an average accuracy of 99.0% across the infographics.
- The average completeness for GPT-4V was 93.7%, significantly higher than other models.
- Cohen’s kappa for coder agreement was 0.8205, indicating high reliability.
Takeaway
We tested some smart AI models to see if they could understand pictures and information meant for people taking care of dementia patients, and one model did the best job.
Methodology
Four large multimodal models were tested on 5 infographics, with their interpretations analyzed for accuracy and completeness by independent coders.
Statistical Information
P-Value
0.04
Statistical Significance
p=0.04
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website