Review article · 2025 · Open access

Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations

Liying Wang, Tanmay Bhanushali, Zhuoran Huang, Jingyi Yang, Sukriti Badami, Lisa Hightow‐Weidman

JMIR Mental Health, 12, e70014 · Published 15 May 2025 · DOI 10.2196/70014

Open PDFView at the publisherCite this paper

In brief

“Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations” is a 2025 review article by Liying Wang and 5 co-authors, published in JMIR Mental Health. OpenAlex counts 132 citations as of 22 November 2026, which puts it in the top 1% of works published in 2025 by citations. It is open access under a CC BY licence: the full text is free to read at JMIR Mental Health.

Citations
132
Most cited in
2026 (115)
Field-weighted impact
4.85
References
43
Authors
6

Abstract

Background: The global shortage of mental health professionals, exacerbated by increasing mental health needs post COVID-19, has stimulated growing interest in leveraging large language models to address these challenges. objectives: This systematic review aims to evaluate the current capabilities of generative artificial intelligence (GenAI) models in the context of mental health applications. Methods: A comprehensive search across 5 databases yielded 1046 references, of which 8 studies met the inclusion criteria. The included studies were original research with experimental designs (eg, Turing tests, sociocognitive tasks, trials, or qualitative methods); a focus on GenAI models; and explicit measurement of sociocognitive abilities (eg, empathy and emotional awareness), mental health outcomes, and user experience (eg, perceived trust and empathy). Results: The studies, published between 2023 and 2024, primarily evaluated models such as ChatGPT-3.5 and 4.0, Bard, and Claude in tasks such as psychoeducation, diagnosis, emotional awareness, and clinical interventions. Most studies used zero-shot prompting and human evaluators to assess the AI responses, using standardized rating scales or qualitative analysis. However, these methods were often insufficient to fully capture the complexity of GenAI capabilities. The reliance on single-shot prompting techniques, limited comparisons, and task-based assessments isolated from a context may oversimplify GenAI's abilities and overlook the nuances of human-artificial intelligence interaction, especially in clinical applications that require contextual reasoning and cultural sensitivity. The findings suggest that while GenAI models demonstrate strengths in psychoeducation and emotional awareness, their diagnostic accuracy, cultural competence, and ability to engage users emotionally remain limited. Users frequently reported concerns about trustworthiness, accuracy, and the lack of emotional engagement. Conclusions: Future research could use more sophisticated evaluation methods, such as few-shot and chain-of-thought prompting to fully uncover GenAI's potential. Longitudinal studies and broader comparisons with human benchmarks are needed to explore the effects of GenAI-integrated mental health care.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Wang, L., Bhanushali, T., Huang, Z., Yang, J., Badami, S., & Hightow‐Weidman, L. (2025). Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations. JMIR Mental Health, 12, e70014. https://doi.org/10.2196/70014

MLA 9

Wang, Liying, et al. “Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations.” JMIR Mental Health, vol. 12, 2025, p. e70014. https://doi.org/10.2196/70014.

Chicago

Wang, Liying, Tanmay Bhanushali, Zhuoran Huang, Jingyi Yang, Sukriti Badami, and Lisa Hightow‐Weidman. 2025. “Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations.” JMIR Mental Health 12: e70014. https://doi.org/10.2196/70014.

Harvard

Wang, L. et al. (2025) ‘Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations’, JMIR Mental Health, 12, p. e70014. doi:10.2196/70014.

IEEE

L. Wang, T. Bhanushali, Z. Huang, J. Yang, S. Badami, and L. Hightow‐Weidman, “Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations,” JMIR Mental Health, vol. 12, p. e70014, 2025, doi: 10.2196/70014.

BibTeX

@article{wang2025evaluating,
  title     = {{Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations}},
  author    = {Wang, Liying and Bhanushali, Tanmay and Huang, Zhuoran and Yang, Jingyi and Badami, Sukriti and Hightow‐Weidman, Lisa},
  journal   = {JMIR Mental Health},
  year      = {2025},
  volume    = {12},
  pages     = {e70014},
  doi       = {10.2196/70014},
  url       = {https://doi.org/10.2196/70014}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education

  • generative AI
  • large language models
  • mental health
  • AI evaluation
  • psychoeducation
  • emotional awareness
  • AI-assisted diagnosis
  • cultural competence

Work with this paper

Most-cited papers in Medicine

All highly cited Medicine papers

Questions about this paper

How do I cite this paper in APA style?

Wang, L., Bhanushali, T., Huang, Z., Yang, J., Badami, S., & Hightow‐Weidman, L. (2025). Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations. JMIR Mental Health, 12, e70014. https://doi.org/10.2196/70014

How many times has this paper been cited?

132 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at JMIR Mental Health: sign in and open the PDF from this page.

Who wrote this paper?

Liying Wang, Tanmay Bhanushali, Zhuoran Huang, Jingyi Yang, Sukriti Badami, Lisa Hightow‐Weidman.

What is the DOI of this paper?

10.2196/70014, permanently at https://doi.org/10.2196/70014. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.