Review article · 2015 · Open access

Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria

Cara C. Lewis, Sarah Fischer, Bryan Jeffrey Weiner, Cameo Stanick, Mimi Kim, Ruben G. Martinez

Implementation Science, 10(1), 155 · Published 4 November 2015 · DOI 10.1186/s13012-015-0342-x

Free PDFView at the publisherCite this paper

In brief

“Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria” is a 2015 review article by Cara C. Lewis and 5 co-authors, published in Implementation Science. OpenAlex counts 447 citations as of 15 November 2026. It is open access under a CC BY licence: the full text is free to read at Implementation Science.

Citations
447
Most cited in
2020 (100)
Field-weighted impact
2.17
References
26
Authors
6

Abstract

BACKGROUND: High-quality measurement is critical to advancing knowledge in any field. New fields, such as implementation science, are often beset with measurement gaps and poor quality instruments, a weakness that can be more easily addressed in light of systematic review findings. Although several reviews of quantitative instruments used in implementation science have been published, no studies have focused on instruments that measure implementation outcomes. Proctor and colleagues established a core set of implementation outcomes including: acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, sustainability (Adm Policy Ment Health Ment Health Serv Res 36:24-34, 2009). The Society for Implementation Research Collaboration (SIRC) Instrument Review Project employed an enhanced systematic review methodology (Implement Sci 2: 2015) to identify quantitative instruments of implementation outcomes relevant to mental or behavioral health settings. METHODS: Full details of the enhanced systematic review methodology are available (Implement Sci 2: 2015). To increase the feasibility of the review, and consistent with the scope of SIRC, only instruments that were applicable to mental or behavioral health were included. The review, synthesis, and evaluation included the following: (1) a search protocol for the literature review of constructs; (2) the literature review of instruments using Web of Science and PsycINFO; and (3) data extraction and instrument quality ratings to inform knowledge synthesis. Our evidence-based assessment rating criteria quantified fundamental psychometric properties as well as a crude measure of usability. Two independent raters applied the evidence-based assessment rating criteria to each instrument to generate a quality profile. RESULTS: We identified 104 instruments across eight constructs, with nearly half (n = 50) assessing acceptability and 19 identified for adoption, with all other implementation outcomes revealing fewer than 10 instruments. Only one instrument demonstrated at least minimal evidence for psychometric strength on all six of the evidence-based assessment criteria. The majority of instruments had no information regarding responsiveness or predictive validity. CONCLUSIONS: Implementation outcomes instrumentation is underdeveloped with respect to both the sheer number of available instruments and the psychometric quality of existing instruments. Until psychometric strength is established, the field will struggle to identify which implementation strategies work best, for which organizations, and under what conditions.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Lewis, C. C., Fischer, S., Weiner, B. J., Stanick, C., Kim, M., & Martinez, R. G. (2015). Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science, 10(1), 155. https://doi.org/10.1186/s13012-015-0342-x

MLA 9

Lewis, Cara C., et al. “Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria.” Implementation Science, vol. 10, no. 1, 2015, p. 155. https://doi.org/10.1186/s13012-015-0342-x.

Chicago

Lewis, Cara C., Sarah Fischer, Bryan Jeffrey Weiner, Cameo Stanick, Mimi Kim, and Ruben G. Martinez. 2015. “Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria.” Implementation Science 10 (1): 155. https://doi.org/10.1186/s13012-015-0342-x.

Harvard

Lewis, C. C. et al. (2015) ‘Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria’, Implementation Science, 10(1), p. 155. doi:10.1186/s13012-015-0342-x.

IEEE

C. C. Lewis, S. Fischer, B. J. Weiner, C. Stanick, M. Kim, and R. G. Martinez, “Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria,” Implementation Science, vol. 10, no. 1, p. 155, 2015, doi: 10.1186/s13012-015-0342-x.

BibTeX

@article{lewis2015outcomes,
  title     = {{Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria}},
  author    = {Lewis, Cara C. and Fischer, Sarah and Weiner, Bryan Jeffrey and Stanick, Cameo and Kim, Mimi and Martinez, Ruben G.},
  journal   = {Implementation Science},
  year      = {2015},
  volume    = {10},
  number    = {1},
  pages     = {155},
  doi       = {10.1186/s13012-015-0342-x},
  url       = {https://doi.org/10.1186/s13012-015-0342-x}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
20171515
20182727
20193939
2020100100
20215252
20225858
20235656
20243333
20252626
20263535

Its field-weighted citation impact is 2.17: it has been cited 2.17 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science

  • implementation outcomes
  • psychometric properties
  • mental health services
  • behavioral health services
  • acceptability
  • adoption
  • fidelity
  • feasibility

Work with this paper

Most-cited papers in Health Professions

All highly cited Health Professions papers

Questions about this paper

How do I cite this paper in APA style?

Lewis, C. C., Fischer, S., Weiner, B. J., Stanick, C., Kim, M., & Martinez, R. G. (2015). Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science, 10(1), 155. https://doi.org/10.1186/s13012-015-0342-x

How many times has this paper been cited?

447 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at Implementation Science.

Who wrote this paper?

Cara C. Lewis, Sarah Fischer, Bryan Jeffrey Weiner, Cameo Stanick, Mimi Kim, Ruben G. Martinez.

What is the DOI of this paper?

10.1186/s13012-015-0342-x. The permanent link is https://doi.org/10.1186/s13012-015-0342-x.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.