Journal article · 2018 · Open access

Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them

Jennifer N. Hill, Sara M. Locatelli, Barbara G. Bokhour, Gemmae M. Fix, Jeffrey Solomon, Nora Mueller, Sherri L. LaVela

BMC Research Notes, 11(1), 560 · Published 4 August 2018 · DOI 10.1186/s13104-018-3650-9

Free PDFView at the publisherCite this paper

In brief

“Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them” is a 2018 journal article by Jennifer N. Hill and 6 co-authors, published in BMC Research Notes. OpenAlex counts 31 citations as of 15 November 2026. It is open access under a CC BY licence: the full text is free to read at BMC Research Notes.

Citations
31
Most cited in
2022 (6)
Field-weighted impact
3.76
References
25
Authors
7

Abstract

OBJECTIVE: The objective of this paper is to demonstrate the utility of the CFIR framework for evaluating broad-scale change by discussing the challenges to be addressed when planning the assessment of broad-scale change and the solutions developed by the evaluation team to address those challenges. The evaluation of implementation of Patient-centered Care and Cultural Transformation (PCC&CT) within the Department of Veterans Affairs (VA) will be used as a demonstrative example. Patient-Centered Care (PCC) is personalized health care that considers a patient's circumstances and goals. The Department of Veterans Affairs (VA) is working towards implementing PCC throughout its healthcare system, comprised of multiple interventions with a singular long-term goal of cultural transformation, however little is known about the factors influencing its implementation. This paper discusses the issues that arose using CFIR to qualitatively assess the factors influencing implementation of cultural transformation. RESULTS: Application of CFIR to this broad-scale evaluation revealed three strategies recommended for use in evaluating implementation of broad-scale change: (1) the need for adapted definitions for CFIR constructs (especially due to new application to broad-scale change), (2) the use of a mixed deductive-inductive approach with thematic coding to capture emergent themes not encompassed by CFIR, and (3) its use for expedited analysis and synthesis for rapid delivery of findings to operational partners. This paper is among the first to describe use of CFIR to guide the evaluation of a broad-scale transformation, as opposed to discrete interventions. The processes and strategies described in this paper provide a detailed example and structured approach that can be utilized and expanded upon by others evaluating implementation of broad-scale evaluations. Although CFIR was the framework selected for this evaluation, the strategies described in this paper including: use of adapted definitions, use of mixed deductive-inductive approach, and the approach for expedited analysis and synthesis can be transferred and tested with other frameworks.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Hill, J. N., Locatelli, S. M., Bokhour, B. G., Fix, G. M., Solomon, J., Mueller, N., & LaVela, S. L. (2018). Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them. BMC Research Notes, 11(1), 560. https://doi.org/10.1186/s13104-018-3650-9

MLA 9

Hill, Jennifer N., et al. “Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them.” BMC Research Notes, vol. 11, no. 1, 2018, p. 560. https://doi.org/10.1186/s13104-018-3650-9.

Chicago

Hill, Jennifer N., Sara M. Locatelli, Barbara G. Bokhour, Gemmae M. Fix, Jeffrey Solomon, Nora Mueller, and Sherri L. LaVela. 2018. “Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them.” BMC Research Notes 11 (1): 560. https://doi.org/10.1186/s13104-018-3650-9.

Harvard

Hill, J. N. et al. (2018) ‘Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them’, BMC Research Notes, 11(1), p. 560. doi:10.1186/s13104-018-3650-9.

IEEE

J. N. Hill et al., “Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them,” BMC Research Notes, vol. 11, no. 1, p. 560, 2018, doi: 10.1186/s13104-018-3650-9.

BibTeX

@article{hill2018evaluating,
  title     = {{Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them}},
  author    = {Hill, Jennifer N. and Locatelli, Sara M. and Bokhour, Barbara G. and Fix, Gemmae M. and Solomon, Jeffrey and Mueller, Nora and LaVela, Sherri L.},
  journal   = {BMC Research Notes},
  year      = {2018},
  volume    = {11},
  number    = {1},
  pages     = {560},
  doi       = {10.1186/s13104-018-3650-9},
  url       = {https://doi.org/10.1186/s13104-018-3650-9}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
201811
201955
202055
202122
202266
202322
202444
202566

Its field-weighted citation impact is 3.76: it has been cited 3.76 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science

  • Consolidated Framework for Implementation Research
  • implementation evaluation
  • patient-centered care
  • cultural change
  • Department of Veterans Affairs
  • organizational change
  • implementation factors

Work with this paper

Most-cited papers in Health Professions

All highly cited Health Professions papers

Questions about this paper

How do I cite this paper in APA style?

Hill, J. N., Locatelli, S. M., Bokhour, B. G., Fix, G. M., Solomon, J., Mueller, N., & LaVela, S. L. (2018). Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them. BMC Research Notes, 11(1), 560. https://doi.org/10.1186/s13104-018-3650-9

How many times has this paper been cited?

31 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at BMC Research Notes.

Who wrote this paper?

Jennifer N. Hill, Sara M. Locatelli, Barbara G. Bokhour, Gemmae M. Fix, Jeffrey Solomon, Nora Mueller, Sherri L. LaVela.

What is the DOI of this paper?

10.1186/s13104-018-3650-9. The permanent link is https://doi.org/10.1186/s13104-018-3650-9.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.