Decision and Simulation Modeling in Systematic Reviews ― Methods Research Report

The overarching goal of our report is to provide guidance to determine when incorporating a decision-analytic model alongside a systemic review would be of added value for decisionmaking purposes. The purpose of systematic reviews is to synthesize the current scientific literature on a particular topic in the form of evidence reports and technology assessments to assist public and private organizations in developing strategies that improve the quality of health care and decisionmaking. However, there is often not enough evidence to fully address the questions that are relevant for decisionmakers. Or, there may be enough evidence on several components to the decision (e.g., diagnostic test characteristics, test risks, risk and effectiveness of treating disease), but no studies that evaluate the relevant clinical strategies that incorporate all of these components. In this example, the most relevant question for decisionmaking purposes is to address the comparison of alternative test-and-treat strategies, which synthesizes all of these data elements. Our framework is informed by two sets of interviews and a focus group discussion; literature reviews to summarize best modeling practices and to profile the modeling literature; and an exploration of the feasibility of the developing of a database of published models. We present our findings in six chapters. Decision and Simulation Modeling Alongside Systematic Reviews provides an overview of models, and describes the differences and synergies between systematic reviews and decision analysis. In Overview of Decision Models Used in Research, we provide a “scan” of the medical literature over the past five years in terms of the use of models in studies that compare intervention strategies using multiple sources of data. Use of Modeling in Systematic Reviews: The EPC Perspective documents the extent to which Evidence-based Practice Centers (EPCs) have incorporated models to data, and presents results from key informant interviews with EPC members. We present a framework for deciding when a decision model can inform decisionmaking alongside a systematic review in Suggested Framework for Deciding When a Modeling Effort Should be Added to a Systematic Review. Potential Modeling Resources explores several possible approaches to use when undertaking a modeling effort and discusses some of the challenges. Lastly, Best Practices for Decision and Simulation Modeling reviews the literature on best practices for modeling, supplemented by a focus group discussion with modeling experts, and lessons learned about the process of conducting a modeling exercise alongside a systematic review using recent experience with the U.S. Preventive Services Task Force (USPSTF).