Use of Markov Decision Process Models in Preventive Medicine
Loading...
Date
2018-08-02
Authors
NEMUTLU, GIZEM SULTAN
Journal Title
Journal ISSN
Volume Title
Publisher
University of Waterloo
Abstract
The biggest trade-off when proposing health care policies is about balancing the effectiveness and the practicality of the policies. The optimal policies providing benchmark performances can be driven through using operations research tools; however, they usually have complex structures that are necessary to sufficiently represent various aspects of the system being modeled. There are also policies either proposed in guidelines or followed in practice but they often vary with the system characteristics, i.e., preferences of the clinicians, available resources of the clinics, etc. Therefore, standardized, simple yet effective policies are needed for many healthcare applications, including preventive medicine. At this point, we study developing health care delivery policies that maximize the effect of the preventive interventions, while providing applicable policy structures that can be easily followed by health practitioners in practice. We focus on two applications of preventive medicine: childhood vaccine administration practices in developing countries; and colorectal cancer screening and surveillance.
Vaccine administration practices in developing countries suffer from open-vial wastage. Doses remaining from opened vials are disposed at the end of a day, due to lack of appropriate cold storage conditions. We propose administering vaccines from different sizes of multi-dose vials to address the open-vial wastage problem. We utilize a Markov decision process model to maximize the expected total number of doses administered via reducing vaccine wastage. The model dynamically decides which size of a multi-dose vial to open next, and when to terminate vaccination service for the day, given the time remaining in the replenishment cycle and available vaccine stocks. We show that the optimal policies are of control-limit type. Using data for routine pediatric vaccines, we show that the proposed optimal policies could cost-effectively reduce open-vial wastage and significantly improve the covered vaccine demand. We also analyze the initial vaccine inventory composition that specifies how many vials of each size should be kept in stock. We show that the optimal policy for the right vaccine inventory composition may improve the expected vaccine demand covered up to target levels without early termination of vaccination service while realizing reasonably small or no additional cost. Although the number of system variables being tracked in our state space is manageable, the optimal policies still require significant effort to be adopted in practice. That is especially challenging in developing countries, where the resources, e.g., clinic staff, are limited. Therefore, we introduce simple vaccine administration policies that are developed with the guidance of the insights from our numerical and structural analyses. Our insights on the simple vaccine administration policies show that these policies can provide promising performance, in terms of costs and expected vaccine demand covered, compared to the optimal policies while requiring only a single system variable, i.e., time of a decision, to be monitored.
Colonoscopy screening prevents, and early-detects colorectal cancer (CRC), one of the most common and deadliest cancers in the world. Considering that the risk of developing CRC significantly increases after age 50, and that the North American population is aging, the colonoscopy screening and follow-up policies employed by gastroenterologists play a vital role in the well-being of the population. Existing clinical guidelines recommend colonoscopy screening policies that are shown to be cost-effective in CRC prevention and early detection. Nevertheless, almost half the practitioners do not follow these guidelines, indicating controversy around the best CRC screening practices. Several studies analyze alternative CRC screening policies using simulation and mathematical models. Especially, dynamic alternative policies, derived by a stochastic dynamic programming approach, can significantly increase health outcome improvements due to CRC screening and follow-up. However, under dynamic policies, colonoscopy screening and surveillance intervals significantly vary in factors such as age, gender, and personal history, which are harder to implement for clinicians. Our study on this second application aims at deriving efficient and simpler-to-implement colonoscopy screening and follow-up policies, but that perform closely to the optimal policies. We employ a patient-level discrete-event simulation model, built and validated using real data, to mimic CRC progression in asymptomatic and higher-risk individuals. We estimate the expected life-years, age-based risk of having CRC, CRC mortality, costs associated with CRC screening, and the number of required colonoscopies for a large set of screening policies. We evaluate the performances of all relevant simpler-to-implement colonoscopy policies, including the periodic screening policies currently used by practitioners, and all feasible periodic policies with n-period switch times (for n=0,1,2). Our analysis identifies under the parameter settings under which alternative and simpler policies are sufficient to provide close-to-optimal performance. These results provide insights on the types of policies on which to focus in future studies, for researchers from both medical and operational research fields.
Description
Keywords
Markov Decision Process, Healthcare optimization, Vaccine wastage, Colonoscopy Screening, Pediatric vaccination, Stochastic Modeling