MCH Student:
Dominic Dharam
Date of Defense:
April 1, 2016
Abstract:
Executive Summary
Introduction
Three out of five Minnesotans are overweight or obese and more than one in seven Minnesotans smoke cigarettes. Obesity and overweight increase the risk of heart disease and cigarettes smoking leads to increased risk of cancer and heart disease. To address obesity, overweight, and tobacco use, the State of Minnesota implemented the Statewide Health Improvement Program (SHIP) – a policy, systems and environment (PSE) change initiative. The PSE framework shifts the public health approach from targeting individual behaviors to targeting the policy, systems, and environmental contexts in which individual behaviors occur. Here, policies are defined as ordinances usually passed by governing authorities, systems are practices within an individual organization, and the environment is the physical space in which the behaviors of interest take place. Using this approach, SHIP seeks to make healthier choices easier choices.
SHIP is coordinated by the Minnesota Department of Health (MDH) and funds are disbursed to grantees – which consist of local public health departments and community health boards – who implement program strategies in different settings. Evidence based strategies to increase physical activity and healthy eating and reduce tobacco use are used in schools, worksites, the community, healthcare facilities, and childcare homes and centers.
The purpose of the project was to analyze evaluation data from the SHIP strategy that targeted the child care setting. Specifically, the focus of the evaluation was to measure the impact of a best practices training on child care providers’ adoption of PSE changes across three areas: healthy eating, physical activity, and breastfeeding support.
Organization
The Office of Statewide Health Improvement Initiatives (OSHII) is a section of the MDH that takes a PSE approach to targeting and preventing illness. The MDH is the State if Minnesota’s public health agency. OSHII focuses on four core areas: healthy eating, physical activity, tobacco use, and alcohol and drug abuse. In terms of the SHIP child care strategy fourteen grantees were involved, hosting trainings conducted by a child care licensing trainer, Providers Choice. Grantees used SHIP funds to host the trainings.
Experience
I analyzed evaluation data to create a technical report for the MDH and shorter reports for local public health entities. The SHIP child care strategy consisted of a training and receipt of technical assistance. Forty-five trainings were conducted between July 2014 and April 2015. Individual child care providers attended only one training. The training covered a few best practices in healthy eating, physical activity and breastfeeding support. For example, trainees were instructed about the optimal playtime duration for toddlers and preschoolers, how to provide family-style meals, and how to properly handle and store breastmilk.
PSE change was measured using computer-based, self-administered pre- and post-surveys. Pre-surveys were completed by providers during an online registration. Providers who did not register for trainings online took the surveys in person at the training. Providers Choice, the training organization coordinated and collected survey data and forwarded it to the MDH. Post-surveys were completed by providers approximately six months after they attended a training. Of the 634 providers who attended trainings, 632 took pre-surveys but only 241 (38%) took post-surveys. Post-surveys were not collected for trainings between January 2015 and April 2015, which was one reason for the low response rates. Providers who took both surveys constituted the evaluation sample. One problem with the data was that some questions were introduced into the pre survey at later dates. For example, those who attended a training earlier in the study period were not asked about best practices in breastfeeding support. Thus, the sample sizes for each pre-post comparison varied according to when questions were introduced into the survey (Range of responses for individual questions, 56-237). The proportion of providers who achieved best practices in each PSE measure was estimated using McNemar’s test for paired binary data. The number of questions that measured PSE change were three for physical activity, four for healthy eating, one for breastfeeding support, and six for adopting and communicating written policies in those areas (these six items were aggregated since they showed high internal consistency, Cronbach’s alpha =.85).
Results
We had two variables to characterize providers who did and did not complete the post-survey: the two groups did not significantly (p<.05) vary by urbanicity (measured according to the U.S. Census metropolitan and micropolitan statistical area designations, on the county level) or by the income level of the neighborhood they served (measured according to the USDA Child and Adult Care Food Program tier level).
There were increases in the proportion of providers achieving best practices in physical activity for preschoolers (54% at baseline, 65% post-training, p<.05), for toddlers (57% at baseline, 78% post-training, p<.001), and in the use of portable play equipment (70% at baseline, 79% post-training, p<.05). There was an increase in the proportion of providers who achieved best practices in having vegetables on their menus (39% at baseline, 52% post-training, p<.001), and an increase in allowing children to serve their own meals (7% at baseline, 14% post-training, p<.001). There was an increase in the proportion of providers who achieved best practices in breastfeeding support (68% at baseline, 93% post-training, p<.001). There was an increase in the proportion of providers adopting and communicating written policies in the areas of physical activity, healthy eating and breastfeeding support (7% at baseline, 21% post-training, p<.001).
Lessons Learned
To complete this project, I had to develop skills in data management and analysis. The dataset required a lot of work in terms of cleaning. I also had to work with content specialists so that I could gain a better understanding of the data and how to handle analysis. I also had to translate the findings as needed. The technical report I wrote was circulated in our office for internal use, while I wrote shorter report to share with the local public health departments that participated in the SHIP child care strategy. Some of the findings were communicated by our unit director to the state legislature in a report, to which I made some brief contributions.
During my time at the MDH three of the five evaluators in my unit left for other positions, including the evaluator that who headed up the childcare project. Because of these transitions, I was asked to do the data analysis and reporting since I understood it better than the other staff, and it was thought I could do the tasks with little mentorship. Through this, I learned the importance of institutional memory. If I had not remained at the MDH it would have been unlikely that the evaluation data would have been analyzed and reported. While the evaluation design was good, the implementation (and consequent incompleteness) of data collection for the post-survey affected the strength of the results. These problems with staffing and survey implementation proved to be instructive, as I learned how to analyze real-world “messy” data and report and interpret the results cautiously.
Overall, the field experience was a rich learning opportunity. I was able to experience the pros and cons of state-level public health work, and benefited from working with, and learning from public health evaluators for at least some portion of the field experience.
Recommendation
Evaluating the SHIP childcare strategy was challenging. For me, the process highlighted the importance of completely funding evaluations. For future evaluations of SHIP strategies, I recommend tighter evaluation designs with more controls of study implementation, especially as it relates to consistency in survey questions and wording throughout the evaluation period, as well as optimal follow-up with participants.
Conclusion
The results suggest that trainings, such as the one offered for the SHIP child care strategy, could be a way to bring about positive PSE change. However, because of the low response rates, the variation in the survey questions during the study and the lack of control variables, such conclusions cannot be made with certainty. A more rigorous study is needed to corroborate the findings.