Master's Project Title:

Program Evaluation and Health Literacy Promotion at a Local Non-Profit (Executive Summary)

MCH Student:

Jenna Hullerman

Date of Defense:

April 28, 2017

Abstract:

1) Introduction: Blood and marrow transplant (BMT) is an expensive, high stress process (Kuba et al. 2016; Cuhadar et al. 2016; Bartley et al. 2014).  To improve outcomes for BMT recipients, I assisted a Health Services Research (HSR) team in addressing two major public health concerns related to BMT.  The first was in the area of health literacy.  Per the definition found on www.health.gov (2017), health literacy is “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions”.  In an effort to increase health literacy among BMT recipients, I worked with a team of editors and reviewers to translate a complex consent form for a clinical trial into plain language.  The second major public health concern my preceptor and I addressed was in the area of effectively evaluating social support programs for BMT recipients.  Together, my preceptor and I planned and implemented an ongoing evaluation for a psychosocial support program for BMT survivors.  

2) Experience: Although I have had the opportunity to work on many projects throughout my time at Be The Match® , I will highlight two major projects.  Through these two projects, four learning objectives were achieved:

  • Learned how to incorporate evaluation components into program planning
  • Learned how to use findings of evaluation research for program improvement and implementation.
  • Worked with senior staff in improving health information accessibility for Be The Match® patient audiences.
  • Became more proficient in determining program evaluation needs, approach, appropriate measurement tools/instruments, and implementation.

The purpose of the first project was to provide support in the Easy-to-Read Informed Consent (ETRIC) Study (Denzen et al. 2012).  The ETRIC Study was borne from a desire to increase health literacy among the BMT population, and through this, increase health literacy for potential clinical trial participants.  This study, a randomized controlled trial (RCT), involved two arms: one arm (control) used standard informed consent forms to provide background on the study to potential participants.  The other arm used a newly created ETRIC format for the consent form design.  The hypothesis was that the ETRIC design will result in “a higher level of patient satisfaction and lower state of anxiety”.  My involvement in the ETRIC study was in providing support for the review and translation of a complex standard consent form into plain language, using the ETRIC format.  We maintained regular contact with the Primary Investigators (PIs) to make sure that any updates did not change, in their view, the accuracy of the information being communicated in the consent.  Once we made edits, received feedback from PIs, and then created the final draft, it was submitted back to the PIs for final review.  

In the second project, an evaluation program was implemented to assess the effectiveness of a monthly phone-based support program for BMT survivors.  This program, “Making Connections – Transplant Survivors Open Chat” (“Survivorship Chats”), was created so BMT survivors could have a safe space to share their experiences with other survivors, without judgment.  In response to the request for an evaluation program, my preceptor and I created an evaluation plan with clearly defined objectives and goals.  The plan laid out a satisfaction survey that would be sent out after each monthly chat session.  After each survey was closed, my preceptor and I created evaluation reports, summarized the results and gave recommendations for next steps.

3) Organization: The NMDP, or the National Marrow Donor Program, is known better by its public name, Be The Match®.  They are a non-profit based in Minneapolis that manages the United States national bone marrow registry.  This organization is also largely involved in research to increase transplant recipients’ survival and quality of life.  Among the kinds of research being conducted are those investigating other conditions that may be treated with BMT.  My primary involvement is with the Health Services Research team, which focuses on improving health services and outcomes for patients, their caregivers, and their families.    

4) Lessons Learned: Prior to involvement in the ETRIC study, the concept of plain language was alien to me.  As a long-time student, it was natural for me to follow certain rules in writing (e.g. writing in the third person and avoiding contractions).  Now, it has become second nature that, when drafting any kind of form, letter, or other document intended for public consumption, I use the basic principles of plain language to ensure the intended message is being received by the greatest number of people.  Secondly, in performing evaluation of a social support program, I learned that, even from afar (via phone), effective support can be provided for survivors of transplant.  It became clear that such support programs are strengthened when partnered with a strong, evidence-based formative evaluation program.  Formative evaluation is “generally any evaluation that takes place before or during a project’s implementation with the aim of improving the project’s design and performance.” (French et al. 2017).  If a department fosters a culture of constant improvement, formative evaluations ensure any program changes truly bring the program closer to reaching its intended goals and objectives.  Finally, I learned the importance of clearly defining the roles of both evaluators and program managers in the evaluation process from the beginning.  As with any project or collaborative effort, a failure to define each party’s roles can lead to confusion, miscommunication, and reduced efficiency (Biddle 1979).  The HSR team I worked with recognizes this, and makes every effort to ensure clear, regular communication is achieved where possible.

5) Recommendations: After the opportunity to become involved in these two projects at Be The Match®, I have three recommendations for the organization.  My first recommendation would be to continue moving toward an organization that is increasingly evaluation-focused.  As the organization continues to expand and broaden its horizons, this would ensure that they remain mission-driven, with patients and donors the center of all they do.  Second, I would strongly recommend expanding the emphasis on using plain language in all public documents and websites.  The health world, for many, is wrought with confusion and stress, and making information easier to read can help ameliorate this.  Last, I would recommend an evaluation of services offered through local clinics and hospitals of transplant recipients.  This could help inform programs offered by Be The Match®, so they can better address any gaps in services.

6) Conclusion: A field experience with Be The Match® has proven an educational experience on many fronts.  By learning how to increase health information accessibility and incorporating evaluation into health programs, I now better understand how to tailor a program so it truly helps a population in need.  Based on my recommendations, Be The Match® would continue to move toward providing services that are increasingly patient-centered.

References

  1. Bartley, E. J., Edmond, S. N., Wren, A. A., Somers, T. J., Teo, I., Zhou, S., … & Shelby, R. A. (2014). Holding back moderates the association between health symptoms and social well-being in patients undergoing hematopoietic stem cell transplantation. Journal of pain and symptom management, 48(3), 374-384.
  2. Biddle, B. J. (1979). Role theory: expectations, identities, and behaviors. New York, NY: Acad. Pr.
  3. Cuhadar, D., Tanriverdi, D., Pehlivan, M., Kurnaz, G., & Alkan, S. (2016). Determination of the psychiatric symptoms and psychological resilience levels of hematopoietic stem cell transplant patients and their relatives. European journal of cancer care, 25(1), 112-121.
  4. Denzen, E. M., Santibáñez, M. E. B., Moore, H., Foley, A., Gersten, I. D., Gurgol, C., … & Murphy, E. A. (2012). Easy-to-read informed consent forms for hematopoietic cell transplantation clinical trials. Biology of Blood and Marrow Transplantation, 18(2), 183-189.
  5. French, J., Crosier, A., McVey, D., Eagle, L., Dahl, S., Low, D. R., . . . Montagu, D. (2017). Social marketing and public health: theory and practice. Oxford: Oxford University Press.
  6. Health Literacy and Communication. (n.d.). Retrieved April 07, 2017, from https://health.gov/communication/about.asp
  7. Kuba, K., Esser, P., Scherwath, A., Schirmer, L., Schulz‐Kindermann, F., Dinkel, A., … & Mehnert, A. (2016). Cancer‐and‐treatment–specific distress and its impact on posttraumatic stress in patients undergoing allogeneic hematopoietic stem cell transplantation (HSCT). Psycho‐Oncology.