Table of Contents

Table of Contents
Schedule of Events (2)
Biographical Sketch of Plenary Speaker (3)
List of University of Michigan Speakers (4)
Program Planning Committee (4)
Program Objectives (4)
Abstracts
Computer Demonstrations (5)
Poster Presentations (9)
Medical Education Day 2009
June 2, 2009
Schedule of Events
11:00 a.m.Poster and Computer Demonstrations on display
Towsley Center Lobby
12:00 p.m.Plenary Session
MCHC Auditorium
Welcome: James O. Woolliscroft, M.D.,
Dean, University of Michigan Medical School
Keynote Presentation:
"The Role of Experience in the Development of
Medical Diagnostic Expertise"
Geoff Norman, Ph.D.
Professor of Clinical Epidemiology and Biostatistics and
Assistant Dean for Educational Research and Development
McMaster University
Question/Answer
Moderator: Stanley Hamstra, Ph.D.
1:00 p.m.Local Response and Panel Discussion
Moderator: Stanley Hamstra, Ph.D.
Panelists: Larry Gruppen, Ph.D
Rajesh Mangrulkar, Ph.D.
Robert Schumacher, M.D.
1:30-3:30 p.m.Reception Poster Presentations/Computer
Demonstrations
Authors will be on-hand to describe and demonstrate the
innovations they have developed, and to discuss their ideas
and experiences with participants.
Guest Faculty
Geoff Norman, Ph.D.
Professor of Clinical Epidemiology and Biostatistics
McMaster University
Geoff Norman, Ph.D., is Professor of Clinical Epidemiology and Biostatistics, McMaster University. He received a B.Sc. in physics form the University of Manitoba in 1965 and a Ph.D. in nuclear physics from McMaster University in 1971, and subsequently a M.A. in educational psychology from Michigan State University in 1977. He is the author of 10 books on education, measurement and statistics, and over 200 journal articles.
Dr. Norman’s primary research is in the area of expert diagnostic reasoning – how clinicians arrive at a diagnosis. His research has revealed that experts use two kinds of knowledge to do diagnosis – the formal analytical knowledge of signs and symptoms and physiologic mechanisms, and experiential knowledge based on the hundreds or thousands of patients they have encountered.
Following from this research is an interest in various aspects of how medical students learn. He has contributed to the theoretical foundation of problem-based learning. He is currently exploring the use of high fidelity simulation in clinical learning. He has also been involved in student assessment, and has developed and validated a number of innovative assessment methods, and has made substantial contributions to assessment methodology.
He has won numerous awards, including the Hubbard Award from the National Board of Medical Examiners in 1989, the Award of Excellence of the Canadian Association for Medical Education in 1997, the Distinguished Scholar Award of the American Educational Research Association, Division I, in 2000, and the Award for Outstanding Achievement of the Medical Council of Canada in 2001. He presently holds a Canada Research Chair. He was elected a Fellow of the Royal Society of Canada in 2007. In 2008, he won the prestigious Karolinska Prize for lifetime achievement in medical education research.
University of Michigan Faculty
Larry Gruppen, Ph.D, Chair, Department of Medical Education, Josiah Macy, Jr., Professor of Medical Education
Rajesh Mangrulkar, Ph.D., Clinical Assistant Professor, Associate Residency Program Director, Department of Internal Medicine
Robert Schumacher, M.D., Associate Professor, Pediatrics and Communicable
Diseases, Medical Director, Holden NICU
James O. Woolliscroft, M.D., Dean, University of Michigan Medical School
Program Planning Committee
Patricia B. Mullan, Ph.D., Associate Director, Medical Education Scholars Program, Associate Professor of Medical Education, Program Planning Chair
Larry D. Gruppen, Ph.D., Chair, Department of Medical Education, Josiah Macy, Jr., Professor of Medical Education,
Stanley J. Hamstra, Ph.D., Director, Medical Education Scholars Program, Associate Professor of Medical Education
Program Objectives
This program is designed for faculty, physicians-in-training, and other health professionals interested in discussions and collaborations about medical education. Participants will have an improved understanding of methods for teaching medical students, residents, and teaching faculty and become aware of the resources that are available in the field of medical education.
The University of Michigan Medical School is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.
The University of Michigan Medical School designates this educational activity for a maximum of 1.50 AMA PRA Category 1 Credit(s)™. Physicians should only claim credit commensurate with the extent of their participation in the activity.
Computer Demonstrations
Collexis Research Profiles: Who knows what at the University of
Michigan Medical School
Athanikar, J.; Comstock, M.; Conte, M., MLIS; Grieb, T., PhD; Hill, M; Horon, J.; Offhaus, H.;
Warden, M.
Objective
To introduce and demonstrate a new expertise portal, Collexis Research Profiles at UMMS, sponsored by the Office of Research. In an effort to synergize inter-disciplinary clinical and translational research, Michigan is committed to using innovative research tools and information technologies to promote. This expertise portal is a key component of Michigan's mission to catalyze the application of new knowledge and techniques to clinical practice at the front lines of patient care.
The expertise portal contains the full PubMed publication records and NIH grants for over 1800 UMMS faculty. Users can search the portal for specific faculty, or by biomedical concept, or browse by department to locate experts in particular domains and identify potential collaborators within UMMS.
Broadening the Impact of Learning Materials Through
Open Educational Resource Publishing
Hanss, T., MBA; Johnson. L., PhD; Ludewig, K.;
Fons, G., MSI; Kleymeer, P., MSI; Fantone, J., MD
In 2007, the University of Michigan Medical School, with the support of the Hewlett Foundation and the sponsorship of Dean Woolliscroft as principal investigator, launched the Health Open Educational Resources initiative. Open Educational Resources (OER) are learning materials offered freely and openly for faculty, students, and self learners around the world to use, improve, and redistribute. In 2009, the project’s primary objectives are to maximize the impact and visibility of our faculty-created educational content by converting our pre-clinical curricular materials into open educational resources and collaborating with universities in Ghana and South Africa to co-create new learning materials.
Central to Health OER is using students as dScribes, “digital and distributed scribes,” who work with participating faculty to publish their materials online. A 2008 summer pilot study involved medical students putting material from five M1 and M2 sequences online at <https:///education>. During the summer of 2009, a new group of medical students aims to publish the rest of the M1 and M2 sequences plus additional learning materials offered by faculty.
Also participating in Health OER and the umbrella Open.Michigan initiative are Dentistry, Public Health, Nursing, Information, Public Policy, Engineering, LSA, and the University Library. Through Hewlett Foundation support we are collaborating on OER policy issues and learning material co-creation with OER Africa (a Hewlett Foundation-funded outreach organization), two universities in Ghana (University of Ghana and Kwame Nkrumah University of Science and Technology), and two in South Africa (University of Cape Town and University of the Western Cape). Additional funding for the efforts in Ghana comes through the Center for Global Health’s Gates Foundation Human Resources for Health grant.
The poster will document the motivation for Health OER, our experience training student dScribes and developing OER materials over the past year, feedback from faculty, our collaboration with the African universities, and the research underway to study the impact of OER on faculty productivity, student learning, and inter-institutional collaboration. We will demonstrate the software tools we developed to support OER publishing, the medical school sequences that have been published as open educational resources, and the learning materials developed in collaboration with African universities (facilitated by the efforts of Dr. Cary Engleberg during his current sabbatical in Ghana). We will invite faculty and other educators to engage with Health OER to broaden the visibility and impact of their health sciences learning materials.
Online, Self-paced Learning Modules in the Family Medicine Residency Skye E. MD; Master-Hunter, T. MD; Locke, A. MD; Wilson, E. MFA; Wimsatt, L. PhD
The transition to online learning has many benefits, particularly in a resident learning environment, where irregular and limited availability renders coordinated or group teaching increasingly challenging.[1] Based on annual retreat feedback, the Department of Family Medicine (DFM) became aware of limitations placed on resident learning due to the size/scope of the required curriculum and to a variety of residency scheduling constraints. In response, a decision was made to develop ten modules that would serve as a required supplement to pre-existing didactic and clinical education experiences. Design and planning for the project followed the same general pattern of development recommended by Cook and Dupras (2004) for creation of effective web-based learning materials.[2]
The purpose of this pilot study was to examine the effectiveness of web-based learning modules developed by the Department of Family Residency in delivering instruction to resident learners and to examine the perceptions of faculty involved in the design and development process. Secondary analysis was used to examine: (1) work process data collected from the core project team; (2) usability testing data collected from residents, faculty, and staff; and (3) preliminary pre-post test data collected from residents. Outcome measures included participant satisfaction, software usability and functionality, workload perceptions, training and support adequacy, and preliminary content knowledge assessment. Five-point Likert scales captured ratings of perceived importance, agreement, satisfaction, and difficulty of use. Frequency distributions, descriptive statistics, and mean comparisons were used to analyze quantitative data. Qualitative data analysis involved thematic analyses of faculty feedback.
Results indicate that faculty satisfaction with aspects of the module development process improved with training and as the design team became familiar with faculty needs. Software limitations and work timelines created the greatest challenge for faculty involved in module design and development. Usability tests revealed that most residents (96%) thought the online material met their educational goals, and that learning outcomes were clear and quizzes well structured. Most (91%) reported the units had a logical flow, 88% thought the units contributed to their understanding of content, and 81% rated the format as effective. A majority of resident quiz scores (102 of 123) have shown pre-/post improvement to date, with 77 of 123 scores moving from “below passing” to “passing” (i.e., passing with scores of 80% or above). Overall, the findings suggest that web-based modules can offer an effective and viable method of delivering required content material to resident learners.
Supported by a grant from the U-M Graduate Medical Education Innovations Program and human subjects study approval from IRB Med (HUM00007398).
[1) Cook DA. Web-based learning, pros, cons & controversies. Clin Med. 2007;7(1):37-42.
[2] Cook DA, Dupras DM. A practical guide to developing effective web-based learning. J Gen Intern Med 2004;19:698-707.
Poster Presentations
Designing A Needs Assessment Tool to Guide Curriculum Development For Pediatric Residency Programs Addressing
Expanded Newborn Screening.
Ahmad, A, MD;1, Mullan, P, PhD2
1Department of Pediatrics, Division of Genetics, University of Michigan
2 Department of Medical Education, University of Michigan
Introduction: All newborns in the United States undergo newborn screening. The process of newborn screening has undergone expansion to include over 40 inherited inborn errors of metabolism. Although this process has now spanned almost a decade, it remains heterogeneous, state-specific, without nationwide uniformity. This heterogeneity has affected appropriate education of current medical trainees about the disorders included in the NBS panel. Disparity of genetics services remains a national reality and many pediatricians do not have direct access to a genetics center. Based on prior studies, many pediatricians feel they are not competent to discuss with parents a positive newborn screen for rare inborn errors of metabolism.
A pediatric residents’ ability to adequately manage a family when faced with a positive newborn screen for a metabolic disorder, is very dependent on the exposure received at the training institution. Unfortunately clinical genetics training, if available, is not a required rotation in most if not all pediatric residency programs.
Objective: Our objectives included development of a survey to evaluate the status of content addressing expanded newborn screening currently incorporated into pediatric residency programs. Our plan is to survey all pediatric residency program directors for purposes of an initial needs assessment before developing a curriculum that addresses expanded newborn screening for pediatric residencies. The purpose of this survey is to evaluate in what format and how many times, content addressing newborn screening is presented during pediatric residency. It is NOT our assumption that all pediatric programs cover this material.
Conclusion: Newborn screening is a critical public health initiative that dates back to the 1960’s. More recently, expanded newborn screening for rare metabolic disorders is allowing pre-symptomatic diagnosis of over 40 metabolic conditions. Individually rare; collectively, these conditions are relatively common. Yet they remain very unfamiliar to the practicing pediatrician. Nationally there is a recognition of the need to adopt a uniform screening panel, establishing appropriate laboratory screening values, establishing disease registries to track long-term outcomes; increasing access to information about newborn screening and genetic resources, and coordinating services and family support systems.
Designing A Needs Assessment Tool to Guide Curriculum Development For Pediatric Residency Programs Addressing Expanded Newborn
Screening (page 2)
To date, there has not been a unified effort to ensure that future pediatricians (current pediatric residents), graduate without the knowledge gap that many current pediatricians are facing after the rapid expansion of newborn screening. Our objective is to improve the education of pediatric residents by developing a tool that allows us to identify and analyze the current status of education addressing newborn screening by surveying pediatric residency program directors. We hope this will lead us to develop an approach to curriculum development for expanded newborn screening that can be readily implemented into most pediatric residency programs.
Stress Factors in Simulation-based Laparoscopic Training
Andreatta, PB, EdD; Hillard, ML, BS; Krain, L, MD
Objectives:
Stress factors are known to effect cognitive and psychomotor performance. Simulation-based medical training typically occurs in environments that are relatively stress-free compared to the applied context. Training is most effective when it occurs in a highly faithful context, so the inclusion of typical clinical stressors in simulated-laparoscopic learning environments may be beneficial. The study objectives were to quantify learner reactions manifesting from a realistic contextual stressor while training with a laparoscopic simulator, and identify learner-derived stress modifying behaviors.
Methods:
Pre-clinical medical students (N=27) completed tasks using a laparoscopic simulator under two conditions: faculty direct observation (stressor), unobserved (no stressor). Data included: simulator performance, BP, real-time heart rates, videotaped behavior, and pre/post surveys of latent anxiety and stress factors associated with participating in the research (e.g. STAI 9).
Results:
Physiological and behavioral manifestations of stress were observed for all participants during the stressor condition and during poor performance on simulator tasks. Stress was highest during periods of poor performance under the stressor condition. Focusing on the task itself mitigated stress reactions and improved performance on the simulator. Conclusions:
Stress reactions can be induced in a lab setting where simulation-based training occurs. Stressors imposed on the learner during simulation-based training may help support the acquisition of stress management skills that are necessary in the applied clinical setting. A ramped up sequence of acquiring technical skills and clinical decision making, followed by stress management techniques may lead to better transfer of learning from the simulated context to the clinical arena.
References
1. Easterbrook, J. A. (1959) “The effect of emotion on cue utilization and the organization of behavior.” Psychological
Review, 66, 187-201.
2. Humara, M. (2002) “The relationship between anxiety and performance: A cognitive-behavioral perspective.”
Athletic Insight, 2, 1-11.
3. Inzana, C. M.; Driskell, J. E.; Salas, E. & Johnston, J. H. (1996) “Effects of preparatory information on enhancing
performance under stress.” Applied Psychology, 81, 429-435.
4. Keinan, G.; Friedland, N. & Sarig-Naor, V. (1990) “Training for task performance under stress: The effectiveness of
phased training methods.” Journal of Applied Social Psychology, 20, 1514-1529.
5. Mandler, G. (1979) “Thought processes, consciousness, and stress.” (pp. 179-201) In V. Hamilton & D. M.
Warburton (Eds.) Human stress and cognition: An information processing approach. New York: John Wiley & Sons.
6. Ozel, F. (2001) “Time pressure and stress as a factor during emergency egress.” Safety Science, 38, 95-10
7.
7. Saunders, T.; Driskell, J. E.; Johnston, J. H. & Salas, E. (1996) “The effect of stress inoculation training on anxiety
and performance. “Journal of Occupational Health Psychology, 1, 170-186.
8. Smith, R. E. & Nye, S. L. (1989) “Comparison of Induced Affect and Covert Rehearsal in the Acquisition of Stress
Management Coping Skills.” Journal of Counseling Psychology, 36, 17-23.
9. Van Galen, G. P. & Van Huygevoort, M. (2000). “Error, stress and the role of neuromotor noise in space oriented
behaviour.” Biological Psychology, 51, 151-171.
10. Van Gemmert, A. W. A. & Van Galen, G. P. (1997) “Stress, neuromotor noise, and human performance: A
theoretical perspective.” Journal of Experimental Psychology: Human Perception and Performance, 23, 1299-
1313.
Clinical Impact of Pediatric Mock-Codes
Andreatta, PB, EdD; Marsh, M, MS; Annich, G, MD
Objectives:
Rapid and accurate clinical response is a critical factor in the successful management of cardiopulmonary arrest. This clinical response is often referred to as a “code” and is a coordinated effort of multiple specialists performing emergency procedures under the direction of a senior resident. The resident’s leadership ability is integral to accurate and efficient clinical response to the patient. Experience managing codes is a contributing factor to a resident’s leadership ability. The purpose of this study was to determine the impact of a simulation-based mock code program for pediatric medicine residents on clinical outcomes of pediatric patients in cardiopulmonary arrest.
Methods:
Mock codes for pediatric medicine residents were randomly called at least monthly. All code team members responded to the mock event as they would an actual event.
36 mock code events were called over 18 months. Mock codes were recorded and used for immediate debriefing facilitated by clinical faculty and including other team members whenever possible. Hospital records for pediatric resuscitations were examined for two years beginning 6-months prior to the mock code program and continuing for 18 months after its start.
Results:
Resuscitation rates for pediatric patients increased from 3% to 48% after the initiation of the simulation-based mock code program.
Conclusions:
A mock code program can significantly benefit patient outcomes in the clinical response to cardiopulmonary arrest in pediatric patients. This study provides clinical evidence of the value of simulation-based training for the benefit of patient care. References
1. Abella, B. S., J. P. Alvarado, et al. (2005). "Quality of cardiopulmonary resuscitation during in-
hospital cardiac arrest." JAMA 293(3): 305-10.
2. Eisenberg, M. S. and T. J. Mengert (2001). "Cardiac resuscitation." N Engl J Med 344(17):
1304-13.
3. Hunt, E. A., A. R. Walker, et al. (2008). "Simulation of in-hospital pediatric medical
emergencies and cardiopulmonary arrests: highlighting the importance of the first 5 minutes."
Pediatrics 121(1): e34-43.
4. Risser, D. T., M. M. Rice, et al. (1999). "The potential for improved teamwork to reduce
medical errors in the emergency department. The MedTeams Research Consortium." Ann
Emerg Med 34(3): 373-83.
5. Roback, M., S. Teach, et al. (1998). Handbook of Pediatric Mock Codes. St Louis, Mosby.
6. Su, E., T. A. Schmidt, et al. (2000). "A randomized controlled trial to assess decay in acquired
knowledge among paramedics completing a pediatric resuscitation course." Acad Emerg Med 7(7): 779-86.
A Typology for Healthcare Teams
Andreatta, PB, EdD
Objectives: The effectiveness of healthcare teams is critical to successful outcomes in patient care, as well as to the successful maintenance of practice across healthcare systems. Because effective teamwork improves both clinical and financial outcomes, interdisciplinary team training to improve team efficiencies and accuracy and assessment of team-based competencies are central to establishing patient safety oriented practice. Proposed competencies for healthcare teams are based on theoretical and empirical work derived from other professional domains. The purpose of this study was to determine which team development and performance models are most applicable to interdisciplinary team training in healthcare contexts comprising all aspects patient care.
Methods: 25 healthcare teams were observed over a 6-month period in multiple public/private hospital and ambulatory care patient care settings. Team composition and role behaviors were compared and categorized through the constant comparative method of analysis.
Results: Four types of healthcare teams were identified: stable role, stable personnel (Type SS); stable role, variable personnel (Type SV); variable role, stable personnel (Type VS); variable role, variable personnel (Type VV). Healthcare teams are more complicated than teams in other professions where teams are typically Type SS, i.e. a specific individual serves in an established capacity to benefit team performance.
Conclusions: A singular model derived from other professional domains will not adequately inform training specific to interdisciplinary healthcare teamwork. To achieve optimal performance, each team type will require adaptable training strategies. The 4-types of healthcare teams designated by this study will help inform the selection of appropriate team training and assessment of competencies. References:
1. Dinka, TJK and Clark, PG., Health Care Teamwork: Interdisciplinary Practice and Teaching. 2000, Westport,
CT: Auburn House. 8-40.
2. Wears RL, Leape LL. Human error in emergency medicine. Ann Emerg Med 1999;34:370-372.
3. Risser DT, Rice MM, Salisbury ML, Simon R, Jay GD, Berns SD. The potential for improved teamwork to
reduce medical errors in the emergency department. Ann Emerg Med 1999; 34:373-383.
4. Heinemann GD. Teams in Health Care Settings. In: Heinemann GD and Zeiss AM (Eds.) Team Performance
in Health Care: Assessment and Development. New York: Kluwer Academic Plenum; 2002: 6-13.
5. Salas E, Rosen MA, King HB. Managing teams managing crises: Principles for teamwork in the ER and
beyond. Theor Iss Ergonomics Sci 2007; 8:381-394.
8. Morey JC, Simon R, Jay GD, et al. Error reduction and performance improvement in the emergency
department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res 2002;37:1553-1581.
9. Shapiro MJ, Morey JC, Small SD, et al. Simulation based teamwork training for emergency department staff:
does it improve clinical team performance when added to an existing didactic teamwork curriculum? Qual Saf Health Care 2004;13:417-421.
10. Gaba DM. The future vision of simulation in health care. Qual Saf Health Care 2004;13(suppl 1):i2-i10.
11. Lemieux-Charles L, McGuire WL. What do we know about health care team effectiveness? A review of the
literature. Med Care Res Rev 2006;63:263-300.
12. Salas E, Sims DE, Burke CS. Is there a big five in teamwork? Small Group Res 2005;36:555-599.
13. Rosen MA, Salas E, Wilson KA, et al. Measuring team performance in simulation-based training: adopting
best practices for healthcare. Simulation in Healthcare Spring 2008; 3(1): 33-41.
GETTING AGREEMENT AMONG SURGEONS ABOUT ERROR: A
STATISTICAL PROBLEM
Arble, E.P, Gauger, P.G., Minter, R.M., Hamstra, S.J., and Hauge, L.S.
University of Michigan Departments of Surgery and Medical Education Background. Performance evaluation of operative procedures is frequently conducted using global ratings. Although these estimations are often informative, their ability to measure specific components of the procedure is uncertain. More recently, researchers have developed instruments to quantify errors incurred during a laparoscopic procedure. The purpose of this study is to examine the reliability of an instrument for codifying operative challenges, and the statistical challenges presented by measuring surgical error. Methods. As part of an efficacy study of a laparoscopic skills curriculum, a series of laparoscopic cholecystectomy procedures were recorded. Videos were edited to focus upon the excisional phase of the procedure, and each video was separated into minute-long units. Two attending surgeons completed rater training which consisted of discussion of error categories and scoring procedures, and independent and collective observations of videotaped procedures. Interrater agreement was calculated at various intervals during the training to determine rater readiness for video coding. Although raw agreement exceeded the recommended rate of 80%, kappas remained somewhat low. Additional training and revisions to the scoring instrument ensued, and an additional technique for calculating interrater reliability as described by Cicchetti and Feinstein (1990) was applied. Raters viewed and score the videos independently, unaware of the other rater’s results. Levels of agreement were calculated for each case and each error category.
Results. Reliance upon raw agreement is susceptible to two essential flaws. Firstly, assuming even a moderate amount of proficiency on the part of the surgeon, the majority of a temporally-based scoring matrix will be comprised of non-errors. Thus, a failure of raters to agree that errors have occurred may be overshadowed by the frequency of agreement regarding a lack of error. Secondly, raw agreement offers no correction for the percentage of agreement accounted for by chance in a dichotomous coding system (i.e. error or no error). To address these deficiencies, Cohen’s kappa is frequently employed. However, the overwhelming proportion of “negative agreement” unbalances the assumption of relative symmetry that kappa requires, thereby causing the statistic to overcorrect and produce unreasonably harsh evaluations. This unbalance can be addressed by utilizing P pos and P neg calculations. Table 1 depicts the results of our analyses which yielded widely discrepant characterizations of interrater reliability. The calculation of raw agreement noted 90% agreement. Conversely, Cohen’s kappa was moderate (k=.39). The values of positive and negative kappa were similarly divergent, with P pos = .44 and P neg = .95.
Conclusion. The quantification of surgical error, especially estimation of surgeon agreement on error, is confounded by traditional techniques for calculating interrater reliability. Interrater agreement estimates that are frequently reported in surgical research on performance error may overestimate actual rater agreement. Error coding methods in surgical research are not adequately categorized by raw agreement or an omnibus index. Rather, an examination of these values, in conjunction with the calculation of separate indices of positive and negative agreement, offers the most effectives means of establishing interrater reliability.。