Doctor Coach

Don't teach more, coach better!

References

The following is a select list of the most important references that have influenced the development of the Doctor Coach framework and principles as well as its implementation through workshops and modules.  We will continue to update this list of relevant literature as sections are added to this website and new resources become available.  

General Coaching

Cooke M, Irby D, O’Brien B. Educating Physicians: A Call for Reform of Medical School and Residency. San Francisco: Jossey-Bass; 2010.

Note

Cooke, Irby, and O'Brien provide a discussion of future directions required for effective medical education, emphasizing the need for continuous learning. They recommend standardization of learning outcomes, strengthening the connections between formal and experiential knowledge, encouraging collaboration with other health professionals, and improving selection and training of medical educators. 

Kolb DA. Experiential Learning: Experience as the Source of Learning and Development. New Jersey: Prentice-Hall; 1984.

Neher JO, Gordon KC, Meyer B, Stevens N. A five-step "microskills" model of clinical teaching. J Am Board Fam Pract. 1992;5:419–424; and a related handout.

Note

Neher et al. present a five-step micro-skill model for beginning teachers to assist in providing a framework for teaching encounters in clinic. The steps include getting a commitment, probing for supporting evidence, teaching general rules, reinforcing what was done right, and correcting mistakes. These steps allow the preceptors to evaluate the knowledge the resident uses and how they use this knowledge. Each step is described in detail with examples of questions preceptors could ask students in order to guide the discussion. These steps can be taught in a 1-2 hour workshop, and data indicates that clinicians who were taught it frequently utilize a similar model. 

Schon D. Educating the Reflective Practitioner. San Francisco: Jossey-Bass; 1987.

Thompson J. Positive Coaching. Portola Valley: Balance Sports Publishers; 1995.


Deliberate Practice

Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VR, van der Vleuten CP, Scherpbier AJ. The role of deliberate practice in the acquisition of clinical skills. BMC Med Educ. 2011(Dec 6);11:101.

Note

Deliberate practice, a learning process in which students repeat a structured activity with the aim of improving their performance of that activity, has been found improve acquisition and maintenance of skills in many fields including sports, music, and economics. In this study, the authors sought to examine how medical students deliberate practice abilities changed during the course of their training. To do so, medical students in the Netherlands were given Moulaert's questionnaire at the end of years 1-3 of their undergraduate medical education. Students planning ability tendency to structure their work increased during the course of their education, while their need for repetition and revision declined over time. These results indicate that students learned how to learn, and that incorporation of workshops on deliberate practice techniques could aid in achievement of educational goals.

Ericsson KA (2004). Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domainsAcad Med. 2004;79:S70-S81.

Note

Ericsson argues that students’ attainable performance is not limited by basic endowments, as the most common explanation argues; rather, acquisition of expert performance can be achieved through deliberate practice. Although expertise has traditionally been correlated with experience, amount of experience does not seem to correlate with outcome. Instead of defining experts as those most experienced in their field, experts should instead be defined as those who display superior performance at a task in a reproducible and measurable manner, although at least ten years of experience appears to be necessary to reach an international level of skill. Improved performance occurs through deliberate practice - when learners are given well-defined tasks to improve their performance, receive immediate and detailed feedback, and have many opportunities to repeat the original task. When deliberate practice is engaged, expert performance can continue to improve. Ericsson then discusses how this model of expertise can be applied to diagnosing disease, diagnosing patients, and success in surgery. 

Issenberg SB, McGaghie WC, Gordon DL, Symes S, Petrusa ER, Hart IR, Harden RM: Effectiveness of a cardiology review course for internal medicine residents using simulation technology and deliberate practice. Teach Learn Med. 2002;14:223–228.

Note

Retention of many basic clinical skills such as cardiac auscultation has been found to be poor, even among practicing physicians. The deliberate practice method of instruction has been found to be an effective approach for encouraging mastery of skills in many disciplines, especially when in combination with simulation tools. Therefore, Issenberg et al. used educational intervention designed in a deliberate practice methodology to teach bedside cardiology skills using a patient simulator. Participants’ cardiology knowledge was evaluated before and after the course. Post-test knowledge was significantly increased for both medical students and residents who participated in the training, indicating the value of a deliberate practice approach to teaching clinical skills.

McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86(6):706–11.

Note

This quantitative meta-analysis aims to compare clinical skill acquisition outcomes of traditional clinical education compared with simulation-based medical education with deliberate practice (SBME-DP). Overall, SBME-DP education appears to improve patient care in a variety of fields, as measured by patient outcomes, improved ability to perform procedures, and overall Comparative Effectiveness Research (CER). 

McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Medical education featuring mastery learning with deliberate practice can lead to better health for individuals and populations. Acad Med. 2011;86(11):e8–9.

Note

This brief discussion of the importance of mastery learning (ML) and deliberate practice (DP) highlights the theory behind these educational techniques and covers the research supporting their employment in medical education. Patient outcomes can be improved by use of ML and DP in combination with competency assessment. The new field of implementation science has promise for guiding the incorporation of ML and DP into medical education as the field continues to evolve. 


Adult Learning Theory

Knowles M, Holton EF III, Swanson, RA. The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development. (6th ed.)Burlington, MA: Elsevier; 2005.

Mylopoulos M, Regehr G. Cognitive metaphors of expertise and knowledge: prospects and limitations for medical education. Med Educ. 2007;41(12):1159–65. 

Note

Identification of the nature of expertise will allow better development of experts and improve curricula design. However, traditional methods of defining expertise contain paradigmatic assumptions including a focus on measuring expertise by evaluating clinical reasoning ability rather than on evaluating the thought process practitioners engage in to complete tasks. Research indicates that there are, however, two types of experts: routine experts and experts who think more creatively, adaptive experts. These have different developmental pathways, and identification of the differences between them can inform medical education toward generating more adaptive experts in these fields. This paper discusses the varying definitions of expertise as well as some of the assumptions that discussions of expertise have made in the past in order to highlight the need to think about expertise differently to ensure that curricula are designed to foster it.

Varpio L, Services U, Albert M. How Pierre Bourdieu’s theory and concepts can apply to medical education: AM Last Page. Acad Med. 2013;88(8):2013.

Note

This infographic describes Pierre Bourdieu's social theory of practice so that medical educators can determine if the concepts are appropriate for incorporation into their studies. Bourdieu's theory combines objectivist and subjectivist world-views, reconciling how outer and internal worlds interact with each other. He argues that practice is a combination of field (site where individuals vie for position), capital (the resources that individuals are vying for), and habitus (disposition). Each of these concepts and how they interact to create practice is defined

Wong BM, Etchells EE, Kuper A, Levinson W, Shojania KG.Teaching quality improvement and patient safety to trainees: a systematic review. Acad Med. 2010(Sep);85(9):1425–39.

Note

In this review, Wong et al. describe the commonly used methods for teaching quality improvement (QI) and patient safety (PS) and the outcomes of such training. To do so, they analyzed studies for curricular descriptors and methodological features and extracted data regarding evaluation of these intervention programs. They found that curricula most commonly focused on continuous QI, root cause analysis, and systems thinking and usually combined didactic and experiential learning. Before-after comparison was the most commonly-used evaluation technique, but since most of the studies only involved one training program and had small participant sizes, methodological strength was often weak. Barriers to the success of these programs included lack of enthusiasm and competing educational demands among students, inadequate expertise and time, among faculty, and difficulty in achieving balance between didactic learning and experiential learning and finding the time for the additional curriculum elements. Overall, the studies indicated that curricula were well accepted and resulted in acquisition of knowledge. Each study included in this analysis is described in table format.


Establishing Learner Milestones

Benson B, Burke A, Behrman R, et al. The American Board of Pediatrics and the Accrediation Council for Graduate Medical Education. American Board of Pediatrics. The Pediatric Milestone Project. January 2012.

Note

This manuscript contains detailed descriptions of each of the Accreditation Council for Graduate Medical Education's (ACGME's) competencies, as applied to the pediatric specialty. Descriptions include components of each competency, how learners should develop as their skills improve, developmental milestones, and references for further reading. 

Cook D, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Mastery learning for health professionals using technology-enhanced simulation: a systematic review and meta-analysis. Acad Med. 2013;88(8):1178–86.

Note

Mastery learning, in which learners become proficient in one objective before continuing to learn the next skill, is one of the tools used for individualization of competency-based research. This method allows for flexibility in learning time while ensuring that all trainees reach the same objectives. However, empiric evidence of the benefits of this teaching method has not been provided. They did so with the hope to establish whether masterly learning, when compared with no learning intervention or non-masterly learning techniques, improved upon students' learning abilities and, if it did so, what aspects of mastery learning were essential to this effect. To this end, the authors conducted a review of studies that used an mastery-learning model in combination with technological simulation interventions in comparisons without such interventions. Mastery learning SMBE was found to result in higher outcomes in comparison with non-mastery models, although it did take longer than non-mastery interventions. In addition to improved outcomes when compared to models with no intervention, in addition to higher learning outcomes, mastery learning models resulted in improved patient-related behaviors. Overall, mastery learning SBME was found to be an effective teaching technique, and may be especially appropriate for competency-based education since both are designed around objectives. 

Englander R, Cameron T, Ballard AJ, Dodge J, Bull J, Aschenbrener C. Toward a common taxonomy of competency domains for the health professions and competencies for physicians. Acad Med. 2013;88(8):1088–94.

Note

The increasing focus on competency-based medical education has highlighted a need for a common taxonomy to describe competence domains and identify specific competencies. In this paper, members of the Association of American Medical Colleges describe the process of designing such a taxonomy that can now provide an infrastructure for those seeking to use MedEdPORTAL or CIR resources. To do so, 153 competency lists were reviewed and coalesced into a reference list consisting of 58 competencies divided into eight domains. 

Hicks PJ, Schumacher DJ, Benson BJ, et al. The pediatrics milestones: conceptual framework, guiding principles, and approach to development. J Grad Med Educ. 2010;2(3):410–8.

Note

The Pediatrics Milestone Project aims to define the six competencies necessary for residents described by the Accreditation Council for Graduate Medical Education (ACGME) in the context of Pediatrics. In this paper, members of the ACGME and American Board of Pediatrics further refine these competencies and provide performance standards for these milestones. Each milestone and sub-competency was designed using a conceptual framework identified in the literature. Performance standards were designed with a focus on observable traits to better allow for assessment as well as ensuring that the developmental milestones provided took into account student abilities at varying stages of training, as determined by the literature. Overall, several themes were identified in the developmental ontology of the milestones, including that learners progress from being dependent learners to more independent learning styles, learners become more intrinsically motivated as their learning progresses, and mature learners have developed an understanding for the need to consult resources outside of themselves. 

Meade L, Caverzagie S, Swing SR, et al. Playing with curricular milestones in the educational sandbox: Q-sort results from an internal medicine educational collaborative. Acad Med. 2013;88(8):1142–8.

Note

To ensure that students are meeting the competency requirements defined by ACGME, clear, measurable milestones must be established to evaluate student progress. To develop a process to identify these milestones, the authors designed a Q sort game in which faculty participants could rank the milestones they considered most important for residents to have achieved before they were considered ready to advance in their training. This game involved either working individually or in groups of up to four members to sort 22 milestones into seven columns ranked by importance. Results were analyzed to determine the eight milestones that were generally considered most important. Six of these eight milestones had 92% agreement on their importance, and faculty reported that the game was fun and a valuable learning exercise, indicating that this Q-sort game method could be employed for future establishment of milestones. 

Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system--rationale and benefits. N Engl J Med. 2012(Mar 15);366(11):1051–6.

Note

This report from members of the Accreditation Council for Graduate Medical Education (ACGME) covers the development and introduction the Next Accreditation System (NAS), which began to be implemented in 2013. The NAS includes changes to the way residency programs will be evaluated, including introduction of annual evaluation and self-reports before site visits. Evaluation will focus more on outcomes and achievement of educational milestones, which are being developed by each residency specialty and fall within the six competencies already defined by the ACGME as vital for resident education.

Smith CS, Hill W, Francovich C, et al. Developing a cultural consensus analysis based on the internal medicine milestones (M-CCA). J Grad Med Educ. 2011;3(2):246–8.

Note

To evaluate competencies, internal medicine has arrived at 142 behavioral milestones. Evaluating each individual resident on all of these, however, is not practical, and so it is necessary to group milestones into assessment tools. Smith et al. used cultural consensus analysis to create a list of sixteen elements of competent performance that may cause operational problems in resident teaching clinics. Participants sorted these elements in order of importance to each participant, allowing for the analysis of whether each group shares the same values, since large differences in held values can result in operational problems. The authors propose that this method can be used to measure the knowledge of a group, the importance of certain competencies to a group, and how the importance of these competencies compares between groups, such as between residents and fellows. 


Goal Setting

George PM, Reis S, Dobson M, Nothnagle M. Using a learning coach to develop family medicine residents’ goal-setting and reflection skills. J Grad Med Educ. 2013;(June):289–293.

Note

Self-directed learning (SDL) is increasingly becoming recognized as essential to continuing, life-long education, but research has indicated that graduating residents lack confidence in their SDL skills. To this end, George et al. designed a coaching program in which residents participated in one hour, monthly individual sessions with learning coaches. These coaches, who were family medicine faculty, worked with each resident on SDL skills particularly in identifying learning needs and goals and strategies to achieve them. Results of the program were evaluated though the observations of the learning coaches, portfolio entries that the residents recorded during the process, and interview with the residents during the intervention year and year after the intervention. Residents' ability to set complex learning goals and reflect on their decisions and progress towards these goals improved over the course of their training, as did their ability to accomplish these goals independently. Residents found meetings with coaches to be motivating in setting goals and many found they were in the habit of goal setting after the training.

Li S-TT, Paterniti DA, Co JPT, West DC. Successful self-directed lifelong learning in medicine: a conceptual model derived from qualitative analysis of a national survey of pediatric residentsAcad Med. 2010;85(7):1229-1236.

Note

Although problem-based learning activities foster some self-directed learning (SDL) skills, little is known about how to further teach and encourage medical students and professionals to engage in SDL. Individual Learning Plans (ILPs) are now required from pediatric residency programs to document self-directed assessment and learning. The authors asked pediatric residents to describe their experiences with these ILPs and identify barriers to their learning goals and strategies they used or could use to overcome these barriers. These responses were then analyzed for recurrent themes, which were then grouped into broader themes. Barriers that residents reported encountering included difficulty with personal reflection, environmental strain, competing demands, difficulty with goal generation, and difficulty with plan development and implementation. Strategies for overcoming these included choosing specific, realistic, measurable goals with a given timeline, choosing goals that were important to development and success, and having a system in which residents were accountable for achieving these goals. These strategies were organized into a modification of the ISMART paradigm, which says goals should be important, specific, measurable, accountable, realistic, and have a timeline.

Lockspeiser TM, Schmitter P, Lane JL, Hanson JL, Rosenberg A, Park YS. Assessing residents’ written learning goals and goal writing skill: validity evidence for the learning goal scoring rubric. Acad Med. 2013;88(10):1558–63.

Note

Lockspeiser et al. sought to develop a rubric to evaluate written learning goals in medical education to allow for evaluation of self-directed lifelong learning (SDLL) skills, which have become one of the central components of accreditation requirements for medical fields. The creation of their rubric, based on the ISMART mnemonic, was described in a previous paper. The authors used questionnaires containing goal-writing prompts and learning goals from students who participated in individualized learning experiences from third-year pediatric residents. These questionnaires and goals were scored using the authors' Learning Goal Scoring Rubric. Using these scorings, the rubric was analyzed for content, response process, internal structure, and relationship to other variables. Results indicated that the rubric is valid, at least when used by at least two well-trained raters.


Environment

Artino Jr. A. It’s not all in your head: viewing graduate medical education through the lens of situated cognition. J Grad Med Educ. 2013;(June):177–179.

Note

Dr. Artino defines and compares situated cognition with more traditional information processing theories. He discusses how situated cognition can be used in graduate level medical education. Situated cognition is the idea that all learning takes place within a larger physical and social context. Situated cognition theorists argue that understanding and learning is about perceiving the environment and acting appropriately, rather than about remembering a certain action and performing it from recall.

Borbye L. Out of the Comfort Zone: New Ways to Teach, Learn, and Assess Essential Professional Skills. Morgan & Claypool Publishers; 2010

Lave J, Wenger E. Situated Learning: Legitimate Peripheral Participation. Cambridge: University of Cambridge Press; 1991.


Performance Observation

Berger I. A hierarchy of clinical observationsPediatr. 1980;65:357–358.

Note

Dr. Berger discusses the four levels of observation that should be used in patient encounters in order to provide through patient care. The first, and the one most commonly focused on, is observation of the physical and medically relevant characteristics of the patient, such as heart sounds, breathing rates, and rashes. However, Berger points out that there are three additional levels of observation, as follow, that can provide meaningful insight into patients' situations and therefore assist in improving care: the second level is personal characteristics of appearance and behavior, including how they are dressed and act; the third, interactions, describes how the patient interacts with family, the physician, and others; and the fourth, is insight into the physician’s feelings and behaviors to ensure objectivity in interpreting observations regarding the patient. These additional observations allow the physician to evaluate the patient and family's strengths and weaknesses, which can allow the physician to help address these and adjust medical care accordingly. The fourth level is additionally valuable in improving the mental health and professional enjoyment of the physicians and other medical staff. In combination, these observational approaches can promote better patient care.

Boudreau D, Cassell EJ, Fuks A. Preparing medical students to become skilled at clinical observationMed Teach. 2008;30: 857–862.

Note

Boudreau, Cassell and Fuks describe the development and effectiveness of a course they designed to teach first-year medical students clinical observation skills, addressing both the perception and interpretation of observations. To design the course, they worked with medical professionals, but also with experts in art, cinema, law enforcement and law. With these professionals, they concluded that, to be a skilled observer, one should be able to identify observable materials, describe one's observations of these, and finally communicate these observations to their patients, other medical professionals, and within the medical record and scientific literature. They also identified eight principles of observation, which are each described in detail within the manuscript. These include learning to identify how one's background affects observation and interpretation, learning to differentiate between observations and inferences, the importance of description in making observations concrete, and the importance of being aware of cultural variations in behaviors that we are observing. They build on Berger's (1980) work that divides observation into multiple levels, discussing the inner response of the observer him or herself. Overall, the goal of the course was to, through interactive learning modules, teach students to see the illness patients suffer from rather than only seeing the disease, allowing them to better take on the role of healer. A rigorous evaluation of the efficiency of the program is underway, but the program has been received enthusiastically by students.

Hauer KE, Holmboe ES, Kogan JR. Twelve tips for implementing tools for direct observation of medical trainees' clinical skills during patient encounterMed Teach. 2011;33(1):27-33.

Note

Although much of the training and learning during the course of medical school occurs in the clinic, students are rarely observed interacting with patients directly, and there are many possible tools that instructors can use to provide direct observational feedback, making it difficult to determine which methods are best. Hauer et al. suggest twelve tips for selecting such a tool in order to better evaluate students in clinical settings. These tips touch on how to integrate direct observation into a program, how frequently to use the tool, and how to ensure that faculty are properly using the tool.

Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: a systematic reviewJAMA. 2009;302(12):1316-26.

Note

Kogan, Holmboe, and Hauer aimed in this study to analyze the varying observation tools used to observe medical trainees' clinical skills. Eighty-five articles were identified that described observational tools designed for use by educational supervisors performing direct observation in clinical settings with actual patients (as opposed to simulated patients). From these 85 articles, 55 observational tools were identified and evaluated using the unitary theory of Messick, which uses validity evidence to determine the construct validity. Although they found that validity evidence and reports of improvement or changes in trainee skill or knowledge were rare, 11 tools were identified that have potential as useful observational tools that could be further studied.

Russell G, Ng A. Taking time to watch: observation and learning in family practiceCan Fam Physician. 2009;55(9): 948-950.

Note

Although adults learn best in direct observation situations, where they are observed by more experienced colleagues and receive direct feedback that they can immediately apply, direct observation is rarely used in many clinical teaching environments. Russell and Ng suggest that this is due to reluctance among teachers to engage in direct observation evaluations and a lack of support for the method from departments. In response, the authors provide steps that can be used to integrate direct observation into clinical education, including ensuring the patient and learner know what to expect and recording observations. They also provide a discussion of the merits and pitfalls of three ways of observing the students: being in the room with the learner, viewing outside of the room through a two-way mirror or closed circuit camera, or using a portable video camera. Key elements of direct observation are also described to help guide clinicians with how to approach using this method with students.

Shapiro J, Rucker L, Beck J. Training the clinical eye and mind: using the arts to develop medical students' observational and pattern recognition skillsMed Educ. 2006;40: 263–268.

Note

Clinical observation is the first step in the medical decision-making process. Observing, though, is not an objective process, and so training clinicians in observation skills could improve clinical skills. To determine the value of using art-based observation instruction to improve clinicians' observational skills, Shapiro, Rucker, and Beck divided students into three groups who received training in the form of photographs and cases, art, or mixed media art and dance. Representational art was used to teach identification of figures and non-representational art was used to encourage students to look for patterns. Less-accessible artwork was used to necessitate observational skills and art with hidden meanings was used to encourage reflection. Success of the program was evaluated using post-intervention group interview comments, written feedback, participant observations, and instructor debriefings. The format and central questions that were discussed, along with student responses to the art and questions, are described in detail. Concepts that were focused on in the arts-based groups included seeing beyond the expected, pattern recognition, awareness of feelings of uneasiness, attention to relationships, and questioning assumptions. Students in both the clinical and art-based sessions reported improvement in observational skills, pattern recognition, and in developing a coherent method of analysis. Students in the clinically-based training reported greater improvement in conveying multiple dimensions of pattern recognition than did those students in the art-based training, but the students in the art-based training reported increased empathy with their patients and an ability to approach observation and pattern recognition from a metaphorical and symbolic angle.


Feedback

Cote L, Bordage G. Content and conceptual frameworks of preceptor feedback related to residents’ educational needs. Acad Med. 2012;87(9):1274–81.

Note

Although there has been much discussion and research on how to provide feedback to residents in preceptor situations, there is little discussion of what exactly this feedback should contain. Often, feedback focuses on patient care, such as diagnosis and management, rather than on the education needs of the resident, such as managing time and relevant reading materials. To determine how clinicians would provide feedback that addressed residents' educational needs, the authors provided 25 clinicians with a series of six vignettes that described junior residents who required feedback of a nature related to their educational needs. Each clinician was interviewed about how he or she would respond to the resident in each vignette and to identify the basis for these responses, particularly any conceptual frameworks that may have contributed. Those preceptors with a greater number of identified conceptual frameworks were found to provide a greater variety of types of responses. More senior preceptors were found to give more advice, ask more probing questions, and ask for more clarifications and also expressed more conceptual frameworks than did the more junior preceptors (those with 13 or fewer years of experience).

Fanning RM, Gaba D. The role of debriefing in simulation-based learning. Simul Healthc. 2007;2(2):115–125.

Note

Although feedback is an essential component of simulation-based medical education, there is little research on the best methods for debriefing students after such an experience. To remedy this, Fanning and Gaba have compiled a review of literature with the aim of identifying the best debriefing practices and where additional research is required. In doing so, they provide a background of simulation-based learning, including the rationale for using this method with adult learners and the importance of reflection in the learning process. Debriefing is identified as essential to this reflection process in that it directs the reflection toward the appropriate level of criticism for the student's abilities and prevents focus on areas that could be damaging for the student's overall self-worth. The common structural elements of debriefing included the impact the experience had on the participants, recollection of the event, and reporting of the event. Facilitators are important in ensuring that participants move through these structural elements of the process in a timely and efficient fashion that is appropriate for the original objectives of the exercise. The value of different methods of debriefing, such as writing a summary of the experience and filming the experience, are discussed and future research directions are identified.

Kusurkar R, ten Cate O. Education is not filling a bucket, but lighting a fire: self-determination theory and motivation in medical students: AM Last Page. Acad Med. 2013;88(6):2013

Note

This infographic defines self-determination theory, the idea that humans are motivated to learn and such motivation is central to learning outcomes. It covers the difference between intrinsic and extrinsic motivation with diagrams to illustrate these concepts.

Nicol DJ, Macfarlane‐Dick D. Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Stud High Educ. 2006;31(2):199–218.

Note

To foster self-directed learning (SDL), feedback from preceptors needs to not just be a transmission of information from teacher to student but also to engage and involve the student. Students also require assistance in understanding the feedback and incorporating this feedback into their learning goals. The authors therefore present a conceptual model of and research in support of formative assessment and feedback in which students are central to the development of their own feedback. They describe how this feedback should be incorporated into models of self-directed learning and provide seven principles of good feedback with evidence supporting each.

Quirk ME. How to Learn and Teach in Medical School: a Learner-Centered Approach. New York: Charles C. Thomas Publishers; 1994.

Rudolph JW, Simon R, Raemer DB, Eppich WJ. Debriefing as formative assessment: closing performance gaps in medical education. Acad Emerg Med. 2008;15(11):1010–6.

Note

Formative assessment allows for individualized feedback, which is essential to self-regulated learning. The authors propose that debriefing, which involves a conversation between the instructor and student regarding an event, can be used to provide the necessary formative assessment for teaching residents in simulation-based education. A description of formative assessment and debriefing are provided, along with the steps for using debriefing and the components of an effective debriefing. These include a reactions phase, an analysis phase, and a summary phase.


Evaluating Competency

Boyatzis RE. The creation of the emotional and social competency inventory (ESCI). Hay Group. 2013:11.

Note

The Emotional Competence Inventory (ECI) is a measure of emotional competency used to predict effectiveness various management, leadership, and professional roles. Although acceptably reliable for industry standards, the test is difficult to administer accurately for non-trained informants. Therefore, Boyatzis endeavored to create a more accurate ECI, resulting in the ESCI, which better measures those behaviors that contribute to effective performance in work environments. Developmental levels were removed by rewording items within each scale, allowing for a more behaviorally focused model and resulting in a more psychometrically rigorous measure. The revision has resulted in a test of greater statistical rigor that is more appropriate for coaching conversations and better accounts for the complexity of behaviors. This document covers the changes to the ECI and provides the results of a pilot study of the ECSI to establish the validity of the new measure.

Carraccio C, Englander R. From Flexner to competencies: reflections on a decade and the journey ahead. Acad Med. 2013;88(8):1067–73.

Note

The authors offer a follow-up of their 2002 article about the enactment of then-new, competency-based medical education (CBME). They discuss progress to date in implementing this new approach and factors influencing its implementation and highlight some of the lessons learned during the implementation process. Among these, they discuss the importance of direct observation, meaningful measures of performance, and the ability to use the role of learners in learning to improve education. The article concludes with a discussion of the recent advances in the CBME approach, including the Milestone Project and Entrustable Professional Activities.

Swing SR. The ACGME Outcome Project: retrospective and prospective. Med Teach. 2007;29(7):648–54.

Note

This article describes the development process of the Accreditation Council for Graduate Medical Education's (ACGME) six competencies, assessment tools, and accreditation requirements. The initial response of many medical educators to these competencies indicated that educators needed assistance in writing clear goals and objectives and developing activities, assessments, and standards to ensure residents meet these competency requirements. The ACGME has been working to provide these resources, and since the introduction of the competencies, they have become the common framework for defining physician competence, educational development activities have increased and become more focused on supporting teaching and assessment of the competencies, and residency programs have been adjusted to incorporate learning objectives, performance feedback, and faculty development related to the competencies. This paper also reviews evidence that this new approach has improved patient care and the steps required to acquire more robust evidence.

Yudkowsky R, Downing S, Klamen D, et al. Assessing the head-to-toe physical examination skills of medical students. Med Teach. 2004;26(4):415-419.


Self-Regulated Learning

Artino AR, Jones KD. Self-Regulated Learning - A dynamic, cyclical perspective: AM Last Page. Acad Med. 2013;88(7):428639.

Note

This diagram describes Zimmerman's three-phase cyclical model for self-regulated learning (SRL). It depicts the components of these three phases of SRL - forethought, performance, and self-reflection. A table of three SRL assessment methods - questionnaires, diaries, and microanalytic protocols - is included, along with pros, cons, and sample items for each method.

Brydges R, Butler D. A reflective analysis of medical education research on self-regulation in learning and practice. Med Educ. 2012;46(1):71–9.

Note

Brydges and Butler present an analysis of medical education research on self-regulated learning (SRL) in medical students, residents, and clinicians. They begin by discussing the role and importance of SRL throughout medical education and into clinical practice. Discussion of SRL is organized around a framework of SRL taken from education research, which is diagramed and explained in the paper. The review of the current literature covers the effect of context, environment, an individual's prior experiences and beliefs affect SRL and how SRL actually occurs. Finally, they discuss how to apply their findings to medical education, the benefits of using the framework to explore SRL research, and the overall conclusions their review indicated.

Cleary TJ, Callan GL, Zimmerman BJ. Assessing self-regulation as a cyclical, context-specific phenomenon: overview and analysis of SRL microanalytic protocols. Educ Res Int. 2012;2012:1–19.

Note

Evaluation of self-regulation skills is usually done in a self-reporting manner, which does not well-measure the actual thoughts or processes of individuals engaged in self-regulated learning. Self-Regulated Learning microanalysis instead involves microanalytic questions aimed at evaluating self-regulation sub-processes including goal-setting, strategic planning, monitoring, self-evaluation, and attributions. This paper includes an extensive discussion of the theory of self-regulated learning and existing forms of assessment of self-regulation, including self-report measures and structured interviews. The self-regulated learning microanalytic assessment, although a form of structured interview, is unique in that it allows for the analysis of regulatory processes before, during and after a task or activity, therefore avoiding retrospective or prospective reports. The process of conducting such an analysis how such measures have been used are also described in detail, concluding with a discussion of the validity of these measures.

Durning S, Cleary T, Sandars J, Hemmer P, Kokotailo P, Artino AR. Perspective: viewing “strugglers” through a different lens: how a self-regulated learning perspective can help medical educators with assessment and remediation. Acad Med. 2011;86(4):488–95.

Note

Students who underperform in medical school or residency require a greater input of resources from their school, including increased time from clinicians, tutors, and other educators. The ability to identify these students and evaluate their self-regulated learning abilities could ease their transition into clinical education. In this paper, the authors propose a assessment framework, the Self-Regulated Learning-Microanalytic Assessment and Training (SRL-MAT) that could be used to identify these struggling students and determine the cause of their poor performance. They propose that this measure could be used to ask whether self-regulation assessment and intervention activities can be used to remediate struggling students. As a microanalytic assay, it will identify thought processes as they are being used, accurately analyzing self-regulation abilities.

Kusurkar R, Croiset G, Mann KV, Custers E, Ten Cate O. Have motivation theories guided the development and reform of medical education curricula? A review of the literature. Acad Med. 2012;87(6):735–43.

Note

Educational psychology identifies three dimensions of learning processes - cognitive, affective, and metacognitive - all three of which should be accounted for when designing curricula. However, the cognitive processes, particularly "what to learn", have long been the focus in medical schools. The introduction of Problem-Based Learning methods has added a focus on metacognitive regulation, or "how to learn" and "why learn". This "why learn" component can be a large motivating factor, and the authors propose that motivation to learn may be a large determinant of student outcomes. Therefore, the authors performed a literature review to identify theories regarding what motivates students to learn, as well as articles on curricular reforms in order to define what guided those reforms, particularly how large a role motivation theory played in the reform. The paper covers historic understandings of motivating factors through present day theories and provides a brief summary of developments in medical education curricula since the 1980s. They found that many of the changes in curricula have allowed for a greater focus on the metacognitive regulation component of learning, but that student motivation has not been a significant factor in the curriculum redesigns.

Schunk DH, Zimmerman BJ, Eds. Self-Regulated Learning: From Teaching to Self-Reflective Practice. New York: Guilford Press: 1998.


Learner Development

Colvin G. Talent is Overrated: What Really Separates World-Class Performers from Everybody Else. New York: Penguin; 2008.

Ericsson KA. Development of Professional Expertise: Toward Measurement of Expert Performance and Design of Optimal Learning Environments. Cambridge: Cambridge University Press; 2009.

Pashler H, Mcdaniel M, Rohrer D, Bjork R. Learning styles: concepts and evidence. J Assoc Psychol Sci. 2009;9(3):105–119.

Note

Although the concept of learning styles is commonly discussed and taught to psychology students and teachers, it has not been established whether teaching in different learning styles actually generates the desired outcome of improved learning. This article discusses commonly used measures of learning styles, the history and appeal of the concept, and then the literature for and against the importance of learning styles in outcomes. Although study preferences do exist, the mere existence of such preferences does not presume a need for different learning styles. Proponents of learning-style-based approaches prescribe to the "meshing hypothesis" that the format of education should match the learner's style. Evidence for a specific aptitude, however, is not the same as evidence for learning styles. To prove the value of a learning-styles based approach, studies would need to demonstrate a crossover interaction between learning style and examination outcome. The authors, however, found only one study that meets these criteria, and this study did not meet the methodological standards the authors determined necessary. In contrast, several studies of greater methodological rigor found that learning styles did not affect testing outcome. The authors also discuss literature supporting an aptitude-by-treatment model of education to provide an example of the sorts of evidence and interaction required to support a learning style model.

Schumacher DJ, Englander R, Carraccio C. Developing the master learner: applying learning theory to the learner, the teacher, and the learning environment. Acad Med. 2013;88(11):1635–45.

Note

The shift toward competency-based education necessitates a focus on developing life-long learning skills to allow for the development of master learners. In this article, Schumacher, Englander, and Carraccio describe the three components of self-determination theory - a sense of relatedness, autonomy, and competence - and how to foster these in students to encourage motivation to learn. They then discuss factors that affect ability to learn, namely cognitive load theory and situated cognition. Finally, they discuss the potential pitfalls of self-assessment, including self-concept, self-efficacy, illusory superiority, and gap filling. Methods for avoiding these pitfalls are also provided, including reflection, self-monitoring, and external information and self-directed assessment seeking. They conclude with overall suggestions for learners, teachers, and those impacting the learning environment in facilitating the creation of master learners.

Stuart E, Sectish TC, Huffman LC. Are residents ready for self-directed learning? A pilot program of individualized learning plans in continuity clinic. Ambul Pediatr. 2005(Sep-Oct);5(5):298-301.

Note

Lifelong learning, which is included in two of the Accreditation Committee for Graduate Medical Education competencies, requires the development of self-directed learning (SDL) skills. SDL includes the ability to identify learning goals and needs, to find and use resources to reach these goals, and to evaluate whether the goals have been accomplished. Individualized learning plans (ILPs) are one proposed method for teaching these skills to residents. In an ILP, the teacher and learner work together to define goals, resources, strategies, and evaluative measures. This paper describes the implementation of an ILP program in a pediatric continuity clinic and evaluates residents' and faculty members' responses to the program. Faculty members were trained in a two-hour workshop, and the program was evaluated with a learning plan program questionnaire. The program was identified as being helpful in providing a framework for learning and increasing awareness of the learning process, but obstacles to it's successful use included difficulty establishing goals, tiredness, passivity, and lack of motivation. Recommendations for those interested in implementing similar educational interventions include skill building in SDL, offering external support, dedicated time for ILP development and discussion.


Faculty Development

Ballou R, Bowers D, Boyatzis RE, Kolb D. Fellowship in lifelong learning: an executive development program for advanced professionals. J Manag Educ. 1999;23(4):338–354.

Note

Ballou, Bowers, Boyatzis and Kolb discuss their implementation of the Professional Fellows Program (PFP), an education program designed specifically for adult learners at the Weatherhead School of Management at Case Western Reserve University. This program, aimed at midcareer professionals interested in pursuing management positions, focused on learner-directed education in which the participants take responsibility for their learning. To eliminate the potentially adversarial relationship between teacher and student, the program did not use grades. To assist the students in guiding their own learning, they take a course focused on designing a personal Learning Plan, during which they diagnose their strengths and weaknesses and evaluate how to improve. The other courses in the program focus on using materials selected for their current relevancy and on employing instructors as facilitators and coaches in learning rather than as lecturers. To evaluate the effectiveness of the program, the 53 members of the first four classes of the PFP were interviewed and surveyed. Participants reported greater self-confidence and that they found the most useful component of the program to be the creation of the learning plan.

Khandelwal S, Bernard AW, Wald DA, et al. Developing and assessing initiatives designed to improve clinical teaching performance. Acad Emerg Med. 2012(Dec);19(12):1350-3.

Note

In the past decade, faculty development initiatives have begun to focus on improving teaching skills of academic physicians. The authors worked with a group of Emergency Medicine physicians and educators in a consensus conference format to provide a list of attributes of expert teachers and faculty development programs that are designed to improve teaching performance. Participants surveyed the literature on faculty development initiatives, finding that there is wide array of formats and opportunities available. The Best Evidence in Medical Education Collaborative assessment of faculty development initiatives was identified as the most comprehensive source for reviews of such programs. Finally, recommendations for further study were identified, which included clearer definitions of the core skill set required for expert teaching and identifying the best methods to deliver education training to faculty members.

Leslie K, Baker L, Egan-Lee E, Esdaile M, Reeves S. Advancing faculty development in medical education: a systematic review. Acad Med. 2013;88(7):1038-45.

Note

The goal of this paper is to synthesize the growing body of literature on faculty development in teaching and mentorship, particularly focusing on providing an understanding of the range of faculty development activities that have been employed, assess the effectiveness of these activities, and identify directions for future research. The most common structure for FD programs was a series or workshops or a longitudinal program with a length ranging from 10 days to 2 years. Most were aimed at individual learners rather than educating teams and mostly focused on improving teaching effectiveness and scholarship among other goals. The most common data collection method was a survey, but analysis teaching scores, student marks, and progress reports. Outcomes that were reported included participant reactions to the program, attitudes and perceptions toward teaching and their own abilities, knowledge and skills, behavior changes as measured by the types of education participants provided after the program, changes in organization, and benefits to students and patients.

McLean M, Cilliers F, Van Wyk JM. Faculty development: yesterday, today and tomorrow. Med Teach. 2008;20(6):555-84.

Note

This extensive review of the role of faculty development(FD) in medical education begins with a discussion of the evolution of faculty development since it began to be focused on improving the teaching skills of educators. This guide designed to assist in the planning and implementation of FD programs with the aim of improving patient and community care through FD. It describes the effectiveness of various interventions and defines principles that contribute to this effectiveness to help readers determine which would best suit their institution. Finally, it details characteristics of effective and sustainable FD and provides a step-by-step instruction guide for establishing such a program.

Shortt SED, Guillemette J-M, Duncan AM, Kirby F. Defining quality criteria for online continuing medical education modules using modified nominal group technique. J Contin Educ Health Prof. 2010;30(4):246–250.

Note

Opportunities for online continuing medical education have been increasing rapidly, but there is little consensus on what qualities a successful online CME program possesses. To define such qualities, Shortt et al. surveyed the literature to identify components that have been proposed as essential for quality online CME programs. These components were compiled into a list that was sent to experts in CME and to a panel of clinical educators, content experts, and representatives of medical education organizations. These participants ranked the qualities based on importance and rankings were analyzed using Modified Nominal Group Technique, which allows for evaluation of the numerical ranking as well as the degree of consensus regarding the ranking. The experts arrived at a consensus on all items, identifying ten as essential and seven as necessary but not essential. The panel did not arrive at a consensus on several items, but the items that they considered important were mostly the same as those the experts identified with the exception of impact assessment. The authors concluded that the following qualities were necessary for a quality online CME module: it would be needs-based, clinical format presentation, use evidence-based information, allow for interaction with content and experts, document practice change, be accessible for review, and include a course evaluation.

Srinivasan M, Li S-TT, Meyers FJ, et al. “Teaching as a competency”: competencies for medical educators. Acad Med. 2011;86(10):1211–20.

Note

The goal of this paper was to design a framework for identifying medical educator skills through review of literature and expert opinion in the format of a conference, regional and national presentations, and individual discussions. Participants in the discussions felt that the framework should address the following five questions: does every person who teaches need educational training? are there foundational principles in medical education, which skills are core versus specialized, which terms best express the continuum of educator skills development, and should we assess teaching or learning? The responses to these questions were used to sort educator attributes into ten medical educator competencies, using the ACGME competency framework. Four specialized competencies for teaching were identified: program design and implementation, evaluation and scholarship, leadership, and mentorship. In addition to defining these, ACGME competencies were redefined to be appropriate for a teaching role, especially given that clinicians should already meet the ACGME competency guidelines. This conceptual framework can be used to direct allocation of CME resources and identify programs' needs.


Resident-as-Teacher

Fromme BH, Whicker SA, Paik S, Konopasek L, Koestler J, Greenberg L. Pediatric resident-as-teacher curricula: a national survey of existing programs and future needsJ Grad Med Educ. 2011;3(2):168-175.

Note

Although over 50 percent of US residency programs have some form of resident-as-teacher (RAT) training program, there is little literature on the elements of successful programs nor the format, content, and timing of these programs. To remedy this, Fromme et al. surveyed the program directors of US pediatric residency programs to define the structure and content of existing RAT curricula and determine the need for national curricular resources for RAT pediatric programs. Results included the types faculty involved in RAT programs, the educational background of these faculty members, how many programs require resident participation and for how many it is optional, how each school evaluates its program, and the frequency of use of three forms of skills practice: role play; direct observation; and objective, standardized learner experience. For schools without an RAT program, reasons for not having the program were explored, and included lack of resources, low priority, lack of resident interest, and preference for direct faculty feedback. Participants identified a resource website, workshops, and electronic handbooks as what would be the most helpful national resources to help them improve their programs. These may help to reduce the variability of RAT curricula, allowing for more consistent objective achievement, and help schools without the resources to establish such a program do so.

Hill AG, Yu TC, Barrow M, Hattie J. A systematic review of resident-as-teacher programsMed Educ. 2009;43: 1129–1140.

Note

Hill et al. seek to identify the needed directions for future research on Resident-as-Teacher programs and identify features that make such programs successful to allow for issues facing established programs to be avoided by future programs. This involved a review of 29 papers that described and evaluated the effectiveness of RAT programs. Data extracted from these papers included the types of residency programs that sponsored the programs, the type of program (such as workshop, seminar, courses, etc.), the instructional method used (lectures, videos, case-discussions, etc.), and the length and frequency of each program. The authors also examined how each program chose to assess their RAT methods, including type of assessment (meaning surveys, interviews, OSCEs, etc.) and who did the assessing (medical students, residents, preceptors, etc.). Although the studies were an extremely heterogeneous mix of all of these factors, the majority found that RAT programs improved residents' teaching abilities.

Post RE, Quattlebaum RG, Benich JJ III. Residents-as-teachers curricula: a critical reviewAcad Med. 2009;84:374-380.

Note

Resident as teacher programs have been found beneficial for residents on many levels, including improving teaching skills, attitudes and perceptions toward teaching, and improved observable behaviors. The impact of these programs on the medical students being taught by residents, however, is not well understood. To evaluate this, residents at the University of Auckland in New Zealand participated in a two-day, teaching skills workshop. Medical students taught by these residents were then surveyed regarding the effectiveness of the residents' teaching, and their ratings were compared with residents from hospitals where the teaching course was not provided. The medical students' OSCE scores were also compared. Although students rated the teachers whom had undergone training as more skilled, OSCE scores did not vary between the two groups, indicating that these programs may not directly affect student learning.