JIT home pageThe Just In Time Medicine (JIT) system is a crucial component of the College of Human Medicine’s competency-based Shared Discovery Curriculum (SDC). This resource functions as the central point for curricular content and assessment. The entire SDC curriculum is accessible via JIT. The vast majority of content is publicly available, making it very convenient for prospective students who would like to learn more about the curriculum. In addition to CHM students, JIT is used by people all over the world.

JIT acts as a portal, making internal content and material from other sources - such as electronic textbooks, YouTube, and Google Drive - more readily available. JIT is a constantly evolving resource; much like Wikipedia, the content on JIT has been created by many authors and is frequently updated.  Having a responsive design allows JIT to work well across many different device types including desktops, laptops, tablets, and mobile phones. JIT makes the SDC curriculum available anywhere.

JIT content page

In addition to content, JIT manages all the assessments that are used in the SDC curriculum. JIT functions as an assessment aggregator, combining data from the multitude of assessments that a student completes each semester. The combined information is presented to students and faculty on the JIT Dashboard. The dashboard quickly shows how each learner is progressing according to the SCRIPT taxonomy (Service, Care of Patients, Rationality, Integration, Professionalism and Transformation). The dashboard helps students easily track their own progress in the curriculum. In addition to high-level information, users can dig deeper and evaluate data that has been imported from other systems, including NBME, Examsoft, New Innovations, and Desire2Learn (D2L). Students can also submit additional educational artifacts to their personal portfolio. JIT has become an essential tool for the Student Competence Committee (SCC) to review the progress of each student at the end of the semester. It provides a holistic view of each learner and facilitates the identification of any academic or professionalism issues.

JIT Student Progress Report page

JIT continues to mature and expand its capabilities. New features are added nearly every semester. The system is supported by a team of dedicated professionals, called the JIT Team. Since JIT was developed internally, we have the flexibility to mold it to fit our specific needs. Feedback from faculty, staff, and especially students is vital as the first few years of the new Shared Discovery Curriculum are developed and delivered. To facilitate feedback, the “JIT Squad” was created to give students an opportunity to share their thoughts and ideas, as well as pilot test new features. Each semester, the JIT Team meets with about a dozen ECE and MCE students to discuss how JIT can be improved. The JIT Squad has already been very valuable for learning how to improve the student experience. As the SDC continues to take form, JIT will play an increasingly important role in content delivery and assessment.

Spectrum Health/CHM logo

Several OMERAD and CHM faculty have been working in partnership with Spectrum Health to create and deliver a series of interactive two-day faculty development programs in Grand Rapids for Spectrum Health physicians and advanced practice providers, led by Spectrum Health’s Lori Schuh, MD, who is Vice President for Academic Affairs. Dr. Monica van de Ridder, an Assistant Professor in OMERAD, has been working closely with several physicians to co-create curriculum for this two-day program. In addition, Dr. van de Ridder leads four courses, on assessing performance by direct observation, giving and receiving feedback, difficult feedback conversations, and bedside procedural teaching.

Needs assessments collected from Spectrum physicians, residents, and MSU CHM medical students revealed that there was a great need for more ways to effectively assess learner performance during rotations, learn how to give and receive feedback, teach providers to handle difficult feedback, increase their capability to teach in time compressed settings and teach clinical procedures. The clinical faculty and OMERAD/CHM faculty set out to design series of courses to meet these needs of the clinical faculty.

Dr van de Ridder interacting with faculty memberThis Spectrum Health/MSU faculty development is a two-day course. First, each faculty development leader introduces major teaching points about each topic, and then clinicians participate in an Objective Structured Teaching Examination (OSTE) as a way to apply their learning. In the OSTE, faculty development providers assess the clinicians’ performance in teaching a procedure, performing a learner assessment, and providing feedback to a learner. Clinicians are also able to view the video and decide how to improve their skills as educators. One key feature of the course that Dr. van de Ridder leads is developing an individual learning plan in which clinicians formulate goals they would like to work on to improve. For example, if a clinician noted a specific behavior in the OSTE video recording, they can set a goal for a targeted way to improve. In this way, the course integrates reflective active learning strategies to supplement faculty development and participant learning. OMERAD’s Dr. Stacey Pylman and Angie Thompson-Busch, MD also teach a course on Coaching, Advising and Mentoring.

Says OMERAD’s Dr. van de Ridder, “We make the course highly interactive and active learning is a big part. You can’t teach these skills without practicing and that is why OSTE is important.”

After the course is over, the clinicians will participate in the OSTE component again, 4-6 months later. This refresher course can help cement the skills taught previously. As van de Ridder reflects, “Sometimes when you are taught something it slips away and it is helpful to repeat what you are taught. While there is no evidence regarding an ideal number of these refresher courses [to cement the topic], it is helpful to practice what you learn and start applying these skills right away to get the most out of the course.” This is a key tenant of active learning: application of new knowledge and skills.

Says Dr. Schuh, “About 40 physicians and advanced practice providers have participated in the program this academic year, and the feedback we’ve received has been extremely positive.” With this two-day course sequence, Dr. Van de Ridder, OMERAD, and Spectrum Health faculty development leaders provide participants the opportunity to experience active learning strategies first hand and use these new skills to grow as clinical educators. Participants are also growing as a faculty community. Dr. Schuh explains, “We are recruiting new faculty development providers from those who have completed the course. An unplanned benefit of the program is the interaction between physicians and advanced practice providers, and those from different specialties.”

Think aloud group activityIn our first teaching tip, Dr. Joel Maurer, CHM Assistant Dean for Admissions and an Academy Fellow who teaches small groups in the Shared Discovery Curriculum, uses a THINK ALOUD to help medical students ‘eavesdrop on his thinking’. When an expert makes his/her thinking visible, the expert helps students understand a complex process, such as clinical reasoning to reach a diagnosis. Dr. Anne-Lise Halvorsen, an Associate Professor in Teacher Education at MSU, describes the features of a think aloud.

As argued by Delaney and Golding in their 2014 article which appeared in BMC Medical Education, “the making thinking visible approach has potential to assist educators to become more reflective about their clinical reasoning teaching and acts as a scaffold to assist them to articulate their own expert reasoning and for students to access and use” (p.1).

 

 

 

Additional Resources:

Extended Footage: Watch an extended version of the Think Aloud to see how Dr. Maurer planned and enacted his Think Aloud

Using a Think Aloud Handout

The mission of the new Shared Discovery Curriculum (SDC) is to nurture, educate and graduate students who are ready, willing and able to be exemplary new residents and practicing physicians. The vision for this new curriculum puts real and simulated patients at the center of the educational enterprise, and emphasizing:

  • Early and ongoing clinical experience for students,
  • Integrated basic and clinical sciences throughout the curriculum,
  • Collaborative learning for faculty and students,
  • Alignment of evaluations with curricular content and real-world performance,
  • Assuring competence and striving for excellence.

The SDC is an experience-based curriculum. A week in the life of a first year (Early Clinical Experience) student is made up of a large group experience, two learning society scholar group meetings, two half-days in an outpatient clinic setting, a half-day in a simulation center and of course, anatomy and histology labs. For second year students, the Middle Clinical Experience has the same basic structure except that clinical experiences involve a variety of ambulatory and inpatient rotations organized to emphasize the integration of clinical work and the basic and social sciences underlying patient care. There is only one learning society small group per week to accommodate a rotational small group focused on the students’ clinical rotations.

The first class of SDC students matriculated in 2016; these students are now in the middle of their second year of medical school.  What have we learned so far?

Student Perceptions

At the end of their first year, the SDC students were more satisfied with the quality of student-faculty interactions compared to legacy students (3.9 vs 3.6) and found their learning experiences to be more meaningful (3.8 vs 3.1).  Otherwise their ratings of the CHM learning environment largely were the same as students in the legacy curriculum. 

Based on data from the mid-semester evaluation, most students agreed or strongly agreed that they felt supported by faculty (86%) and peers (84%) within their learning society scholar groups. Similarly, students reported that the rotational small groups related to their clinical experiences worked effectively together (84%). Interestingly, while students agreed or strongly agreed (69%) that the simulation sessions helped them learn course content, a smaller proportion of students indicated enjoying their simulation sessions (39%). A more complete summary of results available here.

Students were asked to indicate the extent to which each component of their educational experience contributed to their learning of course content. A summary of their ratings is presented in the chart below.  Students’ evaluation of the learning experiences varied widely. For six of the 10 experiences rated, over half of the students agreed or strongly agreed that the experience contributed to their learning. Based on the feedback from this mid-semester evaluation, changes were made to the Large Group Activities to enhance their value to students.

Chart 01

 

Student Performance

Assessment in the SDC is based on a progress testing model, which is a comprehensive assessment of knowledge and skills twice each semester. The assessment is a graduation test for the MD degree, measuring the entire body of knowledge that a student should master by the end of medical school. Rather than aiming for mastery of a small amount of knowledge, progress testing assesses incremental improvement in student performance over an extended period of time. A recent paper provides a review of the educational literature related to progress testing. 

For the knowledge component of our progress test we use the Comprehensive Basic Science Examination from the National Board of Medical Examiners (NBME), supplemented with NBME  customized assessment exams. The graph below shows the knowledge gain by content area for current MCE students (2016 matriculants) compared to the ECE students (2017 matriculants) based on data from the customized assessment tests. As expected, the second year students outperform the first year students in all science content areas.

Chart 02 

 

The NBME customized exams provide specific feedback by content area whereas the NBME Comprehensive Basic Science Exam (CBSE) is an indicator of USMLE Step 1 readiness. The graph below shows the progression of CBSE scores for the first SDC student cohort over three test administrations. From September to February there has been a 10-point increase in the mean score.

 Chart 03

 

The graph below compares the CBSE score distributions of legacy and MCE students tested in fall semester of their second year. The distribution of the SDC students (in blue) shows higher scores than the legacy students (in red) at the same point in their education, with the overlap represented in purple. These data suggest that we would expect higher Step 1 pass rate as a result of the new curriculum.

Chart 04

 

Overall, the early returns on students’ opinions and performance suggest that our efforts to enhance a collaborative learning environment for students and faculty have been successful. The curricular structure integrating basic, social and clinical science in the context of patient complaints and concerns challenges students. Meaningful clinical experiences provide students a context for learning. While the SDC implementation has not been without its challenges, these data provide evidence of our success to create a more supportive and effective medical school experience.

Page 1 of 2