Jan Carline is admired by his colleagues for his knowledge of educational theory and practice, and his generosity and kindness.
Add to this: his strong opinions, tenacity and occasional irreverence. “Jan does his work behind the scenes,” said Dr. Suzanne Allen, vice dean for academic, rural and regional affairs.“ That work is the underpinning that makes us successful.”
As director of educational evaluation and professor of biomedical informatics and medical education at the UW School of Medicine, Dr. Carline was especially influential in developing the program objectives for the new curriculum that began in 2015 (see Sidebar).
He “worked for years to change the culture around UW Medicine,” said Dr. Michael Ryan, associate dean for curriculum. “He believed that every course and clerkship should be evaluated on the basis of the curriculum’s overall objectives and goals. Once all assessments are adequately mapped, we can make curricular changes based on outcomes.”
He has been “enormously influential in setting the standard for student assessment and program evaluation in the new medical school curriculum,” added Dr. Marj Wenrich, associate dean for education strategies.
After 41 years at the UW School of Medicine, Carline retired in January. In his honor, UW Medicine has established the Jan D. Carline Award of Excellence in Scholarship of Education, which will be awarded to one medical student each year (see Sidebar).
Carline has published more than 80 articles in peer-reviewed journals, and he served as co-director of the Teaching Scholars Program for the past 10 years. He has worked with three medical schools in China and made about 18 trips to Saudi Arabia for faculty training and workshops on assessment and evaluation. He is an associate editor of Academic Medicine, the journal of the Association of American Medical Colleges and one of the top medical education journals in the world.
In a recent conversation, Carline talked about his career and future plans. While he will continue to work on special projects as professor emeritus, he looks forward to spending less time in committee meetings and having more time to pursue his personal interests.
What attracted you to your field?
I fell into it. While I was working on a Ph.D. in educational psychology, I took a job as a graduate student research assistant in medical education. I found that my training in education and statistics was a good fit for assessment and evaluation. When I finished my Ph.D., there were very few jobs in counseling with children and parents, my original career goal. I was offered a faculty position in what was called the Office of Research in Medical Education, and I took it. I’ve found that the skills and attitudes I learned as an educational counselor have supported my work in evaluation and teaching.
How are students assessed in the new curriculum?
Testing is a huge part of the students’ life and critical to their success. This includes written tests that assess their clinical knowledge and performance tests for their clinical skills with patients.
In my role as director of evaluation, I had a team of six to eight people. For the Foundations courses, we developed and administered a computerized test at least every two weeks for all six test sites in the WWAMI region. We also administered national examinations for clerkships.
How do you make sure that a test is fair?
The standard nowadays is to assess students’ ability to apply knowledge to solve a problem rather than their ability to memorize facts.
After every test, we hold teleconferences to review the results with the course directors. We want to determine if an item is too easy or too difficult. We also look at differences among sites to see if there are any unexpected variables. Based on this review, we will sometimes change how items are graded and then publish the results to the students.
What are the goals for program evaluation?
Our evaluation program has two principal functions. First, we want to find out whether the courses and clerkships are meeting their objectives. Second, we need to help the students understand how well they are doing.
How successful is the new curriculum?
It is still evolving. The Foundations curriculum is in pretty good shape and is preparing students well for their required clerkships.
The original plan was to transform the clerkships from being discipline-specific to being focused on how medicine is performed in various settings such as a hospital, surgical unit or community. Right now, this has not happened, and it has dampened our ability to determine whether there are threads of content that should be taught across all the clerkships.
One other issue that may not be directly related to the new curriculum is that students have become increasingly anxious about their performance on the U.S. licensing examinations over the past five years. This may be a function of the increased competition to get into residency programs, but it is detrimental to their learning. We are still trying to figure out how to deal with this anxiety.
What might people not know about you?
I am very interested in architecture and, particularly, Romanesque architecture. My wife and I have taken about nine trips to France, Italy and Belgium, and we drive from cathedral to cathedral and palace to palace to view architecture from the period.
I am also interested in Islamic architecture, and we will be going to Uzbekistan this October.
I have eclectic tastes in music, ranging from opera and experimental modern music to The Rascals, Rankin Family and David Bowie.
I have a large collection of bolo ties and wear them consistently to work.
When I traveled to China to work with their medical schools, I had many food adventures. For someone who grew up not liking seafood and only having canned tuna and fish sticks, I ended up eating sea pens, sea urchins and sushi with still wiggling flesh.
I also found that I liked fugu (pufferfish), even though you can die if it’s not prepared properly!
