The College of Family Physicians of Canada exam season is upon us.
In a few weeks, the senior residents in our program will undergo two days of testing that will have a profound impact on their futures.
I’m pretty sanguine about the prospects for most of my own particular group of potential family docs—but being the classic Type As they are, our residents are quite wound up.
They’re studying, practicing, reading up on the 104 Priority Topics, and doing whatever they can think of to prepare.
The exams have two parts: Short Answer Management Problems (SAMPs), a written exam, and the Simulated Office Oral Examination (SOOs).
The SOOs are standardized cases based on stories of real cases. You can understand that, for someone like me with my storytelling bent, I have a sneaky affection for this method of evaluation. I spent ten years on the CFPC’s examination committee and I can honestly say it was some of the best fun I have ever had in my life!
We were a group of family docs from across the country each bringing in a case that we thought was illustrative of the principles of family medicine. Then we’d massage the stories into the examination format to assess the candidates’ abilities in patient-centered interviewing and management skills.
We would argue for hours about what was essential and what was desirable. I was fascinated by the stories: the philandering department store owner, the residential school survivor with PTSD, and one of my own beloved creations, the woman taking three different diuretics, from three different walk-in clinics for “puffiness” and who had a TSH of 100 and potassium of 2.7.
Many complain the exam is unnecessarily stressful, that it’s formulaic, unrealistic and that no one could actually deal with these complex cases in the 15 minutes allotted. True, this is not real life. Real life is what formative evaluations during residency are for. That’s where our responsibilities as preceptors come in as we shepherd our learners through the wild rides of their residency programs.
Yet, with something so high stakes as licensure, some kind of summative evaluation is necessary. It is only prudent to guard against prejudice, unfairness and misadventure in the in-training evaluations.
Recently I took part in a practice SOO for two different McGill teaching units. I was examining their residents as they examined mine, and we gave feedback and advice to each candidate afterwards. We did three different cases, and my case was a 30-year-old, slightly slutty lesbian woman in an abusive relationship. “I don’t think I can pull that off,” I said in the preliminary meeting of examiners. “Why not?” said one of my colleagues wryly. “Which part are you having trouble with? Is it the lesbian part? Or is it the slutty part?”
I laughed, “No, half my patients think I am gay, I think I can pass, as for slutty, maybe, but thirty? I don’t think I can ever pull that off!”
So using a zoom filter to try and hide my bags and wrinkles, I took the residents through their paces.
It was fascinating to watch how the different residents handled the exam. The presenting complaint and first problem was oral allergy syndrome. There was a very modern twist to this issue as the patient, with the advice of a friend and Doctor Google, had decided that what she needed was an EpiPen. With this request as the opening statement, the residents had to work backwards to elicit a proper history and then forwards to put into place a realistic plan that was understandable and acceptable to the patient.
Most of them managed very well, even if they did not “get” the actual diagnosis. All were safe. Some were better able to create the bond of trust than others. One resident in particular was very directive: “No you are wrong, you don’t need an EpiPen. Do this instead.” This is behaviour guaranteed to send any patient down the street to the nearest walk-in clinic to try again.
It was the second and more significant problem that really separated the sheep from the goats. The problem was that the patient was in an abusive relationship, and that the violence in this relationship was escalating. One resident completely missed the point, and thought the second problem was the patient’s and partner’s alcoholism, which was a facilitating factor but not the main issue. She made the cardinal mistake of asking the patient to bring in her partner so that they could discuss the issue together. She only backed down when the patient categorically stated that that was a scary proposition for her. I pointed out to her that this was a recommendation that could get someone murdered.
Giving the residents feedback was a way for me to reinforce principles of patient-centered interviewing; the importance of understanding the patient’s illness within the context of their lives, checking back your understanding of what the patient is saying and making sure that they understand and agree with the plan. This is not just examsmanship, this is important in real life.
The most interesting part of the whole process, for me, was the examiners debriefing after the session. We shared our evaluations and concerns. Remarkably, the exam was able to pick up the problem residents and the star residents from each program. So even with no prior information, we could make good judgements. That is what an exam needs to do. My old teacher, Dr. Mark Clarfield, who was the first Head of the Division of Geriatrics at the Jewish General Hospital used to say, “There are three kinds of doctors: the doctor you send your Bubbie (grandmother) to, the doctor that if you hear that your Bubbie is seeing them, it’s ok, and the doctor that if you hear your Bubbie is seeing them, you get her the hell away from them. This roughly sorts to the categories on the exam or Superior, Certificant and Non-Certificant.
Now that there is a movement to streamline licensing of IMGs and Canadians medically educated abroad, I would argue that a simple multiple choice test is inadequate to screen doctors wanting to work with our Canadian populations. In my many years of teaching, it is not the medical knowledge that is the major issue, although it is necessary. It is the ability to effectively understand and communicate with patients that distinguishes the capable from the dangerous.
So SOO exams are not perfect but, to paraphrase the quote about democracy, often attributed to Churchill: “Democracy is the worst form of government—except for all the others that have been tried.”
In my mind, SOO examinations are the worst possible system of evaluation of communication skills, except for all the others that have been tried.
So to my dear residents, I hope your exams go well. Break a neuron! Go forth and become the best doctors that you can be.