Photo: Hewlett Askew

Should the bar exam be broken up and administered at different times, with some components tested during law school?

Should the exam incorporate mock client interviews or negotiations to more closely mirror what lawyers actually do?

Does the test go too in-depth on too many legal subjects?

Those are just a few of the questions that have surfaced from an extensive, seven-month series of listening sessions conducted by the National Conference of Bar Examiners' Testing Task Force with bar exam stakeholders—the first of a three-phase inquiry into the future of the test.

More than 400 legal educators, bar examiners, lawyers and others shared their thoughts on the strengths and weaknesses of the bar exam, which has come under intense scrutiny over the past seven years amid a significant downturn in pass rates. Just 69% of first-time bar takers nationwide passed in 2018, down from 79% in 2011. The task force's inquiry is looking at every aspect and component of the exam, as well as the rise of the Universal Bar Exam, which has been adopted by 35 states thus far.

It seems that people have plenty to say about the bar exam—even if they don't necessarily agree on how to improve the test. The task force in August released a 34-page report on the feedback it received from stakeholders, which highlights the relative lack of consensus on how the exam should be formatted and what it should cover. While most stakeholders appreciated the value of the bar exam as a way to credential lawyers, their feedback ran the gamut, noted Judge Cynthia Martin of the Missouri Court of Appeals for the Western District, who is heading the task force. Many of the suggestions the task force received are at odds with each other. As a result, the group will conduct more research and surveys to determine what matters most to stakeholders, she added.

"For example, it was a popular refrain from stakeholders to say, 'We want more skills testing or, gee, wouldn't it be nice if you could build in some realistic scenarios such as client interviews or negotiations?'" Martin said. "At the same time, you're seeing stakeholders say, 'We want results generated more quickly. We want the exam to be, perhaps, administered more frequently. We want the exam to be less expensive.' Those things are inherently in conflict."

But the feedback has helped to clarify the many avenues for reform that the task force will examine, Martin said. It will look carefully at the possibility of breaking the exam up and administering components at different points in time, she said. And it will also consider a change under which those who fail the overall exam would not have to retake any portions that they passed, among other potential reforms.

Among the most common feedback the task force received, according to the report:

• The Multistate Bar Exam (the daylong multiple-choice portion of the bar) tests "too deeply" on legal subjects, going beyond entry-level competency; and too many subjects are tested over the course of the entire exam.
• "Constructed response" items such as essay questions are preferable to multiple-choice questions, and simulations such as mock interviews are useful ways to assess skills.
• The test should be broken up, with the Multistate Bar Exam coming after law students have completed their first-year core classes with skills testing coming later; or the bar should be administered more frequently than twice a year.

The task force has now entered the second phase of its work—which focuses on the skills and competencies new lawyers need to be effective in practice. To that end, it is asking stakeholders to take a practice analysis survey.

Once those results are in hand, an independent research firm hired by the task force will use the findings of both research phases to develop a series of recommendations. The task force aims to complete its work by 2021.

"We meant it when we started this task force and said that we are serious about this being an open, transparent, outside-the-box, unencumbered assessment of the exam," Martin said. "That means everything: timing, structure, content. We're looking at all of that."