We've all heard it: "Practice makes perfect!" But, does it really? Does one's experience in doing a task improve the quality of one's performance of that task? And if it does so in one context, does it do so in all?

These are important questions for judges and lawyers who struggle in the tarpit of contested custody litigation. Custody evaluators frequently offer opinions, conclusions, and inferences that have little or no support in the accumulated knowledge of the psychology discipline. When challenged, they typically defend their position by asserting, in the most authoritative tone that their overworked vocal chords can muster, that their "many years of clinical experience" have honed their "clinical judgment" to such an intellectually elegant level that their say-so should be adequate authority for their proclamations. Ipse dixit, anyone?

This article will explore the question of whether the subjective, nonscientific judgment of evaluators, predicated upon their years of experience, is a legitimate basis for their opinions.

Evidentiary Context: Oracles Need Not Apply

Before examining psychological testimony from the epistemic perspective, it is important to understand it in the evidentiary context of the courtroom.

The U.S. Supreme Court has repeatedly made clear that the latitude afforded to the expert witness to go beyond factual, percipient, testimony and into the land of inference, conclusion, and opinion, is conditioned upon the expert predicating those extrapolations on "the knowledge and experience of his discipline." Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579, 592 (1993); Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 148 (1999) (emphasis added).

It is important here to underscore that the court did not say that the latitude to opine afforded to an expert depends upon the knowledge and experience of the individual expert on the witness stand, but, rather, upon the knowledge of the discipline in which the witness's credentials have qualified him or her as an expert. In other words, once the witness is "qualified" in a given discipline, he or she is essentially a spokesperson for that discipline. The expert is not, by virtue of qualification in the particular discipline, deemed to be an oracle, unbound by the intellectual strictures of that discipline, free to spew whatever sentiments may radiate from his or her inner lights.

Psychology as a Scientific Discipline

Within this evidentiary context, let us consider psychological testimony. A previous article in this column discussed the fact that psychology defines itself as a science, without which science it is devoid of any raison d'etre. Tippins, T.M., "Forensic Custody Reports: Where's the Science?" NYLJ (Sept. 5, 2019). Thus, the psychologist is on the witness stand to inform the court of the content of the scientific research ensconced in the literature of the discipline. In some instances, though not always, the witness may go further and testify as to how that research-based, demonstrably valid scientific knowledge might apply to the case before the court. Whether the testimony includes an opinion about the case or is purely a didactic explanation of psychological principles that the judge, as fact-finder, may or may not apply, however, the uppermost question in the mind of the lawyer and, more importantly, of the court must be, "Where's the science?"

Constant focus on this question is crucial because, as previously discussed, the psychology field has fallen victim to a destructive schism within its ranks, a deep divide between Psychology-as-Discipline, i.e., as practiced by those psychologists who remain faithful to the scientific method that represents its defining attribute, and Psychology-as-Hustle, consisting of those practitioners who have abandoned the scientific principles of the discipline, and who, in pursuit of livelihood, routinely stray to the greener (pun intended) pastures of pseudoscience and anti-science. See "Forensic Custody Reports: Where's the Science?,"supra.

It is important here to understand that infidelity to the scientific core that defines psychology as a discipline comes in two flavors—pseudoscience and anti-science. Stephen Hupp, Ph.D., is a professor in the Department of Psychology at Southern Illinois University where he teaches courses in clinical child psychology, child development, and pseudoscience. His recently published text, Pseudoscience in Child and Adolescent Psychotherapy: A Skeptical Field Guide, (Cambridge, 2019), explains the distinction between these two species of nonscience:

In essence, pseudoscientific practices give the superficial appearance of science, using scientific-sounding jargon borrowed from legitimate science, yet lacking the substance of having valid scientific research to back up their claims. Pseudoscience is distinguished from anti-science, another threat to the mental health profession in that anti-science proponents denigrate science itself or maintain that there are different ways of knowing, all equally valid. Proponents of anti-science often unapologetically offer treatments that are utterly lacking in any kind of scientific support and often make no attempt to claim such support exists. Hupp, p. 2 (internal references omitted).

If one substitutes the word "opinions," "evaluations," or "testimony" for the word "treatments" in the final sentence of the above-quoted excerpt, it perfectly captures the essence of the forensic custody reports and testimony all too often tendered by "experts" whose words can alter the lives of countless families whose fates hang in the balance every day in custody courts across the land.

Hupp's text, as well as the others referenced throughout this article, should be required reading for lawyers and judges if we are to be informed consumers of forensic custody opinions. This is essential because, as Hupp notes, "psychology is rampant with pseudoscience" (Hupp, p. 5) and its effect is pernicious:

Pseudoscience is insidious because unless the consumer is educated in how to evaluate claims of scientific evidence, they could be deceived into thinking that such practices … have legitimate scientific evidence supporting their use. Hupp, p. 2.

Accordingly, when confronting psychological testimony, the court and the lawyers must keep constant focus on one overarching question. The question is not what the psychologist who is at the moment warming the witness chair thinks he or she knows; but, rather, what is in the library of psychology. What does the scientific research of the discipline demonstrate to be true, valid knowledge, as opposed to the subjective speculations of the individual on the stand.

Of course, one will rarely, if ever, hear experts characterize their opinions as speculative or subjective, at least not voluntarily. They prefer to speak in the stilted phraseology of clinical judgment, clinical impressions, even clinical intuition, bolstered by their "many years of clinical experience," as though the addition of that hoity-toity jargon, clinical—a word as meaningless as it is pretentious—somehow elevates their subjective speculations to a higher plane of intellectual validity. It doesn't. This negative judgment is not this writer's clinical opinion; it is what decades of scientific research has demonstrated to be a fact.

What the Research Demonstrates

Custody evaluators of the nonscientific variety love to natter on with opinions predicated on personal, subjective beliefs, untested "pet" theories, idiosyncratic impressions, intuitions, and other specious speculations, all of which cerebral effusions are unanchored to the established knowledge of their discipline. When challenged, they justify their pseudointellectual excess by asserting that their "clinical judgment" and their "clinical experience," are sufficient authority for their opinions. More than six decades of research studies say otherwise. That research has demonstrated beyond peradventure that their subjective clinical judgment is not a reliable substitute for demonstrably valid knowledge produced by scientific research. Nor, those studies demonstrate, does their judgmental accuracy improve as a result of their "many years of experience."

The late Jay Ziskin, Ph.D., was co-founder of the American Psychology-Law Society and author of the seminal work, Coping With Psychiatric and Psychological Testimony, Fifth Edition (Law and Psychology Press, 1995), a treatise of such significance that its work has been carried on posthumously by David Faust, Ph.D., in a Sixth Edition published by Oxford University Press in 2012. Ziskin noted that once an expert is forced to acknowledge the lack of scientific knowledge underlying his or her opinion, he or she will often seek refuge in such tired banalities as "based on my experience," "based on my training," or "in my professional judgment," or some combination of these, uttered with the intended implication "that his qualifications, or training, or experience have produced refined and trustworthy judgment skills." Ziskin, Fifth Edition, 1995, p. 195. As he stated, "once the mask of established scientific method is removed, there is little other basis on which the expert can retreat to make his stand." Ziskin, Fifth Edition, 1995, p. 195. This, however, is shaky ground upon which to make a stand, as there is a "vast body of research," that has demonstrated "that the validity of clinical judgment and the amount of clinical experience are unrelated." Ziskin, Fifth Edition, p. 196. Ziskin further stated:

There is an abundance of literature consisting of both authoritative statements and research findings, many in the late 1970s, the 1980s, and the 1990s, which clearly supports the position that psychiatric and psychological assessment skills do not improve as a result of experience or training. Ziskin, Fifth Edition, p. 495.

He concluded that the research on the "reliability and validity of psychiatric diagnosis, prediction, and formulations of other types" clearly demonstrates that "conclusions drawn from psychiatric evaluations are often of low or doubtful reliability." Ziskin, Fifth Edition, p. 476.

Ziskin is hardly alone in concluding that clinical judgment, irrespective of the years of experience behind it, is not a demonstrably valid basis for decision-making. Numerous other notables in the discipline of psychological science have rendered similar assessments.

Howard N. Garb, Ph.D., in his much-heralded book, Studying the Clinician: Judgment Research and Psychological Assessment, published by the American Psychological Association in 1998, analyzes a massive body of research on the subject of clinical judgment and experience. Among the research findings that he analyzed were the following eye-opening revelations:

• Clinicians consistently failed to make more valid ratings of personality and psychopathology than did graduate students. Garb, p. 15.
• Psychologists and psychology graduate students are only "sometimes more accurate" than laypeople. Garb, p. 16.
• Experience with a task has not been related to the validity of judgments. Garb, p. 200.
• Experienced clinicians were not more accurate than less experienced clinicians, clinicians were not more accurate than advanced graduate students, and presumed experts were not, in general, more accurate than other clinicians. Garb, p. 200.

The late author, Robyn M. Dawes, recipient of the APA's William James Book Award, in his groundbreaking text, House of Cards: Psychology and Psychotherapy Built on Myth (Free Press, 1994), made clear that "appeals to experience per se are invalid because experience per se does nothing to enhance accuracy." Dawes, p. 109.

Instead of relying on research-based knowledge in their practice, too many mental health professionals rely on "trained clinical intuition." But there is ample evidence that such intuition does not work well in the mental health professions. (In fact, it is often no different from the intuition of people who have had no training whatsoever). Forty years ago, professionals could be excused for believing in the power of their own intuitive judgment, because at that time there was very little evidence concerning its accuracy one way or the other. That is no longer true. Today there is plenty of evidence about the accuracy of their intuition, and it's negative. Dawes, p. vii.

Jeffrey P. Wittmann, Ph.D., a widely recognized authority in the field of forensic psychology, summed it up nicely some 35 years ago:

Our field is famous for supporting conclusions during testimony simply on the basis of "accumulated clinical experience," a phrase which may mean nothing more than "accumulated personal bias." Wittmann, J.P., "Child Advocacy and the Scientific Model in Family Court: A Theory for Pre-Trial Self-Assessment," The Journal of Psychiatry and Law, 13(1), (1985), pp. 77-78.

The avalanche of research reviewed by the cited authors, as well as many others, has given the lie to assertions that experience increases judgmental accuracy. The problem facing the lawyer seeking to educate the court as to the speciousness of such claims is that these research findings are counterintuitive; they contradict commonly held beliefs about the value of experience in the process of learning. As Ziskin noted, "most of us believe that individuals get better with practice, so certainly we would expect this to apply to experts as well." Ziskin, Fifth Edition, p. 195. Accordingly, it is essential to understand—and to ensure that the court understands—why clinical experience does not make judgment more accurate.

Why Johnny (or Jenny) Can't Learn

So just why does clinical experience fail to produce more accurate clinical judgment? The answer lies in the two indispensable conditions that must exist to learn by experience. Dawes explained it clearly:

Two conditions are important for experiential learning; one, a clear understanding of what constitutes an incorrect response or error in a judgment, and, two, immediate, unambiguous and consistent feedback when such errors are made. In the mental health professions neither of these conditions is satisfied. Dawes, p. 111.

Ziskin brilliantly illustrated this point by analogizing to how one would learn the skill of archery. Ziskin, Fifth Ed., p. 523-24. First, you must have a clearly defined target so that accuracy versus error is objectively determinable. Second, you must have systematic, immediate, unambiguous, and accurate corrective feedback with respect to each of your efforts to hit the target. You then use that feedback to adjust or correct your next effort. Did your arrow land several feet to the left of the target? You adjust to the right. Was it off to the right? You adjust to the left. Assuming no other variables impeding your ability, you will likely become more accurate with experience. But, now, imagine trying to learn archery under different conditions. You are wearing a blindfold and you have no way of knowing where the target is or how near to or far from the target each arrow is landing. You could practice until hell freezes over with no increase in accuracy.

That second scenario closely approximates the setting in which custody evaluators function. They do not have a clear, operationally defined, target; nor do they have any systematic, corrective, feedback against which to measure the accuracy of their evaluation efforts. The target, "the best interests of the child," is nowhere operationally defined; not in law and not in behavioral science. It is, at best, a subjective ideal, an amorphous goal, a fuzzy target; one laden with implicit social and moral value judgments that repose far outside the bounds of the demonstrable knowledge base of the psychology discipline and, therefore, beyond the proper ambit of expert testimony. One refreshingly candid jurist summed it up nicely, "At the bottom line, what is in the child's best interest equals the fact finder's best guess." Montgomery County Dept. of Social Services v. Sanders, 38 Md. App. 406, 419, 381 A.2d 1154 (1977). In such an intellectual quagmire, the custody evaluator has no way to determine whether his or her predictive opinions, case by case, are accurate or erroneous.

With respect to the immediate, unambiguous, and consistent feedback, essential to experiential learning, as Tony Soprano might say, "Fahgettaboudit!" The evaluator has virtually no meaningful feedback. There is no system in place that tracks those children who have been placed in custodial arrangements consistent with the opinion of the evaluator to see how well or poorly those children have fared. The evaluator has no more idea than a box of rocks how many of those children have matured into successful (another value judgment) adulthood versus how many have led lives of addiction, delinquency, depression or other forms of despair; or how many graduated from college versus how many landed in prisons, mental institutions, or morgues at a ripe young age.

Now, it is not the evaluators' fault that they have neither a clear definition by which erroneous judgments can be measured nor the corrective feedback required for them to improve with experience. It is, however, very much their fault when they pretend that their clinical judgment should be trusted because of their extensive experience in doing evaluations when a raft of research has revealed that contention to be a falsehood. In the absence of the two critical precursors to experiential learning, clear definition of the target and immediate corrective feedback, their subjective judgment is simply not to be trusted, whether they have done one, 100, 1,000, or 10,000 evaluations. They are in the same position as the blindfolded student-archer who has no idea where the target is or where the arrows are falling. To pretend otherwise is simply shameful.

Do Custody Evaluators Really Ignore the Research?

As discussed throughout this article and the many sources it references, the scientific research has emphatically refuted the assertion that clinical judgment, unanchored to the scientific research of the psychology discipline, is a valid basis for expert inference. Indeed, among those psychologists who still care about the scientific integrity of their discipline, that debunked notion is viewed as "lore" and "myth." See Garb, H.N., Boyle, P.A., "Understanding Why Some Clinicians Use Pseudoscientific Methods," in Lilenfeld, S.O., Lynn, S.J., Lohr, J.M., Science and Pseudoscience in Clinical Psychology, Second Edition, Guilford Press, 2015, p. 19; Lilienfeld, S.O., Lynn, S.J., Ruscio, J., Beyerstein, B.L., Fifty Great Myths of Popular Psychology, 2010, Wiley Blackwell, p. 230.

Yet, too many custody evaluators ignore the research and cling to their idiosyncratic instincts and intuitions. Randy K. Otto, Ph.D., and David A. Martindale, Ph.D., both highly acclaimed authorities in the field of forensic psychology, have addressed the problem of "ongoing reliance on clinical intuition" in the following terms:

One evaluator's clinical intuition is another's hokum. Clinical insights, hunches, and intuition should not qualify as opinions, or as the basis of opinions, that an expert is expressing with a reasonable degree of professional certainty. Otto, R.K. & Martindale, D.A., "The Law, Process, and Science of Child Custody Evaluation," in Costanzo, M., Krauss, D., Pezdek, K., Expert Psychological Testimony for the Courts (2007) Lawrence Erlbaum Associates Publishers, p. 269.

This stubborn adherence to subjective judgment by the nonscientific infidels of the psychology profession poses a clear and present danger to the integrity of legal adjudication. Paul E. Meehl, Ph.D., arguably the most celebrated psychologist of the 20th century, made the point clearly:

A judge, in admitting expert testimony …, naturally assumes that while experts may disagree, any expert knows the basic facts and tools of the trade, and knows how to reason about them properly … In the "soft" areas of psychology … this cannot be safely presumed." Meehl, P.E., Law and the Fireside Inductions: Reflections of a Clinical Psychologist, 1989, Behavioral Science & the Law, 7, 521-550, 548.

Yet, sadly, too many courts do "assume"—quite unsafely, to be sure—that mental health witnesses do know what they are talking about. Worse, some courts actively encourage irresponsible, nonsensical testimony. Otto and Martindale observed:

What must be addressed is the unfortunate reality that courts often encourage experts to confidently testify to inferences that go well beyond those that can responsibly be drawn based on available data. Judges seeking answers in the adjudication of custody disputes may be more receptive to hearing from experts who confidently claim that they have the answers, rather than experts who acknowledge that the answers are elusive. Particularly where the issues are complex, the expert who offers facile solutions may be perceived as most helpful, whereas the expert who declines to speculate may be considered to be uncooperative. Otto, R.K., Martindale, D.A., p. 270.

Instead of encouraging nonscientific testimony, Otto and Martindale make the case that when custody evaluators fail to adhere to psychological science, "judges and attorneys—must hold their feet to the fire." Lawyers and judges, as "the ultimate gatekeepers," must "understand what experts do and do not know, and what they can and cannot say." Otto and Martindale, pp. 270-71.

Conclusion

In the final analysis, it is the responsibility of the legal system to ensure that forensic evaluations that are not supported by scientific research are kept out of the courtroom. To discharge this gatekeeping obligation will require that lawyers and judges become educated, if they are not already, in the science of psychology, its methods, its knowledge base, and its limitations, so that they can distinguish the anti-scientific infidels and the pseudoscientific pretenders from the legitimate psychological experts who remain faithful to the scientific principles of their discipline. Most importantly, courts must be vigilant in the discharge of their gatekeeping responsibility. Accordingly, such fabled and comfortable idioms as "I'll take it for what it's worth" and "I'll allow it because there's no jury here" must be relegated to their richly deserved niche in the oblivion of legal history.

Timothy M. Tippins is an adjunct professor at Albany Law School and is on the faculty of the American Academy of Forensic Psychology and on the Affiliate Postdoctoral Forensic Psychology Faculty at St. John's University.