The Hippocratic Oath cautions would-be healers to "First, do no harm!" Yet, over the centuries, medical and mental health professionals have, at times with the best of intentions, done enormous harm when they have acted on their unvalidated, subjective beliefs. A previous article in this column noted the epistemological anemia of expert opinions that are based upon clinical judgment, clinical insights, clinical intuition, and clinical experience. T.M. Tippins, "Does Clinical Experience Make One Wiser or Just Wizened?" NYLJ, Nov. 5, 2019. That article described a robust body of social science research, conducted over the span of many decades, that has convincingly demonstrated that the opinions of mental health professionals simply cannot be trusted when they are not supported by research-validated knowledge. The significance of that research extends beyond the ivy-covered walls of academia; it also has profound implications for the courtroom when purported experts come knocking. This article will explore those implications in a historical context and describe recently reported research suggesting that courts are being less than vigilant gatekeepers in the face of shoddy mental health testimony.

Witch-Hunters and Lobotomists

One of the earliest "diagnostic" manuals was the Malleus Maleficarum, first published in 1487. This intellectual gem provided detailed instruction for how to determine whether a suspect, most often a woman, should be classified, i.e., diagnosed, as a witch. Much akin to the American Psychiatric Association's (ApA) Diagnostic and Statistical Manual (DSM), the Malleus is replete with criteria indicative of one's necromantic status. It also posits an array of diagnostic tools (mostly torture techniques) to aid in ferreting out the dreaded witches. All the imparted wisdom was, of course, based on the authors' "experience," a word that was evidently deemed sufficiently impressive in days of yore without annexation of the turgid honorific, "clinical." Behold its experiential grounding:

If he wishes to find out whether she is endowed with a witch's power of preserving silence, let him take note whether she is able to shed tears when standing in his presence, or when being tortured. For we are taught both by the words of worthy men of old and by our own experience that this is a most certain sign, and it has been found that even if she be urged and exhorted by solemn conjurations to shed tears, if she be a witch she will not be able to weep … And it is found by experience that the more they are conjured the less they will be able to weep, however hard they may try to do so.

Kramer and J. Sprenger, The Malleus Maleficarum, Dover, 1971, p. 227 (emphasis added).

Who could argue with such experiential wisdom coming from "worthy men of old?" Certainly not the countless victims who were condemned on its basis.

Early medicine, though perhaps unburdened by the dogma of the religionists who touted the Malleus, labored beneath its own experience-based delusions. "Most medical historians agree that prior to 1890 virtually widespread medical treatments such as bleeding, purging, and blistering—all derived from a mixture of intuition and informal clinical experience—were either useless or harmful." S.O. Lilienfeld, et al., "Pseudoscience, Nonscience, and Nonsense in Clinical Psychology: Dangers and Remedies," in Wright and Cummings, Destructive Trends in Mental Health: The Well-Intentioned Path to Harm, Routledge, 2005, pp. 187-218, 195 (emphasis added) (hereinafter Lilienfeld et al., "Psedoscience").

One need not sojourn back to the 19th century to find such medical barbarisms. "For several decades of the twentieth century, prefrontal lobotomy was accepted by many mental health professionals as an efficacious procedure primarily on the basis of subjective observation and clinical judgment." Lilienfeld et al., "Psedoscience" (emphasis added). Not surprisingly, the practice was arrogantly defended by its purveyors who justified their actions by touting their "clinical experience" and their self-proclaimed (though never proven) superior powers of observation. Lilienfeld, et al., provide a sterling example of such hubris:

One physician who performed lobotomies stated, "I am a sensitive observer, and my conclusion is that a vast majority of my patients get better as opposed to worse after my treatment." This quotation reminds us that informal clinical observations, although helpful in generating fruitful hypotheses, should rarely, if ever, be the final arbiter of treatment efficacy.

Id. at 195.

As the saying goes, "Often in error, never in doubt."

The Homosexuality Debacle

Even more recently, one of the darkest chapters in the history of the mental health profession was its treatment—both literally and figuratively—of homosexuals. It is a sordid example of the horror produced when a misguided gaggle of "experts" relies upon subjective "clinical" judgment that is unsupported by scientific research. As previously discussed, (T.M. Tippins, "Unicorns in the Courtroom: The Case Against Diagnosis," NYLJ, June 6, 2019), homosexuality was designated a mental disorder in DSM-II (ApA, 1968). If you were homosexual you were, by psychiatric fiat, mentally disordered. That classification, unsupported by scientific research, was the product of nothing more than the subjective value judgments (also known as biases) of a self-anointed psychiatric elite who presumed to affix the scarlet stigma.

Stigmas, of course, have consequences—dire consequences in this instance. In addition to the social alienation and emotional devastation of being labeled "disordered," the treatment on offer was as bizarre as it was cruel. The same psychiatric crème de la crème that had branded gay folk mentally disordered used behavior modification techniques such as administering nausea inducing substances to cause vomiting at the sight of homoerotic images. They also used electroshock. "In the more brutal therapy sessions, the shock was delivered directly to the male patient's genitals every time the patient experienced any form of positive response to the slides being shown to him." J. Scot, "Shock the Gay Away: Secrets of Early Gay Aversion Therapy Revealed," Huffington Post, June 28, 2013, updated, Dec. 6, 2017 (hereinafter Scott, "Shock the Gay Away"). Those treatments might be considered paragons of humanity when compared to other curative techniques, some of which were administered involuntarily, such as castration and "ice pick" lobotomies. All delivered to the gay community courtesy of "clinical" judgment. Id.

None of these techniques actually "converted" their subjects into happy heterosexuals; but they did manage to destroy their interest in sex altogether—hardly surprising that voltage blasting through their genitals would have had that effect. D.C. Haldeman, "Sexual Orientation Conversion Therapy for Gay Men and Lesbians: A Scientific Examination," in J. Gonsiorek, J. Weinrich, Homosexuality: Research Implications for Public Policy, Sage Publications, 1991. Years (and who knows how many excruciating "conversions") later, homosexuality was removed from the DSM. The decision to remove it, like the earlier one to include it, was not driven by scientific research. The question was put to a vote of ApA members. The vote was close: 5,854 favored removal; 3,810 voted to retain it. N. Burton, "When Homosexuality Stopped Being a Mental Disorder," Psychology Today (Sept. 18, 2015). Diagnosis by plebiscite! What could be more scientific than that?

Claptrap in the Courtroom

When it comes to the curse of clinical crapola masquerading as expertise, our courtrooms have not been spared. Examples abound.

The Bed-Sharing Folly. The bed-sharing issue exemplifies the estrangement of some mental health practitioners from the research of their profession. It has not been uncommon to encounter custody evaluators who chalk up a demerit against a parent who permits a toddler or older child to share his or her bed, predicting that the practice would lead to negative developmental outcomes in children. Some of that testimony was rendered before there was any reported research on the point. In the absence of such research, the only professionally responsible answer to the bed-sharing question would have been to choke out the three words so dreaded by the intellectual elite: "I don't know." As forensic psychologist, Jeffrey P. Wittmann, Ph.D., has noted: "Too many experts view the phrase 'I don't know' as a mortal sin, when often it is the truest thing that can be said." J.P. Wittmann, Private Communication, Feb. 24, 2020. When research ultimately was reported in the early 2000s, it did not support the condemnation; it refuted it. "In an 18-year longitudinal study of 205 Euro-American families in California, bed-sharing was not associated with sleep disturbance, sexual pathology, or any other problems." See S. Hupp, Pseudoscience in Child and Adolescent Psychotherapy: A Skeptical Field Guide, Cambridge, 2019, p. 248, and the studies cited therein).

This scientific acquittal of the bed-sharing practice, so confidently condemned by untold numbers of evaluators, surely afforded no comfort to those parents whose custody cases were irreparably damaged by their testimonial disparagement. Those experts would have better served the legal system if they had set aside their plethoric self-delusion and acknowledged what they did not know. The cautionary words of the celebrated historian, Daniel Boorstin, are particularly apt in this context: "The greatest obstacle to discovery is not ignorance—it is the illusion of knowledge." 1984 January 29, The Washington Post, The 6 O'Clock Scholar: Librarian of Congress Daniel Boorstin And His Love Affair With Books by Carol Krucoff, Start Page K1, Quote Page K8, Column 2, Washington, D.C. [ProQuest]. Mark Twain made the same point in his inimitable fashion when he reminded audiences that "It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so."

In fact, the picture may be even darker. Some attorneys report that, notwithstanding the referenced research, evaluators can still be heard at times to condemn the bed-sharing practice. While at first blush this seems inexplicable, there is at least one possible explanation. Research suggests that relatively few mental health professionals consider the peer-reviewed literature of their discipline. In one study, 27% of clinicians interviewed admitted outright that "no empirical study had affected their work" and not a single psychologist in the sample "was able to identify a specific study or a specific instance of research use." L.H. Cohen, M.M. Sargent, L.B. Sechrest, L.B., "Use of Psychotherapy Research by Professional Psychologists," American Psychologist, February 1986 Vol. 41, No. 2, 198-206 (1996). In a more recent study of custody evaluators, the findings were similarly appalling:

Thirty-two percent of respondents reported having taken no workshops or seminars on child custody evaluations prior to conducting their first evaluations, whereas 30% had taken one or two workshops or seminars. Almost another 30% had taken three to six seminars or workshops, with the remaining respondents having taken seven or more. The amount of reading in the field prior to the first evaluation was also explored. Twelve percent of respondents reported having read no books or articles on child custody evaluations, whereas 7% and 17%, respectively, reported having read one and two articles or books.

J.N. Bow and D.A. Martindale (2009), Developing and Managing a Child Custody Practice, Journal of Forensic Psychology Practice, 9:2, 127-37.

The Subjective Projective

The freewheeling subjectivity that many forensic evaluators cavalierly apply to their interpretation of the data they gather in the assessment process extends as well to the selection of the techniques they use to gather that data. They often resort to an array of what are called "projective" techniques. While many and diverse projective "tests" have been developed, their common denominator is that they attempt to draw conclusions about the examinee's personality and other psychological characteristics by interpreting his or her unstructured responses to ambiguous stimuli, such as the inkblot cards used in the popular Rorschach test.

The projective hypothesis underlying these techniques is that "when people attempt to understand an ambiguous or vague stimulus, their interpretation of that stimulus reflects their needs, feelings, experiences, prior conditioning, thought processes, and so forth." R.M. Kaplan R.M., D.P. Saccuzzo, Psychological Testing: Principles, Applications, and Issues, Cenage Learning, 2018, p. 373. This hypothesis also underlies the claims of some evaluators that they can accurately interpret the meaning of a child's play with dolls or the pictures they draw of themselves or their families. Spoiler alert: they can't.

Subjectivity, Reliability, and Validity

An overarching problem with projective techniques is that, with rare exception, they have never been empirically demonstrated to be reliable and valid measures of the traits or characteristics that evaluators are attempting to assess. Moreover, because these interpretations are wholly subjective, they are inherently idiosyncratic. How a given evaluator interprets the subject's behavior may be markedly different from the interpretation that a different psychologist would fashion. So idiosyncratic are these projective approaches that the late Anne Anastasi, recognized as a testing guru for decades, famously wrote that the "lack of objectivity in scoring and interpretation" that inheres in the projective approach is so severe that "the final interpretation of projective test responses may reveal more about the theoretical orientation, favorite hypotheses, and personal idiosyncrasies of the examiner than it does about the examinee's personality dynamics." A. Anastasi and S. Urbina, Psychological Testing, Seventh Edition, Prentice Hall, 1997, p. 434.

David A. Martindale, Ph.D., a prominent and prolific figure in the forensic community, has addressed the unreliability of projective techniques in the context of custody evaluations. In discussing projective drawings, he has noted that what children draw may represent perceptions, wishes, or nothing more meaningful than something to which they were recently exposed. He cautioned:

If one fully understands the concept of projection, one is more likely than not to conclude that neither drawing nor doll play is a "reliable source of information." Even if it had been demonstrated that the dynamic of projection consistently operates as children draw and play with dolls, there would still be no basis for our relying upon it.

Martindale, D. A. (2006), Play Therapy Doesn't Play in Court: A Review of Anita Trubitt's (2004) Play Therapy Goes to Court: Implications and Applications in Contested Child Custody, 2nd ed, (Kailua, HI: Author), Journal of Child Custody: Research, Issues, and Practices, 3:1, 77-86.

The reliability deficiency becomes starkly evident when one considers the premises underlying these projective procedures. Hupp and Jewell provide a revealing and marvelously entertaining peek into the world of the projective enthusiast simply by listing many of the farcical assumptions that underlie the interpretation of projective drawings, such as the "Draw-a-Person" procedure. As its name suggests, the examinee draws a picture of a person and then the examiner, perhaps channeling Johnny Carson's famed Carnac the Magnificent, decides what it means. Welcome to the land of the ludicrous:

For example, a large head in the drawing suggests the person may be "paranoid, narcissistic, intellectually righteous, and vain." In another example, "over-emphasis of the mouth is frequently tied up with food faddism and gastric symptoms, profane language, and temper tantrums." Other signs include small eyes representing self-absorption, large eyes representing hostility, noticeable ears representing symptoms of schizophrenia, emphasized nostrils representing aggression, hands in pockets representing masturbation, large hands representing guilt, and buttons representing dependence on the mother (especially if drawn in the position of the belly-button). Also, the following drawing components can have sexual significance: mouth, lips, hips, hair, hats, hands, nose, neck, Adam's apple, buttocks, pockets, shoulders, shoes, cigarettes, feet, ties, guns, canes, waistlines, and, of course, the drawing of private parts.

Hupp and J. Jewell, Great Myths of Child Development, Wiley-Blackwell, 2015, p. 107-08.

Hupp and Jewell further point out that when the Draw-a-House "test" is used, if the child includes a chimney in the drawing it is seen as having phallic symbolism. They entertainingly offer a blank space in their book and challenge the reader to draw a chimney that does not have a phallic shape. Id. at 109. Little wonder that Robyn M. Dawes, recipient of the American Psychological Association's William James Book Award, has strongly cautioned against the use of projective techniques, particularly in forensic settings:

If a professional psychologist is "evaluating" you in a situation where you are at risk and asks you for responses to inkblots or to incomplete sentences, or for a drawing of anything, walk out of that psychologist's office. Going through with such an examination creates the danger of having a serious decision made about you on totally invalid grounds. If your contact with the psychologist involves a legal matter, your civil liberties may be at stake. If you have been mandated to see the psychologist by a court order, quietly object to the test if possible, and ask to make an appointment later. Immediately consult a lawyer … Do not respond first, then object later. Any of your responses may be "interpreted" as indicating that you have an objection to the whole procedure based on some form of psychological pathology.

R.M. Dawes, House of Cards: Psychology and Psychotherapy Built on Myth, The Free Press, 1994, p. 152-53 (emphasis added).

Nothing If Not Persistent

Notwithstanding the cogency of these admonitions against the use of unreliable projective techniques, their use persists. Martindale has noted that "in 2015, we find evaluators extolling the virtues of assessment procedures that lack reliability and, as a result, also lack validity." D.M. Martindale, "Much is Forgotten in 123 Years," AFCC E-News, October 2015.

Anita Trubitt, L.C.S.W., an advocate of projective interpretation of child-play activities, concedes the absence of scientific support for such techniques. "Is the practice of play therapy intuition or science? Our assumptions about what children's play reveals is based on our clinical experience, but not adequately supported by science … There are currently no studies on the incorporation of play therapy techniques in custody evaluations." A. Trubitt, "To Play or Not to Play in Child Custody Evaluations: Recent Challenges in Current Practice," Association of Play Therapy Conference, Toronto, Oct. 13, 2006 (emphasis added) (hereinafter Trubitt, "To Play or Not to Play").

Despite the lack of scientific support, Trubitt opined that "there are likely many clinicians … who still rely upon play therapy modalities" and "make tentative inferences about it." A. Trubitt, "A Call for Humility in a Small World," AFCC E-News, October 2015. Such reliance, of course, flies in the face of psychology's core commitment to reliability and validity and reflects the professional apostasy of a significant contingent of the psychology profession that was discussed in a previous article. See T.M. Tippins, "Forensic Custody Reports: Where's the Science?", NYLJ, Sept. 4, 2019.

Stunningly, Trubitt's proposed solution to the reliability/validity void is not to abandon use of unscientific techniques but, rather, to conceal their use from the courts:

We can continue to use play therapy methods as a confirmation (or lack thereof) of what the other data suggests. We employ it for our own edification and this helps us to be more confident about our recommendations. We do not have to mention its use at all in the report to the court.

Trubitt, "To Play or Not to Play".

Lest there be any doubt about what Trubitt is suggesting, she more recently wrote, "We collect a mountain of material and we therefore have to be selective about what we include. It is understandable that we would select those examples that best support our own position …" A. Trubit, "A Final Word," AFCC E-News, October 2015. Understandable or not, such tactics are inexcusable. There is a name for what Trubitt has proposed. In 2005, Martindale coined the term "confirmatory distortion," to describe the process "by which an evaluator, motivated by the desire to bolster a favored hypothesis, intentionally engages in selective reporting or skewed interpretation of data, thereby producing a distorted picture of the family whose custody dispute is before the court." D.A. Martindale (2005), "Confirmatory Bias and Confirmatory Distortion. pp. 31-48, at 33, in J.R. Flens and L. Drozd (eds.) Psychological Testing in Child Custody Evaluations, New York: Haworth. The maneuver has also been described as "coloring the data." C. Steinberger, AFCC E-News, November 2015. A more common term is "cherry-picking." Whatever the label, it is a deceitful practice that runs counter to the all-important principle of forensic transparency and has no proper place in the courtroom.

O Gatekeeper, Where Art Thou?

Given the avalanche of asininity that has historically spewed forth from the mental health profession, both in and out of the courtroom, the need for strict evidentiary gatekeeping cannot be gainsaid. A recently published study suggests, however, that courts are far from vigilant in scrutinizing proffered expertise. The authors examined the extent to which various psychological assessment techniques, some of dubious reliability, were critically analyzed by the courts. They report that "legal challenges to the admission of this evidence are infrequent," occurring in only 5.1% of cases in the sample studied. Further, when challenges were raised, "they succeeded only about a third of the time." Particularly distressing is the finding that challenges to "the most scientifically suspect tools are almost nonexistent. Attorneys rarely challenge psychological expert assessment evidence, and when they do, judges often fail to exercise the scrutiny required by law." T.M.S. Neal, et al., "Psychological Assessments in Legal Contexts: Are Courts Keeping 'Junk Science' Out of the Courtroom?", Psychological Science in the Public Interest, 2019, Vol. 20(3) 135-64, Association of Psychological Science, Sage. They further report that "courts routinely admit evidence with poor or unknown scientific foundations (citation omitted). When poor science is not recognized as such and is used to reach legal decisions, the risk of error rises and the legitimacy of the legal system is threatened. (citation omitted)." Id.

Conclusion

Even the partial history of the harm produced by subjective "expert" judgment described above is enough to make Hippocrates weep. That a seemingly significant segment of the mental health profession ignores the research of their discipline is more than a bit troubling. Even more disturbing is the willingness of some to advocate—publicly, no less—in favor of using methods of dubious validity and then concealing their use from the courts. If history teaches us anything it is that when mental health professionals disregard the core principles of reliability and validity, and instead rely upon their subjective judgment, people get hurt—badly. When that harm is inflicted in the courtroom, the legal system necessarily shares responsibility for the resulting trail of tears. Clearly, lawyers and, most importantly, judges—the evidentiary gatekeepers—must be acutely skeptical when these "experts" come calling and must employ the most exacting standards of reliability when determining the admissibility of the proffered evidence. Anything less ignores the sins of the past and, as Santayana warned, condemns us to repeat them in the future.

Timothy M. Tippins is an adjunct professor at Albany Law School and is on the faculty of the American Academy of Forensic Psychology and on the Affiliate Postdoctoral Forensic Faculty at St. John's University.