What Lawyers Need to Know About Search Tools

It will come as no surprise to anyone who has handled complex litigation during the past five years that the volume of electronically stored information that must be reviewed in the course of discovery can be staggering. It may be more surprising to learn that keyword search is not nearly as effective at identifying relevant information as many lawyers would like to believe. See David C. Blair and M.E. Maron, “An Evaluation of Retrieval Effectiveness for a Full-Text Document-Retrieval Sys.,” 28(3) Comm. of the ACM 289 (1985) (showing lawyers estimated their search had identified 75 percent of the relevant documents when only about 20 percent were found); Douglas W. Oard, et al., Overview of the TREC 2008 Legal Track (March 17, 2009), (showing Boolean search identified only 24 percent of the relevant documents); Stephen Tomlinson, et al., Overview of the 2007 TREC Legal Track (April 30, 2008), (showing Boolean search identified only 22 percent of the relevant documents).

Litigators today face severe challenges in identifying and producing documents responsive to requests for production, on time, within budget and without waiver of privilege. See, e.g., In re Fannie Mae Sec. Litig., 552 F.3d 814 (D.D.C. 2009) (involving delayed production in which 400 search terms yielded 660,000 documents, costing $6 million — or 9 percent of annual budget — to review); Mt. Hawley Ins. Co. v. Felman Prod., No. 3:09-CV-00481, 2010 WL 1990555 (S.D. W. Va. May 18, 2010) (finding waiver of privilege for inadvertent production of 377 privileged documents in 346-gigabyte production). To assist lawyers in these efforts, there are a dizzying array of vendors and search tools on the market, each claiming to offer the “silver bullet.” For time-strapped lawyers who have little — if any — interest in technology, sorting through the options can be overwhelming. But the consequences of getting it wrong — and using a shovel when one really needs a crane — can be severe, in terms of cost and otherwise. See, e.g., In re Fannie Mae Sec. Litig., 552 F.3d 814 (D.D.C. 2009) (upholding contempt citation for failure to comply with deadline in stipulated discovery order).