At the 2011 InsideCounsel SuperConference, a distinguished panel of judges engaged in a spirited roundtable discussion of e-discovery issues. This rare view of how judges see the costly and time-consuming process of e-discovery and their suggested solutions was moderated by Patrick Oot, special counsel, electronic discovery, for the Securities & Exchange Commission. The panelists were Cook County, Ill., Circuit Court Judge Peter A. Flynn; Judge James C. Francis of the Federal District Court for the Southern District of New York; Judge Nan R. Nolan of the Federal District Court for the Northern District of Illinois; and Judge David J. Waxse of the Federal District Court for the District of Kansas.

PATRICK OOT, moderator: These panelists are four of the leading judges in the area of e-discovery. We have four Ps related to e-discovery that we're going to talk about: preservation, proportionality, privilege and pilot projects.

We're hearing a lot of discussion in the corporate community on the burden of preservation. So Judge Francis, where are the courts headed on this?

JUDGE JAMES C. FRANCIS: Preservation, as you know, is a matter of when do you start preserving, what do you preserve, who's responsible for preserving it, and how do you go about issuing a litigation hold. One of the key problems now is really the 'when' issue.

Generally, the rule is that you start preserving when litigation is reasonably anticipated. It is an objective standard. That is, you look to see what a reasonable corporation would do in terms of anticipating litigation. And they take into consideration certain factors.

Does the potential litigant who's going to come against you have a history of litigiousness? Are they more likely to sue than the average Joe or not? Was there pre-litigation activity? Was there a grievance filed? Was there an EEO proceeding in an employment context? Did somebody say 'We're going to sue you?'

We, as judges, recognize that this is a costly and risky proposition for corporations and for litigants generally. The Civil Rules Committee [of the Judicial Conference of the United States] is beginning to look into different ways of dealing with it.

MR. OOT: Judge Nolan, how are you seeing proportionality in your courts?

JUDGE NAN R. NOLAN: We've gotten a lot of help from the Sedona Conference with their Cooperation Proclamation. Cooperation is no longer a snickering joke at conferences. This is really the way of the future for e-discovery.

When we're talking about cooperation in discovery, what we're really talking about are the parties actually talking about every one of the issues, starting with preservation.

They may sit down at a meet and confer and say, 'Hey Jay, what do you really need in order to prove your case?' And Jay could turn to Nan and say, 'Nan, what do you need in order to be able to defend your case?'

Now, where does the judge come in? I think the judge can actually encourage the parties to do it.

JUDGE FRANCIS: A lot of the time I see a knee-jerk reaction by the party that is going to be the predominant producing party against a preservation agreement or preservation order. I think that's just wrong-headed.

Defendants ought to embrace the idea of a preservation order because it not only requires them to preserve, but it identifies the limits of their obligations. They can do away with information that's beyond those limits.

Secondly, if you cooperate early on preservation, you can often get a cost-sharing arrangement. I've had a number of parties who have agreed to share costs with respect to preservation because they recognize it's important to keep some stuff, but there's a cost involved.

JUDGE DAVID J. WAXSE: It appears to me that too many parties in litigation ignore the fact that you may need information for your side of the case. They are always looking at what is the other side trying to get. You've really got to think of it both ways because your best piece of evidence may be something that you forgot to preserve.

JUDGE PETER A.FLYNN: There is no company that undergoes collective corporate Alzheimer's at the end of every day. You all have, independent of lawsuits, data preservation and data retention and data destruction policies. You need them. So preservation is not some sort of alien strangeness. Preservation is something that, in fact, you do.

How you preserve with reference to litigation depends in large measure on what you regularly do, in my view. We do not have a bright-line set of rules yet about what people have to preserve and how they have to preserve it. Frankly, I doubt that we ever will because no set of bright-line rules can cover the entire range of companies.

Let me give you one simple example of that. I had a case where a lawyer for a large company told me that they couldn't possibly retrieve those e-mails, it would cost a quarter of a million dollars to do it. I'm thinking to myself, this company either sues or get sued five times a week, at least. They're in that sort of business. They're telling me that they maintain their database in such a way that they can't find anything relevant to those disputes. I don't believe that. I just don't believe it.

So if the nature of your company is that it gets sued a lot or that it sues people a lot, telling a court that information isn't accessible to you is not likely to be real persuasive.

Another big red flag is a situation in which a company's conduct with regard to preservation suddenly takes a marked departure from the historical status quo for the company and it so happens that a lawsuit is filed the next week.

Well, we don't actually have to park our common sense at the courthouse door when we come to work. These things are examples of a common sense addressing of preservation issues.

As to proportionality, we are not as knowledgeable about the corporate internal expenses of data preservation as we'd like to be. I could make much better rulings if I knew really how much it costs to keep this database or that database, and what proportion of that cost is simply storage and what proportion is accessibility.

We can deal to some degree with a lack of cooperation among counsel. My favorite trick, which is real simple, is to tell the lawyers they can manage the discovery process in the case, or I can. Only a fool would want me to because I guarantee they're not going to like how I do it.

MR. OOT: Judge Waxse, you have an interesting example of how you use video to accomplish something that Judge Flynn has just identified.

JUDGE WAXSE: I've said to attorneys,'Have you attempted to resolve this by agreement and cooperation?' and you get the stock response:'I want to but that a**hole and blah, blah, blah.' And I've said,'We're going to try this again, but we're going to use a new technique. What I want you to do is have another session to see if you can reach an agreement on these discovery disputes, but this time, I want it videotaped, and you either send me an agreement or the video tape and then I'll decide who the problem is.' And I have yet to watch a video tape.

JUDGE NOLAN: If you [in-house counsel] demanded that your outside counsel read the Sedona Conference Cooperation Proclamation and you told them that this is how you want them to act, I would bet your legal bills would go down 50 percent.

I cannot tell you what is done in your name. You would cringe. When I say, 'Oh, for God's sake, what is going on here?,' many times a lawyer will say to me, 'My client is making me do this.' And I don't believe that because if you ran your corporation like this, you'd be out of business. This is downright silly, this kind of fighting.

JUDGE FRANCIS: To put a more optimistic spin on that, it seems to me that one of the benefits of the explosion of e-discovery is that it has forced lawyers into cooperation. I think lawyers more and more are recognizing that the costs of non-cooperation have gotten so immense that you can't afford to do that anymore.

JUDGE FLYNN: One thing the lawyers need to do is cooperate with their IT people. I hear an awful lot of stuff from lawyers that I have come to recognize as nonsense. The reason it's nonsense is either they never talked to the IT people or they didn't understand the answers they got. I've begun insisting that lawyers in discovery disputes bring the IT people with them to court. That way we can get some straight answers.

MR. OOT: Let's talk about privilege issues.

JUDGE WAXSE: I've been on a lot of panels recently where I've heard lawyers telling other lawyers that it may be legal malpractice to engage in production of documents without a clawback order because you've got this mechanism to avoid any waiver of privilege on attorney-client or work-product and why would you not use it?

Obviously, in certain litigation, a waiver can be disastrous for the client if something gets out. So clients are going to have to ask their lawyers, 'Do we have a clawback order in place and, if not, why not?'

JUDGE NOLAN: I think we over-label things as privilege. If so much did not get over-designated in the beginning, we would not have the problem at the end.

JUDGE FLYNN: Yes, there is over-designation of privilege. That is because the risk of over-designation is

a heck of a lot less than the risk of under-designation.

Our law of attorney-client privilege is utterly cock-eyed in two respects.

The first respect is that inadvertent disclosure of a privileged document is a 'gotcha.' Now, to my way of thinking, a justice system that has any self-respect doesn't do 'gotcha.' 'Gotcha' is for video games, not justice systems. But we have 'gotcha' rules in most of the states and they're picked up by the federal system by and large.

The second problem is that we also have this notion that if this document is privileged, then for any document that might be a cousin or second cousin twice removed or descendant from the same great grandparent of this document, there's also a waiver. And because of this kind of fruit-of-the-tree notion, you're always going to have over-designation.

That's a cock-eyed system and we need to change those rules. They will probably require legislative change. And it'll save everybody a lot of money and grief.

MR. OOT: I read an article recently in the New York Times, [“Armies of Expensive Lawyers, Replaced by Cheaper Software”]. It's really interesting because we're seeing a total change in the way people think about the document review problem and how people are producing documents in litigation. I'd like to ask Judge Flynn how he is thinking about these technologies in his courtroom.

JUDGE FLYNN: Let me give you two sentences from this New York Times article: 'Now thanks to advances in artificial intelligence, e-discovery software can analyze documents'—notice, it says not just find, but analyze documents—'in a fraction of the time for a fraction of the cost. In January 2011, for example, [a service provider] of Palo Alto, Calif., helped analyze 1.5 million documents for less than $100,000.'

The risk is that judges like me are going to read articles like this and when you come in and say it will cost a very large sum of money to retrieve those e-mails, we're going to say, 'Oh quit whining. You can go hire a provider and it won't cost you very much at all.'

Patrick suggests that The New York Times article, while conceivably prescient, is rather over optimistic. Patrick, can you comment on that?

MR. OOT: I'm a big proponent of search technology, but one thing we're forgetting is that year over year, we're seeing more than a 50 percent gain in data volumes. The challenge is that these search technologies are broad brush approaches. But how wide can you actually make the brush?

So the Electronic Discovery Institute, the non-profit that I co-founded with Anne Kershaw, is starting to study some of the analytics in the space. We recently did a report comparing document review technology to human review in the MCI acquisition by Verizon. Now Oracle has just donated a set of data that we're going use to conduct a similar analysis with seven different service providers. We're going to compare how they did to the original review.

JUDGE WAXSE: I want to remind you that technology is not perfect by any means, it's not as cheap we hope it is, but you've got to compare it to human review. And what amazes me is every study that's been done about human review says it is terrible.

Studies indicate a good human reviewer is lucky if they get 20 or 30 percent of what they're supposed to find in a set of documents. So if you can get technology that is cheaper and gets 60 to 70 percent, sure, it's not perfect, but it's better than the humans and it's cheaper than the humans.

But I constantly run into lawyers telling me, 'Judge, we're just not comfortable without human eyes on the documents.' And I keep asking, 'Where do you get comfort from having humans that every study shows can't do a very good job looking at a document?'

JUDGE FRANCIS: I hear counsel say, 'We don't know how the judges will accept this new technology.' I guess my answer to that is, have some faith in us.

I've drunk the Kool-Aid. I think there is really good reason to utilize the search technologies that are available now, even if they're not as perfect as they're going to be down the line.

JUDGE FLYNN: One thing you can do is ask the judge beforehand. Or ask your opponent.

JUDGE WAXSE: Yes. An agreement on what technology you're going to use will also solve this problem. And you can have huge cost savings if you share it. Put your data on separate databases, agree what technology you're going to use, and both sides can pick what they want the technology to search for. It's a lot cheaper.

JUDGE NOLAN: One part of my day job is serving as co-chair of the 7^th Circuit Pilot Program for E-discovery. One of our new sub-committees is a technology sub-committee. I think a lot of lawyers just don't know about the new technologies. I can send them to our website for information about that technology.

My job is not to say one is better than the other, or this is a preferable, but to be able to say, 'There are things out there that you may want to look at.'

At the 2011 InsideCounsel SuperConference, a distinguished panel of judges engaged in a spirited roundtable discussion of e-discovery issues. This rare view of how judges see the costly and time-consuming process of e-discovery and their suggested solutions was moderated by Patrick Oot, special counsel, electronic discovery, for the Securities & Exchange Commission. The panelists were Cook County, Ill., Circuit Court Judge Peter A. Flynn; Judge James C. Francis of the Federal District Court for the Southern District of New York; Judge Nan R. Nolan of the Federal District Court for the Northern District of Illinois; and Judge David J. Waxse of the Federal District Court for the District of Kansas.

PATRICK OOT, moderator: These panelists are four of the leading judges in the area of e-discovery. We have four Ps related to e-discovery that we're going to talk about: preservation, proportionality, privilege and pilot projects.

We're hearing a lot of discussion in the corporate community on the burden of preservation. So Judge Francis, where are the courts headed on this?

JUDGE JAMES C. FRANCIS: Preservation, as you know, is a matter of when do you start preserving, what do you preserve, who's responsible for preserving it, and how do you go about issuing a litigation hold. One of the key problems now is really the 'when' issue.

Generally, the rule is that you start preserving when litigation is reasonably anticipated. It is an objective standard. That is, you look to see what a reasonable corporation would do in terms of anticipating litigation. And they take into consideration certain factors.

Does the potential litigant who's going to come against you have a history of litigiousness? Are they more likely to sue than the average Joe or not? Was there pre-litigation activity? Was there a grievance filed? Was there an EEO proceeding in an employment context? Did somebody say 'We're going to sue you?'

We, as judges, recognize that this is a costly and risky proposition for corporations and for litigants generally. The Civil Rules Committee [of the Judicial Conference of the United States] is beginning to look into different ways of dealing with it.

MR. OOT: Judge Nolan, how are you seeing proportionality in your courts?

JUDGE NAN R. NOLAN: We've gotten a lot of help from the Sedona Conference with their Cooperation Proclamation. Cooperation is no longer a snickering joke at conferences. This is really the way of the future for e-discovery.

When we're talking about cooperation in discovery, what we're really talking about are the parties actually talking about every one of the issues, starting with preservation.

They may sit down at a meet and confer and say, 'Hey Jay, what do you really need in order to prove your case?' And Jay could turn to Nan and say, 'Nan, what do you need in order to be able to defend your case?'

Now, where does the judge come in? I think the judge can actually encourage the parties to do it.

JUDGE FRANCIS: A lot of the time I see a knee-jerk reaction by the party that is going to be the predominant producing party against a preservation agreement or preservation order. I think that's just wrong-headed.

Defendants ought to embrace the idea of a preservation order because it not only requires them to preserve, but it identifies the limits of their obligations. They can do away with information that's beyond those limits.

Secondly, if you cooperate early on preservation, you can often get a cost-sharing arrangement. I've had a number of parties who have agreed to share costs with respect to preservation because they recognize it's important to keep some stuff, but there's a cost involved.

JUDGE DAVID J. WAXSE: It appears to me that too many parties in litigation ignore the fact that you may need information for your side of the case. They are always looking at what is the other side trying to get. You've really got to think of it both ways because your best piece of evidence may be something that you forgot to preserve.

JUDGE PETER A.FLYNN: There is no company that undergoes collective corporate Alzheimer's at the end of every day. You all have, independent of lawsuits, data preservation and data retention and data destruction policies. You need them. So preservation is not some sort of alien strangeness. Preservation is something that, in fact, you do.

How you preserve with reference to litigation depends in large measure on what you regularly do, in my view. We do not have a bright-line set of rules yet about what people have to preserve and how they have to preserve it. Frankly, I doubt that we ever will because no set of bright-line rules can cover the entire range of companies.

Let me give you one simple example of that. I had a case where a lawyer for a large company told me that they couldn't possibly retrieve those e-mails, it would cost a quarter of a million dollars to do it. I'm thinking to myself, this company either sues or get sued five times a week, at least. They're in that sort of business. They're telling me that they maintain their database in such a way that they can't find anything relevant to those disputes. I don't believe that. I just don't believe it.

So if the nature of your company is that it gets sued a lot or that it sues people a lot, telling a court that information isn't accessible to you is not likely to be real persuasive.

Another big red flag is a situation in which a company's conduct with regard to preservation suddenly takes a marked departure from the historical status quo for the company and it so happens that a lawsuit is filed the next week.

Well, we don't actually have to park our common sense at the courthouse door when we come to work. These things are examples of a common sense addressing of preservation issues.

As to proportionality, we are not as knowledgeable about the corporate internal expenses of data preservation as we'd like to be. I could make much better rulings if I knew really how much it costs to keep this database or that database, and what proportion of that cost is simply storage and what proportion is accessibility.

We can deal to some degree with a lack of cooperation among counsel. My favorite trick, which is real simple, is to tell the lawyers they can manage the discovery process in the case, or I can. Only a fool would want me to because I guarantee they're not going to like how I do it.

MR. OOT: Judge Waxse, you have an interesting example of how you use video to accomplish something that Judge Flynn has just identified.

JUDGE WAXSE: I've said to attorneys,'Have you attempted to resolve this by agreement and cooperation?' and you get the stock response:'I want to but that a**hole and blah, blah, blah.' And I've said,'We're going to try this again, but we're going to use a new technique. What I want you to do is have another session to see if you can reach an agreement on these discovery disputes, but this time, I want it videotaped, and you either send me an agreement or the video tape and then I'll decide who the problem is.' And I have yet to watch a video tape.

JUDGE NOLAN: If you [in-house counsel] demanded that your outside counsel read the Sedona Conference Cooperation Proclamation and you told them that this is how you want them to act, I would bet your legal bills would go down 50 percent.

I cannot tell you what is done in your name. You would cringe. When I say, 'Oh, for God's sake, what is going on here?,' many times a lawyer will say to me, 'My client is making me do this.' And I don't believe that because if you ran your corporation like this, you'd be out of business. This is downright silly, this kind of fighting.

JUDGE FRANCIS: To put a more optimistic spin on that, it seems to me that one of the benefits of the explosion of e-discovery is that it has forced lawyers into cooperation. I think lawyers more and more are recognizing that the costs of non-cooperation have gotten so immense that you can't afford to do that anymore.

JUDGE FLYNN: One thing the lawyers need to do is cooperate with their IT people. I hear an awful lot of stuff from lawyers that I have come to recognize as nonsense. The reason it's nonsense is either they never talked to the IT people or they didn't understand the answers they got. I've begun insisting that lawyers in discovery disputes bring the IT people with them to court. That way we can get some straight answers.

MR. OOT: Let's talk about privilege issues.

JUDGE WAXSE: I've been on a lot of panels recently where I've heard lawyers telling other lawyers that it may be legal malpractice to engage in production of documents without a clawback order because you've got this mechanism to avoid any waiver of privilege on attorney-client or work-product and why would you not use it?

Obviously, in certain litigation, a waiver can be disastrous for the client if something gets out. So clients are going to have to ask their lawyers, 'Do we have a clawback order in place and, if not, why not?'

JUDGE NOLAN: I think we over-label things as privilege. If so much did not get over-designated in the beginning, we would not have the problem at the end.

JUDGE FLYNN: Yes, there is over-designation of privilege. That is because the risk of over-designation is

a heck of a lot less than the risk of under-designation.

Our law of attorney-client privilege is utterly cock-eyed in two respects.

The first respect is that inadvertent disclosure of a privileged document is a 'gotcha.' Now, to my way of thinking, a justice system that has any self-respect doesn't do 'gotcha.' 'Gotcha' is for video games, not justice systems. But we have 'gotcha' rules in most of the states and they're picked up by the federal system by and large.

The second problem is that we also have this notion that if this document is privileged, then for any document that might be a cousin or second cousin twice removed or descendant from the same great grandparent of this document, there's also a waiver. And because of this kind of fruit-of-the-tree notion, you're always going to have over-designation.

That's a cock-eyed system and we need to change those rules. They will probably require legislative change. And it'll save everybody a lot of money and grief.

MR. OOT: I read an article recently in the New York Times, [“Armies of Expensive Lawyers, Replaced by Cheaper Software”]. It's really interesting because we're seeing a total change in the way people think about the document review problem and how people are producing documents in litigation. I'd like to ask Judge Flynn how he is thinking about these technologies in his courtroom.

JUDGE FLYNN: Let me give you two sentences from this New York Times article: 'Now thanks to advances in artificial intelligence, e-discovery software can analyze documents'—notice, it says not just find, but analyze documents—'in a fraction of the time for a fraction of the cost. In January 2011, for example, [a service provider] of Palo Alto, Calif., helped analyze 1.5 million documents for less than $100,000.'

The risk is that judges like me are going to read articles like this and when you come in and say it will cost a very large sum of money to retrieve those e-mails, we're going to say, 'Oh quit whining. You can go hire a provider and it won't cost you very much at all.'

Patrick suggests that The New York Times article, while conceivably prescient, is rather over optimistic. Patrick, can you comment on that?

MR. OOT: I'm a big proponent of search technology, but one thing we're forgetting is that year over year, we're seeing more than a 50 percent gain in data volumes. The challenge is that these search technologies are broad brush approaches. But how wide can you actually make the brush?

So the Electronic Discovery Institute, the non-profit that I co-founded with Anne Kershaw, is starting to study some of the analytics in the space. We recently did a report comparing document review technology to human review in the MCI acquisition by Verizon. Now Oracle has just donated a set of data that we're going use to conduct a similar analysis with seven different service providers. We're going to compare how they did to the original review.

JUDGE WAXSE: I want to remind you that technology is not perfect by any means, it's not as cheap we hope it is, but you've got to compare it to human review. And what amazes me is every study that's been done about human review says it is terrible.

Studies indicate a good human reviewer is lucky if they get 20 or 30 percent of what they're supposed to find in a set of documents. So if you can get technology that is cheaper and gets 60 to 70 percent, sure, it's not perfect, but it's better than the humans and it's cheaper than the humans.

But I constantly run into lawyers telling me, 'Judge, we're just not comfortable without human eyes on the documents.' And I keep asking, 'Where do you get comfort from having humans that every study shows can't do a very good job looking at a document?'

JUDGE FRANCIS: I hear counsel say, 'We don't know how the judges will accept this new technology.' I guess my answer to that is, have some faith in us.

I've drunk the Kool-Aid. I think there is really good reason to utilize the search technologies that are available now, even if they're not as perfect as they're going to be down the line.

JUDGE FLYNN: One thing you can do is ask the judge beforehand. Or ask your opponent.

JUDGE WAXSE: Yes. An agreement on what technology you're going to use will also solve this problem. And you can have huge cost savings if you share it. Put your data on separate databases, agree what technology you're going to use, and both sides can pick what they want the technology to search for. It's a lot cheaper.

JUDGE NOLAN: One part of my day job is serving as co-chair of the 7^th Circuit Pilot Program for E-discovery. One of our new sub-committees is a technology sub-committee. I think a lot of lawyers just don't know about the new technologies. I can send them to our website for information about that technology.

My job is not to say one is better than the other, or this is a preferable, but to be able to say, 'There are things out there that you may want to look at.'