The law firm panel starts with good intentions: the GC of a large company looks at the number of firms retained each year, realizes how fragmented the company's spend is and reacts by kicking off a firm consolidation initiative. Everybody jumps on board, sweeping in the head of legal operations, senior lawyers, support staff, and perhaps finance and procurement. Several months (or a year) later, the legal team anoints a “preferred provider panel” of a smaller number of firms, and looks forward to deepening relationships, marked by eager and responsive service and across-the-board savings.

According to the latest data from the GC Thought Leaders Experiment, that's not how things typically work out. To the contrary, looking at law firm performance on over 1,400 actual legal matters for 28 large companies, law firms on client panels don't appear to excel. While panel firms perform well on a client's legal matters, they don't outperform the same client's non-panel firms. In the graph below, we see that non-panel firms perform slightly better than panel firms on all key criteria.

While the statistical significance of these gaps is minimal, the overall trend of overperformance by non-panel firms can't be ignored. At the very least, it's clear that panels are not delivering on their promise.

We also compared clients with panels to clients that don't have panels at all. The results there were surprising too: clients with panels do not see meaningful differences in outside counsel performance for their matters (on quality, cost-efficiency, responsiveness, solutions focus, and the like) as compared to clients without panels.

Taking all this together, the data fail to show outsized (or quite frankly any) returns from panels. This news has to be dispiriting for those looking to address law firm expense, service and quality issues by consolidating spend with a select set of providers.

Spoiler alert: before you despair—we believe in the power of panels, and, as discussed below, we have an idea of what separates the successful ones from the rest.

How could panels be yielding such poor returns? The received wisdom around consolidation seems so clear. In fact, it is partly why AdvanceLaw exists (we vet firms for a community panel serving 200 GCs), and as discussed elsewhere, we've helped many GCs with their panels.

By looking further into the data, we've found that the problem isn't panels, per se.  Instead, it's about poor choices and execution when creating and managing panels. This article focuses on issues relating to panel creation; our next article focuses on panel management (i.e., what happens after the panel has been formed).  

One panel creation mistake stems from the widely held view that increasing spend with a firm drives better results from that firm. The logic is enticing: such a firm will understand the client's business better and will have a lot to lose if fired. But what about the countervailing force—that clients sending a disproportionate share of their work to a firm will find it more difficult to replace that firm, and so the firm may be less worried about losing the client's business?

In the chart below, we look at the strength of the correlations between concentration of legal spend in a firm (whether on a panel or not), and each of our performance metrics—positive correlations rising above the line, and negative ones sinking below.

On the vertical axis, +1 and -1 are the maximum possible correlations. If you're having trouble reading the chart, it's because of the data, not your eyes. These are extremely weak correlations. Put simply: on the whole, consolidating spend doesn't help or hurt much.

We think this is important, as GCs often presume that increasing spending with a firm will naturally generate better results. (And while the relationships in the above chart are tiny, the most pronounced among them suggests that, if anything, cost-effectiveness suffers—not improves—as clients consolidate their legal spend.)

Some of this happens because of high switching costs when a firm receives a disproportionate share of a client's work, as noted in this American Lawyer article (on the performance of different law firms, and how law firms can best generate client loyalty).  

The point is not that fragmented spend and relationships is the answer. It is just that there is a need to strike the right balance. And based on our experience, some GCs create panels with too few firms—running the risk of overcommitting to the firms, which can become complacent without a credible threat of being replaced by the client.

Worse, when clients select too few firms for their panels, they often need to select the largest (and most expensive) firms. This is because that's the only way to cover numerous legal practice areas and geographies with a small number of firms. And as the second major finding from the Thought Leaders Experiment revealed, the largest, most pedigreed firms may not deliver as well on solutions focus, responsiveness, efficiency, and a host of other criteria.

As Bill Deckelman, a Fortune 200 GC participating in this experiment, thoughtfully explained last week, some of these firms are now so large that they aren't particularly eager to overperform for any one client. So, while many pursue convergence to create efficiencies, GCs who select too few firms end up turning to firms that provide the least efficient service.

Panels have the potential to drive strong value (because firms get to know the client well, law firms have skin in the game and clients can more easily manage fewer firms). However, clients need to avoid making panels too small because: highly consolidated spend decreases clients' ability to switch firms if performance is weak, and clients will need to default to firms that may not be as eager.

Beyond problems inherent in small panels, there are three other common missteps we've observed in setting up panels with GCs (keeping in mind we'll discuss the management of panels in our next article):

• clients are not actively ensuring diversity of firm size and price point, even though lower cost firms on a panel can serve as an effective check on less efficient firms;

• clients don't always ensure there at least two panel firms that are strong in each practice area, which can drive firm complacency; and

• for understandable reasons, many legal teams rubber stamp incumbents (which can confer a sense of entitlement), rather than including promising new firms.

In sum, the way clients form panels contributes to complacency, such that the benefits of consolidation, based on the above data, are entirely offset. For a good discussion of how best to create a panel to maximize client (and law firm) value, see this white paper on convergence by our colleague, Dan Currell. His experiences are informed by our work with GCs, as well as by the market for Louis Vuitton and Hermes apparel.

But beyond how to set up a panel, there is the issue of how to manage and work with panel firms. We find that after expending so much time and energy creating a panel, most clients breathe a sigh of relief and assume (or hope) the results will flow. And the firms that were selected may feel they no longer need to compete each day for the client's work—after all, they just competed and won.

This relates to how clients should manage their relationships with firms (whether on a panel or not), and whether firms actually become complacent over time. When we looked at the performance of law firms on actual legal work, based on how long the firm has been working with the client, an interesting pattern emerged:

While new relationships tend to start off well, that warm glow seems to fade after an initial honeymoon period. Performance tends to bounce back over time (though not always permanently), but even then, the data may reflect what's known as survivorship bias. Matters on the right side of the chart reflect higher performance in part because of a thinning of the herd, as clients eventually stop working with some of the underperformers.

Now, the post-honeymoon dip (whether a firm is on a panel or not) may not seem like a large difference statistically, and it isn't for any of the individual metrics. But the overall trend showing declining performance on all measures is meaningful and telling. It reinforces that without effective management, firm complacency or underperformance is not only a possibility, but a likelihood.

In our next article, we look at how in-house lawyers can work with firms on their panels to offset or eliminate this pattern.

Firoz Dattu is founder and chair of AdvanceLaw and Aaron Kotok is managing director.