With regulation – and document volumes – on the increase, the need for fast and efficient review systems has become more important than ever. Greg Wildisen reports

One inevitable consequence of the credit crunch is that the weight and scope of regulation facing companies will increase.

Even before the current crisis, regulators of all kinds – from the Financial Services Authority and Serious Fraud Office, through to the European Commission – had already been given enhanced powers to investigate potential corporate misfeasance.

The credit crunch has persuaded the government to give these bodies yet further powers – and provide a framework that can be seen to attempt to provide economic stability. For lawyers and their clients, this presents a significant logistical challenge.

One of the more onerous tasks faced by litigation lawyers has been the disclosure process. Regulatory lawyers have faced similar challenges with document review. Over the past few years the growth of electronic documents has made the process increasingly unwieldy, potentially inaccurate and, ultimately, more expensive.

While technology has enabled the cost per unit of reviewing a document to fall in recent years, the number of documents requiring review has increased exponentially as the transfer of documents from paper to electronic form has seen volumes mushroom.

At the same time, the growth of email has created a whole new set of 'documents' to review.

Where, in the past, reviewing 50,000 documents would have been considered a big project, it is now not uncommon for the number of items involved in reviews to run into the millions. This problem has become particularly acute in respect of regulatory proceedings.

In litigation, the time available for parties to disclose documents to the other side can be months, or even years. But regulators barely give their targets time to think, commonly imposing a 30 day time limit for companies to produce the evidence they are looking for.

Moreover, in a dispute both parties will often have a good idea at the outset in which set of documents the smoking gun is likely be found, while proportionality rules limit the scope of the disclosure process. By contrast, regulatory investigators are far less likely to be able to define the parameters of the search and are therefore usually much more wide-ranging in their scope.

So how would you cope if the regulator knocked on your client's door?

Work smarter, not harder

The key to coping with the sudden information request from a regulator is to pare down the document set as quickly as possible without sacrificing accuracy. In many circumstances, manual review is still necessary to determine the relevance and privilege status of documents. Clearly, though, reading every document is impossible when there are tens or hundreds of thousands to be reviewed.

Despite the explosion in volumes, cases still tend to turn on a small proportion of documents – typically no more than 5% of the sample.

It is imperative to quickly identify the documents that are likely to contain the relevant information, therefore significantly narrowing down the range of what needs to be reviewed manually. Modern electronic review systems can achieve this end by utilising a number of techniques.

The simplest form of electronic document review is keyword searching. This has become increasingly accurate as the number of documents that are originated electronically has increased.

Previously, the original paper documents had to be scanned and read by OCR (optical character recognition) systems, creating increased scope for error. Now that almost all business documents are created as word processor files, spreadsheets or emails, keyword searches are almost 100% accurate, making them very effective at picking out those documents that are most likely to contain relevant information.

Further refinement of the document set can then be achieved by running de-duplication search and near-duplication searches, which remove multiple copies of identical and very similar documents.

On its own, however, a keyword search only returns documents contain-ing those exact terms and it is unlikely that a search set will be exhaustive. More sophisticated review systems can now also cluster documents together by using 'meaning based' or 'concept' searches.

These apply linguistic algorithms to identify the 'meaning' of a document by comparing it with a set of specimen documents. If, for example, a reviewer finds a particularly relevant document, rather than looking for similar documents by running keyword searches, it is possible to use the document itself as a basis to find others like it, even if they don't contain the same keywords.

The ability to use mathematical concepts and auto-categorisation allows the reviewers to identify the relevant documents with a higher level of precision and greater confidence than a simple keyword search.

An advanced search can also automatically classify documents, meaning that review teams will no longer need to look at each document before attempting a consistent classification.

Another useful electronic review tool is email threading, which identifies related emails and alerts reviewers if messages in a chain are missing. Again, this saves time by automatically grouping related documents. It also enables lawyers to spot wrongdoers attempting to hide or disguise their trails.

Meanwhile, a common issue for reviewers is the plethora of different languages that many documents (especially emails) are often written in. Advances in character recognition technology means that sophisticated electronic review systems can quickly identify the predominant language of a document – whether that language uses the Roman or any other alphabet or character set – and assign it directly to reviewers with the appropriate language skills.

Each of these applications saves a good deal of time and effort, but the aggregated effect of utilising these techniques is dramatic. When faced with a huge volume of documents and a tight timescale, sophisticated review techniques are becoming increasingly essential..

Greg Wildisen, a qualified solicitor, is the International managing director at Epiq Systems.