IN THE STACKS

Though general-purpose search tools are pretty big, there are large areas of the Web that their indexing software – called crawlers, spiders, bots, etc.- cannot access.



Crawlers index pages by following one hypertext link to another. If a source does not have a permanent address or for some reason blocks the crawler, that source won’t be indexed. Examples of this are:



* Searchable databases



* Dynamically-created Web pages



* Newspaper archives



* Non-HTML files



* Sites requiring a login



This content is called the “hidden Web,” the “Invisible Web” or even the “Deep Web”. Whatever it is called, it is sobering to consider some conclusions drawn by BrightPlanet.com in its July 2000 “White Paper,” available at www.completeplanet.com/Tutorials/DeepWeb/index.asp:



* The hidden Web (HW) is more than 500 times larger than the known, indexed portion of the Web.



* More than 100,000 HW sites exist.



* The HW is the largest growing category of new information on the Internet.



* About 95 percent of this content is publicly accessible and is not fee based.



Much HW content exists in the areas of business, law and government, and people, and is therefore of interest to attorneys. This article will cover some of the most valuable free HW content in these fields, as well as HW search engines and alerting services.

Business Resources

Three of the most popular Web sites featuring searchable databases of company information are Hoover’s Online (www.hoovers.com), EDGAR (www. sec.gov) and the U.S. Patent and Trademark Office (www.uspto.gov). They are all part of the HW because search engines do not index their information.



However, since these three sites are well known, they will not be discussed in this article.



A good source for company profiles – especially for privately held firms – is Business.com (www.business.com). The site provides detailed information, including officers, competitors, financials and news, for more than 10,000 U.S. public, 44,000 U.S. private, and 14,000 international, companies. You can search either by company name or ticker symbol.



If you need information on international companies, try Corporate Information (www.corporateinformation.com), an index to more than 350,000 company profiles worldwide.



To research a non-profit organization, visit GuideStar (www.guidestar.org). Search the database of 700,000 non-profit organizations by name, location or keywords. The data is obtained from the U.S. Internal Revenue Service.



For information on customer complaints, try the Better Business Bureau, Mid-Atlantic Region (www.easternpa. bbb.org/search.html). The searchable database provides access to BBB reports on companies located in the Washington, D.C., and Philadelphia metropolitan areas. Data is based on customer feedback that the BBB has received during the three previous years. Search by company name, address, or telephone number.



News sites are very useful when researching companies. You can search Bizjournals.com (bizjournals.bcentral. com/search.html) for articles in business journals covering 40 major U.S. markets – including Philadelphia – from 1996 to the present.



You can also use the Financial Times’ (www.ft.com) global archive, a unique free source containing more than 10 million articles from more than 2,000 European, Asian and American news sources. The database is updated on a 24-hour, seven-days-a-week basis – more than 1,200 articles are added every hour!



Finally, many state home pages allow you to search corporate records online. Corporate records provide date of incorporation, status (active or expired), name of registered agent, and address, among other things. They will generally be available from the Department of State. Unfortunately, Delaware and Pennsylvania do not offer online corporate records. New Jersey allows you to search records for free, but you must pay a fee to see the complete record.

Law and Government