The first time I heard the word "web," I immediately thought of spiders. Aren't they the original creators of the web? A web is described as an intricately woven network of silken, sticky threads created by a spider. It's like a network of interconnected mesh that entangles and traps its prey. Sounds pretty much like the web, doesn't it? Maybe that's what the creators of the World Wide Web had in mind prior to adopting the name. So what is the "web," and who is its creator? What is the "deep web," why is it deep, and who has access to it? My research on this topic will endeavor to enlighten and inform the reader and provide insight on the subject and intended purpose of įŗhe "deep web."
The World Wide Web was invented by English scientist Tim Berners-Lee in 1989. He wrote the first web browser in 1990 while employed at CERN in Switzerland. In March 1989, Tim Berners-Lee issued a proposal to the management at CERN for a system called "Mesh" that referenced ENQUIRE, a database and software project he had built in 1980, which used the term "web" and described a more elaborate information management system based on links embedded in readable text. This seems to parallel my earlier description of a spider's web.
The deep web exists because of the method that the majority of search engines use to search the WWW. Search engines are websites whose primary purpose is to enable people to find information on the web. These devices and their related software have the daunting tasks of indexing or attempting to index the entire WWW. All search sites create and maintain their own enormous searchable databases. Contents on the deep web are not accessible through search engines such as Google and Yahoo. The information on the deep web is special and must be protected against this automated access. The deep web, or invisible web, contains an enormous amount of valuable information but it's difficult to find. That is because search engines use web crawlers that follow hyperlinks to find information on the web, but they do not crawl to what is beneath the surface: the databases and other deep data. Wikipedia, in describing the deep web, estimates that it consists of about 91,000 terabytes (hard drive space) of information. The surface Web consists of about only 167 terabytes, less than 1 percent of the Deep Web. Recently calculated estimates by Completeplanet.com estimate the surface at 1 billion documents, while the deep web is estimated to be a mind boggling 550 billion documents.
Due to the sheer volume of information on many sites today, Webmasters, or web administrators, must use database technology to maintain and skillfully control their sites. These databases are hidden from search engines. Deep web content is relevant to every information need and market. At least one-half of the deep web information is found in topic-specific databases and is accessible to the public with no fees or subscription to pay. In order to access the deep web, a number of sites have been created to access the numerous amount of online databases or directly search them. One such site is Libraryspot.com, which contains links to more than 2,500 libraries around the world. Another site is Beaucoup.com. This search site has links to more than 2,500 databases and directories.
At the present, deep web information can only be located by using some type of direct query, and not by Google or Yahoo. The only way to search the deep web is to use a type of directed query, such as Lexibot, or the related website Completeplanet.com.
Deep web research has certainly provided me with information that has proven to be both invaluable and "deep."