How to Penetrate and Search the Invisible Web

For most of our online activities, a general searchinformation retrieval expert.
engine like Google does the trick. But whenThe Invisible Web Directory ( also from Gary
researching obscure or complicated topics, beingPrice and search expert Chris Sherman, is a
able to effectively penetrate the invisible web willdirectory of searchable databases, categorized by
take you places no automated web crawler evertopic.
could.InfoMine ( offers a mind-boggling number of links
What is the invisible web?and databases, and is maintained by the University
The invisible web (also known as the "deep" orof California, Riverside.
"hidden" web) is the term ascribed to the vastAnother academic site is the SJSU Academic
amount of information stored online that isGateway ( from San Jose State University; it'll
inaccessible to regular search engines. This isget you into the SJSU library and San Jose public
mostly stored in databases and alternative filelibraries.
formats, as opposed to basic, public htmlThe Virtual Library ( has annotated subject links,
websites. It is, by some estimates, 500 timeswhich can help cut down superfluous searching.
bigger than the "visible" web. In other words,Complete Planet ( claims access to "over 70,000
when you search using MSN or Yahoo, you're onlysearchable databases and specialty search
drawing from 1/500th of the content out there.engines."
This leaves a lot to explore!Another comprehensive guide is WebData ( which
Why is it invisible?offers browse and search options.
There are several reasons why certain contentThe Librarians' Internet Index ( has a collection of
stays hidden. Pages with no links to other pagesover 20,000 quality sites and offers a weekly
(that is, no "backlinks" or "inlinks") may preventnewsletter with updates and relevant links.
web crawling programs from retrievingThe Educator's Reference Desk ( presents access
information. Sites that involve registration orto 3,000-plus educational resources, which you can
typing in a query also bar surface searches fromperuse by category.
entering (web "spiders" just aren't that smart).The Education Resources Information Center (
Content that is scripted or non-text generallyprovides free access to more than 1.2 million
won't come up either-though Google does accessbibliographic records of journal articles and other
PDF and DOC file formats and offers these aseducation-related materials and, if available,
HTML views.includes links to full text.
How do you access it?OAIster ( provides one-stop "shopping" for users
The easiest way to breach the invisible barrier isinterested in useful, academically-oriented digital
by starting with search engines put together forresources. It gathers all potential digital resources
this purpose. Many of them provide links toavailable in an effort to build a comprehensive
databases that are likely to be useful for yourdigital union catalog.
topic; others are put together by academicFindArticles ( now searches over 10 million articles
institutions, which generally provide high-qualityfrom academic, industry, and general interest
results. Below is a list of the tried-and-true thatpublications.
guarantee to help you get below the surfaceMagPortal ( is another great tool for searching
(some may require that you pay a fee tomagazine articles online.
subscribe to services).Remember that these are just a few of the
Turbo10 ( connects you to niche-specific searchmany, many search engines that specialize in
engines, and enables you to access databases"hidden web" searches-not to mention the
from various levels of government, businesses,hundreds of thousands of databases you can
and universities.search directly once you know where to find
Direct Search ( is an excellent compilation ofthem. You may find that once you go "deep,"
specialized search tools by Gary Price, anthere's no going back!