Why Google doesn’t just Index Everything?
Some search engines Index a lot more. There is stuff that Google or Bing might not show you. This is because google is family safe, shows content that is acceptable to a majority of users and does not care about a small amount of users who need something specific that is not usually considered either safe or worth indexeing.
Search engines also maintain a certain quality and a lot of websites are never shown in search as they might violate some of guidelines by these search engines. These websites include sites that show copyright content, that has a lot of DMCA violations, adult content that is possibly illegal in a few countries, research and journalism sites banned in some regions, movies and image websites that violate webmaster guidelines, unrecognized or rare language websites that google doesn’t recogonize and many more content that is not generally seen by everyone.
Deep web is is not dark web, these search engines have access to deeper parts of web, if you know what you are searching for and its not illegal, its not dark web.
Why should you even use any other search engines?
While the major reason to use any other search engine is viewing content not indexed by Google or Bing, there are some other reasons of their popularity.
A big reason is ANONYMITY, means that you do not want anyone to know what you are searching for, for an example, if you search for certain phrases like “how to kill someone?” on Google before doing that, you will get in trouble. This is absoluetely TRUE! there have been many reported cases that have been solved by google search of the criminal.
Another reason is searching specific document, image or video. Google is primarily a text based search engine. Even google image search seems to get relevant images from alt tags and content on the page along with the image itself, it is difficult to find pdf reports, certain videos and other content on google where text is almost non-existant.
There are some
- Duckduckgo – Just like google, exept the fact it doesn’t follow everywhere you go, your privacy is protected and your searches are gone as soon as you leave, it does not track your ip. Also there are some websites that are banned in countries and are not shown in google search, DMCA rewquests, government policy requests and more, can be seen here.
- Archive.org – Huge behemoth of media now public domain – rare books, sound recordings, video, 20 year archived images of all old websites. You can add your own website here and see an older version of your site years later from this archive.
- The WWW Virtual Library – Got some rare old articles, books and documents that you won’t find anywhere in google searches.
- Infomine – Infomine indexes information from databases, electronic journals, electronic books, bulletin boards, mailing lists, online library card catalogs, articles, directories of researchers, and many other resources. It contains information from various university website resources and a lot of libraries, you can search for information and a lot of references for that information from trusted sources.
- DeepWebTech– Here you can search deep web in 5 different search engines, from 5 different categories from medicine, science, buisiness which derives information from deeper parts of the web that googledoes not access.
- Wolfram Alpha – A collection of datbases. According to Wolfram “We aim to collect and curate all objective data; implement every known model, method, and algorithm; and make it possible to compute whatever can be computed about anything. Our goal is to build on the achievements of science and other systematizations of knowledge to provide a single source that can be relied on by everyone for definitive answers to factual queries.”
- Deeperweb.com This is my favorite search engine. It breaks your results down into categories
For Dark Web
- Duckduckgo – Probably the most widely used search engine used for deep web, due to almost google like capability and added anonymity. Duckduckgo claims not to use your ip adress to track any kind of information or data about you, and not store your IP address information that could be used against you.
- Not Evil
- Memex Deep Web Search Engine