MEMEX IS INDEXING THE DARK WEB
https://hacked.com/memex-indexing-dark-web/
by CARTER GRAYDONFEBRUARY 19, 2015
TweetLink ShareDiggPostStumbleuponWeiboG+Pinterest
The dark web sounds foreboding, but it refers to the parts of the internet not crawled and indexed by popular search engines like Google. Sites aren’t indexed for a number of reasons and Google only indexes 10 percent of the Internet This leaves the majority of the Internet falling into the category of the “dark web”.
The dark web currently works more like the surface web did back in 1999. You either find the site you’re looking for by following a link on someone else’s website, or you use a crude search engine. That might be about to change.
Memex Crawls the Dark Web
Defense Advanced Research Projects Agency (DARPA), has released its own search engineto crawl the dark web links in hopes of combating human trafficking. Memex searches and indexes websites that traditional search engines do not, and serve up the results graphically in order to help determine if there are any hidden links between them.
According to DARPA, human trafficking relies on forums, job postings, chat services and other mediums on the internet to attract customers. Before Memex, law enforcement agencies had to search for listings made by sex traffickers, create a list of suspicious URLs and then analyze them to detect any patterns that would suggest a trafficking ring. With Memex law enforcement can now analyze these networks using visual displays to show relationships between web pages.
For instance, if an investigator were to find a suspicious posting, they could run a search with the phone number posted as well as a name or location. Memex would return a diagram showing links to other ads containing the same name, phone number, location or a mix of both. Unlike Google, Memex also searches images, and latitude and longitude coordinates encoded in photos. It can read numbers in images, and can recognize the backgrounds in photos, which means it can tell if pictures of women share the same backdrop in their images.
Memex is unlike other search engines by going into greater depth and creating a graphical interface of results. Outside of the realm of fighting criminal activities, Memex could be used by stock analysts looking for more information on a company, it could help uncover fraud, and aid researchers looking for an exhaustive list of relevant information. It not only can detect patterns in massive amounts of data but can also fine tune its knowledge to specific domains with user feedback.
There are currently no accurate ways to measure the size of the dark web because most of the data is hidden or locked inside databases. Estimates put the size of the dark web between 400 to 550 times larger than the surface web. Because these places aren’t indexed, criminals can operate without easily being detected. As the adage goes, sunlight is the best disinfectant. However, as most things go, Memex could be used for good or evil. The same tools used to track down sex traffickers could be used by stalkers to follow their victims, trace and follow private data and further criminal endeavors.
Right now, Memex is free of charge for anyone who wants to download, distribute or modify it. But you can rest assured anything you search for is being collected and monitored by the government.
https://hacked.com/memex-indexing-dark-web/
by CARTER GRAYDONFEBRUARY 19, 2015
TweetLink ShareDiggPostStumbleuponWeiboG+Pinterest
The dark web sounds foreboding, but it refers to the parts of the internet not crawled and indexed by popular search engines like Google. Sites aren’t indexed for a number of reasons and Google only indexes 10 percent of the Internet This leaves the majority of the Internet falling into the category of the “dark web”.
The dark web currently works more like the surface web did back in 1999. You either find the site you’re looking for by following a link on someone else’s website, or you use a crude search engine. That might be about to change.
Memex Crawls the Dark Web
Defense Advanced Research Projects Agency (DARPA), has released its own search engineto crawl the dark web links in hopes of combating human trafficking. Memex searches and indexes websites that traditional search engines do not, and serve up the results graphically in order to help determine if there are any hidden links between them.
According to DARPA, human trafficking relies on forums, job postings, chat services and other mediums on the internet to attract customers. Before Memex, law enforcement agencies had to search for listings made by sex traffickers, create a list of suspicious URLs and then analyze them to detect any patterns that would suggest a trafficking ring. With Memex law enforcement can now analyze these networks using visual displays to show relationships between web pages.
For instance, if an investigator were to find a suspicious posting, they could run a search with the phone number posted as well as a name or location. Memex would return a diagram showing links to other ads containing the same name, phone number, location or a mix of both. Unlike Google, Memex also searches images, and latitude and longitude coordinates encoded in photos. It can read numbers in images, and can recognize the backgrounds in photos, which means it can tell if pictures of women share the same backdrop in their images.
Memex is unlike other search engines by going into greater depth and creating a graphical interface of results. Outside of the realm of fighting criminal activities, Memex could be used by stock analysts looking for more information on a company, it could help uncover fraud, and aid researchers looking for an exhaustive list of relevant information. It not only can detect patterns in massive amounts of data but can also fine tune its knowledge to specific domains with user feedback.
There are currently no accurate ways to measure the size of the dark web because most of the data is hidden or locked inside databases. Estimates put the size of the dark web between 400 to 550 times larger than the surface web. Because these places aren’t indexed, criminals can operate without easily being detected. As the adage goes, sunlight is the best disinfectant. However, as most things go, Memex could be used for good or evil. The same tools used to track down sex traffickers could be used by stalkers to follow their victims, trace and follow private data and further criminal endeavors.
Right now, Memex is free of charge for anyone who wants to download, distribute or modify it. But you can rest assured anything you search for is being collected and monitored by the government.