Tag Archives: Bing

The Declining Search Engine?

ask-jeevesThere is a subtle battle going on for control of the web. The web as we have come to know it is built upon the search engine. Those who’ve been on the web long enough remember search engines like Archie, Excite, Aliweb, Infoseek, AltaVista and Ask Jeeves. Today Google dominates the search market along with others including Bing, Yahoo and DuckDuckGo. Search engines operate by the simple premise of ‘crawling’ through publicly available web spaces and categorizing web pages by topics that can be searched.

But we might have reached the point in the life of the web where the search engine will decline in value. This is because search engines rely on public content and an ever-increasing amount of our collective information is now being created for and stored in the dark web. This term refers to networks which use the public internet but which require specific software to access.

The best example of the dark web is social media sites like Facebook, Twitter and LinkedIn. There are huge amounts of content now created for and stored inside of these platforms. Facebook is the best example of this. There are now many businesses that no longer have a web site but which instead have a Facebook page. There are many organizations that do the same and that communicate with members only through a Facebook group. Every social media site has something similar. For instance, there are now thousands of articles that are written for and published within LinkedIn.

But the dark web includes a lot more than just social media. Corporations and trade associations now routinely keep information hidden from the general public by requiring password access to read whatever they have published. You can understand the motivation behind this – a trade association might have more luck recruiting members if membership includes access to unique content.

But corporations also hide a lot of content that used to be public. For example, cable companies like Comcast require a customer to enter a valid address before showing them current products and prices. Those prices used to be openly published, but the act of asking for an address now ‘hides’ this information from search engines. Comcast certainly wants this iformation hidden since they don’t offer the same prices everywhere.

While the information behind corporate and trade association web sites is already unavailable to search engines, for now the information behind most social media sites is still searchable. But there is nothing that requires it to stay that way. Google and Facebook are now engaged in a fierce battle to win web advertising and there is nothing to stop Facebook from flicking a switch and hiding all of its content from Google search. To do so would instantly devalue Google since businesses listed only within Facebook would disappear from the search engine results.

It almost seems inevitable that this day will come, and probably not long from now. Both Google and Facebook have requirements from stockholders to continue to grow and both businesses are fueled largely by advertising revenue. It’s hard to think at some point that Facebook won’t deliberately try to gain an edge in this battle.

But the consequences of the dark web to all of us an ever-increasing lack of information. I remember spending evenings in the early days of the web just browsing the web for interesting content. It seems like every college and high school pushed huge amounts of interesting content onto the web – because in those early days that’s what the web was all about. All of the content from textbooks and homework assignments were on the open web for anybody to read. But there is no longer any major motivation to push information to the web. And many schools and universities are now behind a dark web wall as well.

The web has shifted massively towards entertainment and content is no longer king. Most of the early content-heavy web sites have died since they’ve either been pulled down or are no longer maintained. And so every day more content is removed from the web, and every day search engines become a little less valuable to the human race.

We may already be in a world where there is more useful data on the dark web than on the open one. Facebook and a few others could push the search engines onto the road to decline. History has already shown us that search engines can come and go. Excite was one of the first web companies that sold for billions, but the company that bought it, @Home went bankrupt. There is nothing to say that the Google search engine or any other is something that we can always rely on. And in fact, they may just fade away someday due to irrelevance.

Search Engine Privacy

Keep OutIn this country we have developed an interesting relationship between privacy and the web. We don’t treat all web content the same. And we do things very differently than Europe and the rest of the world.

For example, recently over a hundred female celebrities had their personal photos hacked from Apple and links to the pictures quickly proliferated over the web. One has to assume that millions of people saw the pictures and still have access to sites that have copies of them. Yet, if you go to Google or Bing to try to find sites with these pictures you will not find them. You will find dozens and dozens of stories about the hacking, but the search engines have effectively killed links to the pictures. This doesn’t mean that the pictures don’t exist all over the web, but it means that the search engines have stopped listing any page that has them.

This is due to US copyright law and we very efficiently enforce copyright in this country. In the case of these celebrities it was quickly very clear that each of them had ownership of the pictures involved, and it is that ownership that allowed them to request a takedown request to the search engines. The search engines have processes in place for this kind of takedown request because they process thousands of similar requests every day from content providers such as Hollywood studios, musicians and photographers.

But suppose instead that somebody like a paparazzi had been able to take naked pictures of one of these celebrities. In that case the celebrity would be powerless in the US to get such pictures removed from the web because they do not own the pictures. They would be free to sue the person who took and posted the pictures, but such suits are expensive and often are not successful. And meanwhile the offending pictures would proliferate all over the web.

Also suppose that instead that somebody wrote something mean and slanderous about one of the celebrities. The celebrity would have no power to remove the written word from the web due to the fact that the writer has a First Amendment right of free speech to say whatever they like. Again, the celebrity could go after the author and possibly could get a judgment against them, but the celebrity does not have any way to get Google or Bing to pull the offensive content off the web, unless somehow the content was illegal in some way. The search engines are allowed to block illegal content like child pornography or anything else that is clearly illegal.

Contrast all of this to Europe where they have similar copyright laws and a person can ask for a takedown request for content that they own. But Europe now has much stronger laws in other areas as well. It’s been about half a year since Google began implementing the court order in Europe that requires it to remove personal information that people find harmful about themselves. The original court order came from a Spanish man who objected to having records posted that showed that twenty years previously he had not paid his property taxes on time. The original posting came from a newspaper that had been scanned into Google. The man did not dispute that the facts were incorrect, but instead argued that he had a right to get rid of things on a web search that hurt his reputation. And the courts agreed. This new law is being referred to as the right to be forgotten.

Europe looks at privacy very differently than we do in the US. There, the laws and the general sentiment of most people is that privacy is more important than free speech. This is largely the result of various governments in Europe’s history, from the Nazis to the countries behind the Iron Curtain that tracked their citizens closely and trampled human rights. The European reaction to that history has been to make the right to privacy a fundamental human right. Here, we instead have made free speech into a fundamental right, and this difference is reflected in how we each treat the web.

Google and other search engines have been put into an awkward position in Europe in that they are having to decide what content should or should not be blocked on their search engines. Google has gotten about 120,000 requests to delete web data since the passage of the law and has granted about half of them. The way that Google is handling this is rather clever. In Europe each country has always had their own version of Google and when people in Spain ask for Google they actually get Google.es. This is true around the world and Google operates subsets of the database in different countries so that Google can give a different priority in search results in each country.

So when Google gets a request to delete something, it only blocks it in the version of Google that is seen in that country. But the underlying document is not deleted in the larger database of Google.com and the information can still be found by searching there. Google says this works because 95% of the searches it gets in Europe come through the country specific Google and not through Google.com. Interestingly, Google is getting criticized for not being aggressive enough with deletions and there is talk of imposing more restrictions on them.

It is going to be interesting to see over time what this difference means. For now the process is very new and Google and Bing are still feeling their way. But over time one has to think that having two very different philosophies controlling web content is going to make a difference between the web here and the web there.