The Crazy World of Web Advertising

auction-hammerLately you might have noticed that while you are browsing the web that you are experiencing a big delay in loading some web pages. A web page will pop up on your browser, but then it will sit for a while before it lets you navigate the page. The delay can last for many seconds and always feels longer than it is.

These delays are due to web advertisers. Web sites contain two kinds of advertising. There are embedded ads that a web site owner puts onto their site and lock. Embedded ads cannot be accessed or changed by an outside party and are integrated into the web page. But there are also remnant ads, which are ads that fit into blank spaces left for that purpose by the website owner. It is the process of putting ads into these remnant spaces that is causing the delay in loading pages.

There are a lot of companies that sell advertising into the remnant ad space including: Google (Doubleclick), Yahoo, Amazon, Facebook, AOL, AppNexus, Openx, Adroll, RightMedia, and dECN. The whole process is fascinating and is not much talked about outside of the advertising world.

These companies compete to put their ads into the remnant spaces. Each of these companies sell internet advertising and the remnant ads are where they are able to place most of their ad inventory. I always assumed that the web advertisers made deals with the popular web sites to place ads on those pages. But it doesn’t happen that way at all.

When somebody puts a remnant ad space on a web site it is open real estate and any advertiser that gets access to that web site gets to put advertising into the slots. So all of these advertisers play a game of real estate grabbing to try to be the first company to get to the remnant ad space.

Some advertisers dominate this market because they are associated with a web service that has lot of eyeballs. For example, if you go to a web page through the Google search engine or through another Google site like YouTube, then Doubleclick is going to grab the opportunity to place the ads. This is why the Google search engine is so lucrative. They charge companies to get a top ranking from searches, and they make more money placing ads on those sites that contain remnant ad space.

But the same is true for other large companies. If you go to a web page from Facebook or Yahoo or AOL then those companies are first in line to grab the remnant ad spaces. So a web site will allow any ad companies depending upon how the customer came to be at that site. As an aside, this is why it’s big news when some web service says they are changing browsers, because it means a big shift of dollars between these advertisers.

But web pages don’t run slow because of the primary advertiser on the web page. If Doubleclick gets to a web site and has enough ads in its inventory to fill all of the ad spaces the process is fast and nearly imperceptible to the end-user. The delays happen when the primary advertiser doesn’t have enough ads to fill all of the remnant ads on a given page.

Ad companies will only place ads that they have pre-sold. Since there are billions of requests per day to fill remnant ad spaces there will be many times when an advertising company doesn’t have enough ad inventory to fill a given page at a given moment. At that point, the first advertiser in line will fill the slots they can, and then they will send the ad to what is called the ad exchange. This is a consortium of all of the web advertisers.

At the ad exchange the remaining open remnant ad spaces are offered at auction to the highest bidder. Money is exchanged from these auctions through a process called the ad trading desk. So let’s say in this example that OpenX purchases the open slots on the web site in question from the ad auction. That gives them temporary control of the web site and they place ads in the remaining slots.

But if OpenX doesn’t have enough ads to fill all of the remaining slots, it goes back to the ad exchange again and there is another auction. Web pages can get stuck in this process and keep going back for auction, and that is when you will see a really big delay before you can navigate the page. But even when there are only a few advertisers involved the delay can be a few seconds.

There is one ugly part to this process in that the prices in the ad exchange can get really low, as in having four or five zeroes in front of the price. These low prices have lured the guys who push malware to the ad exchanges. These are the same guys who try to spread malware through spam. They will buy really cheap ads and then attach their malware to the remnant ad space. This is really insidious because an end user doesn’t have to click on the ad to get the malware – just loading the web site is enough to give them the malware. At the end of 2014 Cisco identified this ad malware, which is now called malvertising, as the second biggest source of malware in the web ecosystem.

The Explosion of Malware

virusIt seems the on-line world is getting more dangerous for end-users and ISPs. Numerous industry sources report a huge increase in malware over the last two years. AV-Test, which tests the effectiveness of anti-virus software says that their software detected 143 million cases of malware, up 73% from the year before. In 2012 they saw only 34 million. Over the last two years they found more malware than in the previous ten years combined. Another security software vendor, Kaspersky said that it saw a fourfold increase in mobile malware last year.

What’s behind this exponential increase in malware? Experts cite several reasons:

  • This is partially due to the way that antivirus software works. It generally is designed to look for specific pieces of software that has been identified as being malicious. But hackers have figured this out and they now make minor changes to the form of the software without changing its function to get it to slip past the antivirus software.
  • Some hackers are now encrypting their malware to make it harder for antivirus software to detect.
  • Hackers are now routinely launching waterholing attacks where they create a denial of service attack against a website for the purpose of infecting it with malware, which they then hopes spreads from there.
  • It’s getting easier for hackers to obtain the code of malware. It’s published all over the web or is widely for sale giving new hackers the ability to be up and running without having to develop new code.
  • There is a new kind of tracking cookie called a zombie cookie because it comes back after being deleted. The best known case of this is tracking being done by Turn which is putting this software on Verizon Wireless cell phones.
  • Malware is being delivered in new ways. For instance, it used to be mandatory for malware to somehow be downloaded, such as downloading an attachment from spam. But in the last few years there are new delivery methods like attaching malware to remnant ad space on web sites that download automatically when somebody opens a popular web page. Cisco just warned that they see social media being the newest big source of malware in 2015.
  • Malware isn’t just for computers any longer. Cisco warms that the biggest new target for malware this year is going to be cell phones and mobile devices. And they believe Apple is going to be a big target. Cisco and others have been warning for several years that the connected devices that are part of the early Internet of Things are also almost all vulnerable to hacking.
  • Due to dramatic cases where millions of credit card numbers and passwords have been stolen hackers now have reasons and to target specific people to do things like empty their bank accounts and don’t always attack the public at large.
  • Cyber-warfare has hordes of government hackers from numerous countries unleashing malware at each other and the rest of us are often collateral damage.

The scary thing about all of this is that the malware purveyors seem to be getting ahead of the malware police and there seem to be a lot of malware that isn’t being caught by antivirus programs. This has always been a cat and mouse game, but right now we are at one of those dangerous places where the bad guys are ahead.

Larger businesses have responded to the increase in malware by having malware attack plans. These are step-by-step plans of what to do during and after an attack on their systems. These plans includes a lot of common sense ideas like backing up data often, making sure all software is licensed and up to date, and even little things like making sure that there are hard copies of contact information for employees and customers should systems go offline.

But there really is no way to plan for this on a home computer and if you get infected with bad enough software you are going to probably be paying somebody to clean your machine. It’s hard to know what to do other than maintaining a virus checker and backing up data.