Posted on 2009-06-26

If the news of a pop star dieing is enough to cripple the biggest sites on the internet, what would happen if there was a truly important news item of global interest? This is a matter of the utmost urgency, what if something serious were to happen, and you couldn't find out about it?

This recent development has shown that in the event of a real emergency, the major news sites won't be able to cope with the upsurge in demand. CNET reports that Google, ABC News, CBS News and CNN all suffered drastically decreased performance, errors, and unavailability for hours last night. This is a fairly basic minimum service that these news organizations should be able to provide. During prior events, when CNN was swamped, they have reverted to a minimalistic text only version of their site, although this seems like a logical step, it was likely not necessary. The Infrastructure that was failing was most likely their basic HTTP service, not because there was insufficient bandwidth (this would be solved by removing the multimedia content), but because the system couldn't sustain sufficient concurrency (number of visitors being served at one time). Although removing the multimedia would have a slightly correlated effect on load (by reducing the number of requests from each visitor), the multimedia should be served from a separate part of the content distribution network so that it does not hamper the performance of the main content. In the event of an excessive load, priority can be given to the main content, rather than the additional multimedia content, and nodes from the media network can switch to serving the main content, increasing the capacity of the primary distribution network. The major news sites have the infrastructure to handle much greater load than they were subjected to last night, but are just not utilizing it effectively. Hopefully this incident services as a warning, and they invest some real time and energy in making their sites scale.

One proposed solution to this problem is a Peer to Peer based news system that would take over during major events where the main sites cannot handle the traffic. The problem I foresee with such a system is the inability to verify the source of the news. Unless CNN were to cryptographically sign the news item when placing it into the P2P system, and each P2P user had a copy of CNNs key to verify the message again (remember, CNNs website is down), there would be no way to prevent fake news. This is of particular note when you consider that after the announcement of two celebrity deaths last night, rumors started to take told that Jeff Goldblum and Clint Eastwood had died as well. If such news items were forged, or became accepted in such a P2P system, there would be no way to debunk them until the authoritative sites returned. The world has shown a lack of interest in a PGP/GPG style Web-Of-Trust system due to it being slightly difficult to manage and scale, so as far as I can see, for now we are stuck with the authoritative sources of information, so it would be nice if they didn't go down every time something of note happened. If one of the major news sources could manage to be the only one left standing during the next major event, it would be a major marketing coup, and I'd expect to see a major shift in market share towards the site that will be there when you need it.

By: Allan Jude