11-08-11 | Blog Post
Yesterday morning, Time Warner Cable experienced “a large but brief Internet outage” affecting most of their service areas, as they reported via Twitter. Supplying high speed data services to nearly 9.7 million residential and commercial customers, the outages were reported in Washington D.C., Los Angeles, San Francisco, Raleigh, N.C., Dallas and New York – and right here in Michigan.
Online Tech experienced an issue with our own Internet service from one of our many providers between 9-9:30 AM EST, and the outage affected only a small portion of the traffic coming to and from the data centers. Our operations team also received reports of a blip in a few of our clients’ service, but never received any alerts from both Online Tech’s internal and external monitoring systems. Additionally, it could not be traced back to our equipment.
Then we found out the outage affected nearly everyone. The outage affected multiple service providers and appeared to have occurred sometime after 6 AM PT, and is said to be resolved according to Time Warner Cable’s tweet.
Even the U.K. appears to have been hit with the “global outage,” according to Silicon.com – their own publishing site was affected, in addition to bit.ly, CBS interactive, and RIM’s Blackberry service. An article by BusinessInsider.com reports the outage lasted approximately 30 seconds, although some on Twitter claim nearly 20 minutes of downtime, as reported by TheBlaze.com.
According to TheRegister.CO.UK, some of the outages were a result of a problem with firmware in Juniper Network routers that corrupted BGP, or border gateway protocol, tables. Level 3 Communications, Time Warner’s transit provider, is reported to have housed the routers. According to Silicon.com, the outage has also affected other networks running Juniper routers with the majority of them seeing their devices core dump and reload. Level 3 has issued a statement:
“Shortly after 9 a.m. ET today, our network experienced temporary service interruptions across North America and Europe apparently due to a router software issue,” Level 3 said in a statement. “It has been reported that a similar issue may have affected other carriers as well. Our technicians worked quickly to address the issue and service is now fully restored.”
Juniper Networks has also acknowledged the routers had to be restarted when they crashed while performing a BGP update. They are offering a software fix and are currently working with their customers for deployment.
Time Warner Cable’s twitter is addressing individual issues, asking users to direct message them with their account information for assistance.
