The Story Behind the Red Ring of Death

The hardware fault that should have brought down a Microsoft brand… but didn’t

Published: Thursday, October 26, 2017 - 12:03

Nearly two decades ago, rising from the ashes of the once-giant video game hardware manufacturer Sega, Microsoft debuted the Xbox and entered into the video game market with the intent of competing directly with Sony’s PlayStation brand. By 2006, Microsoft’s launch of the second-generation of Xbox video game consoles actually one-upped Sony’s PlayStation 3, which was riddled with one baffling marketing decision after another. However, Microsoft would soon find itself on the edge of one of the most prolific and devastating hardware failures in consumer electronic history... only to ignore it for as long as possible.

The story begins back in 2001. Peter Moore, then president and COO of Sega of America, made the call on January 31 to end production of the popular but underperforming video game console Sega Dreamcast. The Dreamcast was Sega’s fourth video game system, famously released on 9-9-99, and after decades of competing with Nintendo for dominance, Sega fell victim to Sony’s PlayStation 2 (PS2), released at the end of 2000. The PS2 would go on to become not only the best selling console of its generation, but of all time; Sega would immediately exit the hardware business and become a third-party publisher of video games, allowing its flagship character, Sonic the Hedgehog, onto other consoles for the first time.

Ten months after the discontinuation of the Sega Dreamcast, Nintendo released its competitor to PS2, the Nintendo Gamecube, and Microsoft released its first video game system, the Xbox. Announced originally in March 2000, the system struggled through development and was met with heavy resistance from the industry. Despite ultimately holding only a 23-percent market share, the system was seen by the public as an innovator that pushed consoles into the future. It was the first to integrate a dedicated system for online game play, and was also the first to have a built-in hard drive, both of which are standard features on modern consoles today.

Still, despite having stronger hardware and more innovation, the Xbox was outsold by the PS2 five to one, costing Microsoft somewhere in the range of $4 billion. Frustrated with the defeat, Microsoft CEO Steve Ballmer hired Peter Moore in 2003 to help craft the next Xbox console, which would compete directly with Sony’s inevitable PlayStation 3 (PS3). Many saw this as a chance for Moore to redeem the Dreamcast’s defeat.

In Xbox Revisited: A Game Plan for Corporate and Civic Renewal, (Brown Books, 2015), author Robbie Bach, formerly  Microsoft’s president of entertainment and devices, says, “We would have been dead. The original Xbox, nobody loved the console. They like Halo [the most popular video game on the system] and Xbox Live, but the console was this big black thing that looked like a radiator and had the world’s largest game controller. It wasn’t something people were in love with.” Bach goes on to describe the first Xbox as a rehearsal for what they had planned next. This mindset became the strategy for the second Xbox console, dubbed the Xbox 360. Push the things the public liked, such as the first-party exclusive franchise Halo and the online service, Xbox Live, while revamping the image of the Xbox brand. The system would go from a video game console to an entertainment console, being the first to offer online video streaming with Netflix before it was commonplace on our phones. It was also the first to offer wireless controllers, now an industry standard. It supported HD graphics at a time when HD was just about to boom, and it was packed with memory and power to the point that the power supply had to be housed outside of the console to keep the system’s physical size down.

The last piece of the strategy was the release date: It would have to come out by November 2005, because that was when the world expected Sony to release the PS3. Microsoft knew that Sony had spent years (since 2001) researching and designing the heart of the PS3, with hundreds of engineers working on the project, and it represented a radical departure from the standard brains you’d find inside video game consoles to that point. Launching behind the PS3—as the follow-up to one of the most successful consoles of all time—was not an option for Microsoft. So, developers cut corners and rushed through the manufacturing process to meet Sony head on in November.

Microsoft hired third-party companies to make chips and hardware, seeing that as a cheaper and faster move compared to Sony, which did everything under its own roof at a high cost. The benefits were obvious, but the risks were high; when pieces of hardware come from so many different sources, dealing with defects becomes problematic and difficult to predict or fix in a timely manner.

In the end, the Xbox 360 launched containing 1,700 different parts from 200 different manufacturers, and was announced in May 2005, in an attempt to beat the PS3 to public awareness. It would be days later, at the annual Electronic Entertainment Expo (E3) that Sony would drop an industry bombshell: The PlayStation 3 wouldn’t be out until late 2006.

Microsoft found itself with a year lead over the competitor that had crushed it so badly the generation before.

The seventh generation of video game consoles. From left, Sony’s PlayStation 3, Microsoft’s Xbox 360, and Nintendo’s Wii

Though the public perception was positive, behind the scenes there was great concern. The manufacturing process was not going well. According to VentureBeat:

“The problems began before the Xbox 360s got to the market. The testing machines were not ready, and the battery of tests that they ran were not fully developed. That meant the testing machines would inspect the Xbox 360s coming off the line and approve them for shipment, even though there were likely flaws.

“The test machines were not properly debugged, due to an ill-advised cost-cutting initiative that shaved $2 million from $25 million paid to Cimtek, a test-machine maker in Canada. The Microsoft team decided not to pay the consulting fee to Cimtek to build, manage, and debug the test machines. Sources familiar with the matter said there were only about 500 test machines at the time of launch, a third of the 1,500 needed.

“‘There were so many problems, you didn’t know what was wrong,’ said one source about the test machines. ‘The [test engineers] didn’t have enough time to get up and running.’”

By August 2005, three months before launch, Microsoft engineers reported a plethora of issues with the system, almost all of which came down to one problem: heat. The system was packed with too much tech with little room to breathe, and the overheating issues were manifesting in various failed components. At this stage in production, thousands of units were expected to be created every week, not the hundreds they currently had. Beyond that, the defect rate for the systems coming off the line was an atrocious and unacceptable 68 percent... this meant that for every 100 units created, 68 were deemed unshippable, three months before launch.

Engineers pleaded with higher-ups to shut down production. They needed time to source the malfunctions, but ultimately, Bach and Moore made the decision to launch. As the date approached, the media got wind from retailers that supplies coming in for the system were well under what was ordered, the first sign of trouble to the general public. Despite this, Xbox 360 launched on Nov. 22, 2015, one year ahead of its competitor (even though the Xbox 360 began production after the PS3), at two price points: $399.99 or $299.99.

The system owned the holiday season that year. The 1.5 million units that were shipped could not meet the high demand. eBay prices skyrocketed as consumers became desperate, and the media cried foul that Microsoft had artificially created a shortage to raise brand prestige. The ironic truth is Microsoft did have a huge supply of Xbox 360 consoles that weren’t on store shelves, but they were all faulty.

By 2007, Xbox users began to dread seeing this display when booting up their system.

Microsoft could take the financial hit of not being able to sell as many of the consoles as were demanded, but the real punch to the gut came when news broke that the PS2 outsold the Xbox 360 in November, moving its 100 millionth unit. Still, the Xbox 360 soldiered on and was able to meet up with the demand as production continued. All the while Sony continued its infamous march toward the launch of the PS3, which included one insane decision after another. Bad public demonstrations of the system and a shocking $599 price point allowed the Xbox 360 to rest comfortably with its early lead. Moore famously told consumers that they could purchase both an Xbox 360 and a Nintendo Wii (Nintendo’s abstract entry into this generation of video game consoles) for the price of a PlayStation 3.

The story behind the scenes did not change, however. While Microsoft celebrated to the public that the Xbox 360 had shipped 3 million units, its warehouse inventory of faulty units had piled up to half a million. The concern about the longevity of the Xbox 360 units on the market was becoming a real thing, despite Moore downplaying it. Reports from consumers of overheating and glitches were becoming more frequent, but that was nothing compared to the coming storm.

In spring 2006, consumer Chris Szarek’s story made the news. He reportedly had three Xbox 360 units die on him, each one displaying three red lights on the front of the console’s LED design. The three red lights indicated a general hardware failure, but nothing more specific than that. It could have been any number of issues, and Szarek claimed to be just one of many who had experienced this problem. At the time, however, he was largely denounced as a crybaby, with the public focusing more on the fact that he was given free replacements from Xbox for his troubles. He was seen as simply unlucky.

Heading into the launch of the PS3 in November 2006, Xbox stayed on the attack. Microsoft announced two new SKUs for the brand to be released in 2007, both a higher-end unit and a lower-end unit, making the best effort to cover every corner of the market. Internally, this just meant that the engineers were always focused on the next thing, rather than perfecting the core unit that was already riddled with problems and faulty yields. Xbox was on a warpath to sell 10 million units before Sony, but by August 2006, Bach and team finally realized they had a problem they had to face head on.

The three red lights were becoming more prolific, to the point that the phenomenon was christened the “Red Ring of Death,” similar to the infamous Blue Screen of Death that plagued Microsoft Windows operating systems for years. Engineers struggled to find any kind of pattern or general fix for the problem, and the issue was becoming so big that Microsoft’s silence would be damning. In September 2006, two months before the launch of the PS3, Microsoft admitted for the first time that the initial launch units of the Xbox 360 were being returned at a rate higher than expected, and a program was launched to compensate owners of systems manufactured in 2005.

A good start (with a lot of spin), but merely a band-aid on a bullet hole. By that point the 2005 models represented roughly 1.5 million units out of the more than 5 million on the market, all of which were presumed to have an “expiration date” in the form of three red lights. Meanwhile production on the Xbox 360 wasn’t slowing down.

By December 2006, all three major consoles for this generation had launched, and the PS3 was crushed, selling fewer than 500,000 units compared to Microsoft’s 1.5 million. The celebration was bittersweet, however, as the PS2 sold a miraculous  1.4 million units despite having a replacement unit on the market, and Nintendo managed a shocking upset with the Nintendo Wii, which outsold both the Xbox 360 and the PS3. Still, Xbox had accomplished its goal: It was a serious contender in the video game world, and the landscape of the console market share was drastically different compared to the previous generation. If it weren’t for this early success, the Xbox brand likely would not have survived 2007.

A mere two days into 2007, Microsoft announced an extension to the 90-day warranty the Xbox 360 initially came with, and owners were now covered for a full year after purchase. It was also around this time that Xbox would suspiciously halt production of the core unit; speculation has been around for years that this production halt was not only to prep the upcoming launch of the higher-end system, but also to also give Xbox engineers time to finally suss out the red-ring errors. When the new SKUs launched months later, crafty consumers quickly noticed that while the systems looked the same on the outside, the inside had been rearranged and adjusted. The problem had finally been addressed... but it was too late for the 9 million units already shipped worldwide. Consumers now knew it wasn’t a matter of if their Xbox 360 failed, but when.

This kind of situation would have buried most companies. In May 2007, the Xbox team descended into Bach’s basement and had a “long and difficult” conversation. When they emerged, the decision was made: The Xbox 360 warranty would be extended to three years, with all expenses to the consumer (including shipping) covered. According to Bach, it was the biggest decision he made in his 22 years at Microsoft.

Moore recounted the situation on an episode of IGN’s Podcast Unlocked series. “If we don’t do this, this brand is dead,” he said. “This is a Tylenol moment. [Microsoft CEO Steve Ballmer] asked how much it would cost, and I always remember taking a deep breath, looking at Robbie and going, ‘We think it’s $1.15 billion dollars, Steve.’ He said, ‘Do it.’”

In July 2007, Microsoft formally announced the warranty extension. Any Xbox 360 unit that displayed the red-ring error would be covered for up to three years after purchase. Beyond that, the company also promised to reimburse consumers for any money spent repairing faulty units prior to the warranty extension. It’s considered one of the most expensive consumer electronic fixes in history.

However, Moore, who would leave Xbox the same year and join publisher EA Sports for the next decade before eventually becoming the CEO of Liverpool Football Club, later credited the three-year warranty to single-handedly saving the Xbox brand. By the end of 2007, reports came out that the Xbox would be profitable for Microsoft for the first time in the brand’s history, although this claim is highly debated, with recent reports suggesting the Xbox has never turned a profit for Microsoft. It would go on to become the sixth best selling console of all time, trailed by the PS3 at seventh.

Unfortunately, the “third console curse” that has plagued every major video game console manufacturer (Nintendo, Sega, and Sony all saw their third systems underperform) has struck the Xbox brand with its follow-up to the Xbox 360, humorously dubbed the Xbox One. Although an extended warranty and goodwill to the consumers can solve a hardware failure of epic proportions, and even save a brand if the parent company has deep enough pockets, Microsoft would prove that it’s harder to come back from blatant anti-consumer decisions, such as the ones it would make in the lead-up to the Xbox One. Sony took the opportunity to go on the warpath, correcting all of PR mistakes it made prior to the launch of the PS3 before it.

Despite Microsoft backpedaling before launch, consumers made their decisions early on. The PS4 is estimated to have double the sales of the Xbox One after nearly four years. Microsoft might have learned the lessons regarding hardware manufacturing and goodwill warranty extensions, but while dealing with those, it completely ignored the lessons Sony was learning pertaining to marketing a product with the consumer’s best interests in mind.

Also, some internet users owe Chris Szarek an apology.

References
“Xbox 360 defects: an inside history of Microsoft’s video game console woes” (VentureBeat)
“Moore’s Law: What Does Peter Moore Think of the Xbox 360’s Past, Present, and Future?” (EGM, archived)
“Podcast Unlocked 201: Xbox Bosses Past and Present Share Stories, Secrets” (IGN)
History of the Xbox 360 Red Ring of Death (SSFF)

About The Author

Christopher Martin

Christopher Martin is an account manager at Quality Digest and a freelance journalist in his nonexistent spare time. With roots in covering the entertainment industry, he has expanded his reporting to include the ever-growing and ever-important role of quality management in everyday life.