Bumpgate: Difference between revisions
Fixed up the Intro section to be more brief and edited for clarity. (This article is a bit too specific/covers too old of an incident to show up in the anticipated extension, I think, but you never know.) |
SyrianClippy (talk | contribs) Added wii defects to the page Tags: Mobile edit Mobile web edit Visual edit |
||
| (5 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
{{IncidentCargo | |||
|Company=AMD, Nvidia | |||
|StartDate=2006 | |||
|EndDate=2010 | |||
|Status=Resolved | |||
|ProductLine= | |||
|Product= | |||
|ArticleType= | |||
|Type=Poor Quality | |||
|Description= | |||
}} | |||
[[File:Underfilled Die.png|alt=The image shows a diagram of a computer processor. On the bottom, there is a green rectangle labeled "substrate". On top of the substrate, there is a black rectangle labeled "chip", which refers to the die. Between the die and the substrate, there are small silver bumps equally spaced apart, encased within a white "filling". The bumps are the solder bumps connecting the die to the substrate, and the white filling is the underfill- meant to strengthen the solder bumps.|thumb|A diagram of a computer processor. When the underfill becomes too soft at any point in the processor's normal operating temperatures, the solder bumps under the die ("chip") can crack, disconnecting the die from the substrate. This leads to the processor failing, and in turn, leads to a critical system failure for the device it's in.]] | [[File:Underfilled Die.png|alt=The image shows a diagram of a computer processor. On the bottom, there is a green rectangle labeled "substrate". On top of the substrate, there is a black rectangle labeled "chip", which refers to the die. Between the die and the substrate, there are small silver bumps equally spaced apart, encased within a white "filling". The bumps are the solder bumps connecting the die to the substrate, and the white filling is the underfill- meant to strengthen the solder bumps.|thumb|A diagram of a computer processor. When the underfill becomes too soft at any point in the processor's normal operating temperatures, the solder bumps under the die ("chip") can crack, disconnecting the die from the substrate. This leads to the processor failing, and in turn, leads to a critical system failure for the device it's in.]] | ||
'''Bumpgate''' (also known as '''Nvidiagate''') was a scandal where [[Nvidia]] and ATI Graphics Processing Units (GPUs) made from 2005-2010 were prone to high failure rates because of a design flaw that caused cracked solder bumps under the die.<ref name=":0" /> | '''Bumpgate''' (also known as '''Nvidiagate''') was a scandal where [[Nvidia]] and ATI Graphics Processing Units (GPUs) made from 2005-2010 were prone to high failure rates because of a design flaw that caused cracked solder bumps under the die.<ref name=":0" /> | ||
Despite the name "Nvidiagate", this defect was widespread in the industry. It impacted many Nvidia GPUs (manufactured 2006-2010), but also affected ATI GPUs (2006-2008). Among retro console enthusiasts, the defect is best known to have caused the high failure rate of the ATI Xenos GPUs in [[Microsoft]]'s early Xbox 360 models<ref name=":3" /> (2005-May 2008) and the Nvidia-based RSX GPUs in [[Sony]]'s early PlayStation 3 models (2006-Fall 2008).<ref name=":4">{{Cite web |last=RIP Felix |first= |date=23 Dec 2022 |title=A PS3 Story: The Yellow Light of Death |url=https://www.youtube.com/watch?v=Za7WTNwAX0c |url-status=live |access-date=2 Jun 2025 |website=YouTube}}</ref> Retro computer enthusiasts may also know of the defect from certain models of Dell and [[HP Inc.|HP-Compaq]] laptops (2005-2010), as well as certain [[Apple]] Macbook Pros (May 2007-September 2008).<ref name=":16">{{Cite web |title=Affected Models |url= | Despite the name "Nvidiagate", this defect was widespread in the industry. It impacted many Nvidia GPUs (manufactured 2006-2010), but also affected ATI GPUs (2006-2008). Among retro console enthusiasts, the defect is best known to have caused the high failure rate of the ATI Xenos GPUs in [[Microsoft]]'s early [[Xbox]] 360 models<ref name=":3" /> (2005-May 2008) and the Nvidia-based RSX GPUs in [[Sony]]'s early [[PlayStation#PlayStation_3_Incidents|PlayStation 3]] models (2006-Fall 2008).<ref name=":4">{{Cite web |last=RIP Felix |first= |date=23 Dec 2022 |title=A PS3 Story: The Yellow Light of Death |url=https://www.youtube.com/watch?v=Za7WTNwAX0c |url-status=live |access-date=2 Jun 2025 |website=YouTube}}</ref>, Nintendo Wii consoles produced before 2008 are also affected by this issue but only if WiiConnect24 was active in standby mode, as this caused the Scarlet ARM CPU in the SoC to get hot therefore damaging the solder bumps when the fan is not on , Also Retro computer enthusiasts may also know of the defect from certain models of Dell and [[HP Inc.|HP-Compaq]] laptops (2005-2010), as well as certain [[Apple]] Macbook Pros (May 2007-September 2008).<ref name=":16">{{Cite web |title=Affected Models |url=https://www.nvidiasettlement.com/affectedmodels.html |url-status=dead |archive-url=https://web.archive.org/web/20101001080616/http://www.nvidiasettlement.com/affectedmodels.html |archive-date=1 Oct 2010 |access-date=7 Jun 2025 |website=The NVIDIA GPU Litigation}}</ref> | ||
==Background== | ==Background== | ||
| Line 13: | Line 24: | ||
In 2006, IBM and Amkor published a study that explained that use of a low T<sub>g</sub> underfill was not acceptable with high-lead solder bumps, and high T<sub>g</sub> would be necessary to avoid defects.<ref>{{Cite journal |last=Ray |first=S. |last2=Kiyono |first2=S. |last3=Waite |first3=K. |last4=Nicholls |first4=L. |date=2006 |title=Qualification of low-K 90nm Technology Die with Pb-free Bumps on a Build-up Laminate Package (PBGA) with Pb-free Assembly Processes |journal=56th Electronic Components and Technology Conference |pages=139-144 |via=IEEE}}</ref> Therefore, because Nvidia and ATI chose to use high-lead solder bumps, they needed a high T<sub>g</sub> underfill. However, this study was not out at the time that GPUs from 2005 were made, and the companies ended up using low T<sub>g</sub> underfill in these processors. | In 2006, IBM and Amkor published a study that explained that use of a low T<sub>g</sub> underfill was not acceptable with high-lead solder bumps, and high T<sub>g</sub> would be necessary to avoid defects.<ref>{{Cite journal |last=Ray |first=S. |last2=Kiyono |first2=S. |last3=Waite |first3=K. |last4=Nicholls |first4=L. |date=2006 |title=Qualification of low-K 90nm Technology Die with Pb-free Bumps on a Build-up Laminate Package (PBGA) with Pb-free Assembly Processes |journal=56th Electronic Components and Technology Conference |pages=139-144 |via=IEEE}}</ref> Therefore, because Nvidia and ATI chose to use high-lead solder bumps, they needed a high T<sub>g</sub> underfill. However, this study was not out at the time that GPUs from 2005 were made, and the companies ended up using low T<sub>g</sub> underfill in these processors. | ||
This low T<sub>g</sub> underfill would become too soft at high, but normal operating temperatures for these GPUs. When the processor went through normal thermal changes, the solder bumps would soften under heat and harden as they cooled.<ref name=":0">{{Cite web |last=Demerjian |first=Charlie |date=1 Sep 2008 |title=Why Nvidia's chips are defective |url= | This low T<sub>g</sub> underfill would become too soft at high, but normal operating temperatures for these GPUs. When the processor went through normal thermal changes, the solder bumps would soften under heat and harden as they cooled.<ref name=":0">{{Cite web |last=Demerjian |first=Charlie |date=1 Sep 2008 |title=Why Nvidia's chips are defective |url=https://www.theinquirer.net/inquirer/news/1004378/why-nvidia-chips-defective |url-status=dead |archive-url=https://web.archive.org/web/20090520152257/http://www.theinquirer.net/inquirer/news/1004378/why-nvidia-chips-defective |archive-date=20 May 2009 |access-date=1 Jun 2025 |website=The Inquirer}}</ref> This would happen over and over again until they cracked under the thermal stress. When enough solder bumps cracked, it would cause a failure in the unit, hence the term "Bumpgate". | ||
==Companies involved and responses== | ==Companies involved and responses== | ||
| Line 25: | Line 36: | ||
"Three flashing red rings" on the Xbox 360 simply means "core digital error"- or, a general hardware failure. This error can be triggered by a variety of issues, including GPU failure. A Bumpgate-related malfunction would require multiple power cycles to cause a failure in the solder bumps, but it is not uncommon for consoles to see that during testing before being shipped. It is possible that Bumpgate-affected Xbox 360s with GPU failures were part or even all 43% of those defective consoles, but there is no way to be certain. At initial launch, 200,000 non-functional Xbox 360s were in what Microsoft termed "the bonepile"- systems that they planned to repair and ship out later. Approximately 56% of systems worked on the first attempt after manufacture, and that figure improved to 71% after component reworks. At the peak of the "Red Ring of Death" crisis, some consumers discovered that reflowing the motherboard- a process similar to what Microsoft may have done to improve their yield of working Xbox 360s- they could restore functionality to their console, but this was typically for a limited period that was inconsistent between systems.<ref name=":2" /> | "Three flashing red rings" on the Xbox 360 simply means "core digital error"- or, a general hardware failure. This error can be triggered by a variety of issues, including GPU failure. A Bumpgate-related malfunction would require multiple power cycles to cause a failure in the solder bumps, but it is not uncommon for consoles to see that during testing before being shipped. It is possible that Bumpgate-affected Xbox 360s with GPU failures were part or even all 43% of those defective consoles, but there is no way to be certain. At initial launch, 200,000 non-functional Xbox 360s were in what Microsoft termed "the bonepile"- systems that they planned to repair and ship out later. Approximately 56% of systems worked on the first attempt after manufacture, and that figure improved to 71% after component reworks. At the peak of the "Red Ring of Death" crisis, some consumers discovered that reflowing the motherboard- a process similar to what Microsoft may have done to improve their yield of working Xbox 360s- they could restore functionality to their console, but this was typically for a limited period that was inconsistent between systems.<ref name=":2" /> | ||
[[File:GPU late.jpg|alt=An image of a 65 nanometer graphics processing unit from a Microsoft Xbox 360. The die is in the center of the green square, and the eDRAM is located to the left of it. The die has a light, whitish underfill- indicating that it is not one of the defective units.|thumb|204x204px|A ''non-defective'' 65nm "Xenos" GPU from a later revision of Xbox 360 (Jasper V2 on "Kronos 1" package). Note the smaller die and the light, whitish underfill.]] | [[File:GPU late.jpg|alt=An image of a 65 nanometer graphics processing unit from a Microsoft Xbox 360. The die is in the center of the green square, and the eDRAM is located to the left of it. The die has a light, whitish underfill- indicating that it is not one of the defective units.|thumb|204x204px|A ''non-defective'' 65nm "Xenos" GPU from a later revision of Xbox 360 (Jasper V2 on "Kronos 1" package). Note the smaller die and the light, whitish underfill.]] | ||
At the height of the Bumpgate-related defect, approximately 600,000 to 1,000,000 Xbox 360s had the "Red Ring of Death". For several months, Microsoft charged consumers to repair affected consoles. However, in 2007, Microsoft extended the warranty for Xbox 360 consoles displaying an E74 error (an on-screen error also associated with "Red Ring of Death") from one year to three years from the date of purchase. This extension allowed most impacted consumers to get their consoles refurbished at no additional cost. In addition, Microsoft refunded any consumers who had previously paid for repairs related to this issue.<ref>{{Cite web |last=Moore |first=Peter |date=2007 |title=Open Letter From Peter Moore |url= | At the height of the Bumpgate-related defect, approximately 600,000 to 1,000,000 Xbox 360s had the "Red Ring of Death". For several months, Microsoft charged consumers to repair affected consoles. However, in 2007, Microsoft extended the warranty for Xbox 360 consoles displaying an E74 error (an on-screen error also associated with "Red Ring of Death") from one year to three years from the date of purchase. This extension allowed most impacted consumers to get their consoles refurbished at no additional cost. In addition, Microsoft refunded any consumers who had previously paid for repairs related to this issue.<ref>{{Cite web |last=Moore |first=Peter |date=2007 |title=Open Letter From Peter Moore |url=https://xbox.com/en-ca/support/petermooreletter.htm |url-status=dead |archive-url=https://web.archive.org/web/20071023004948/http://xbox.com/en-ca/support/petermooreletter.htm |archive-date=23 Oct 2007 |access-date=4 Jun 2025 |website=Xbox}}</ref> By mid-2008, the cause was confirmed to be an issue "within the components"- the low T<sub>g</sub> underfill. After this point, ATI and Microsoft completely fixed this issue in Xbox 360s. | ||
Microsoft appears to have taken this approach to protect the Xbox brand. The Xbox 360 was only the company's second home console, and the original Xbox hadn't performed as well as anticipated. Microsoft confirmed this in Chapter 5 of their 2021 documentary, ''Power On: The Story of Xbox''.<ref name=":3" /> | Microsoft appears to have taken this approach to protect the Xbox brand. The Xbox 360 was only the company's second home console, and the original Xbox hadn't performed as well as anticipated. Microsoft confirmed this in Chapter 5 of their 2021 documentary, ''Power On: The Story of Xbox''.<ref name=":3" /> | ||
| Line 32: | Line 43: | ||
<blockquote>''"We entirely refute the suggestion that PS3 consoles have an inherent defect or other design issue which is akin to any warranty issue experienced by another console manufacturer. [...] We think it is highly unfair to suggest that from an installed base of 2.5 million that the numbers you mention somehow are evidence of a 'manufacturing defect'..."'' | <blockquote>''"We entirely refute the suggestion that PS3 consoles have an inherent defect or other design issue which is akin to any warranty issue experienced by another console manufacturer. [...] We think it is highly unfair to suggest that from an installed base of 2.5 million that the numbers you mention somehow are evidence of a 'manufacturing defect'..."'' | ||
- Ray Maguire, managing director and senior vice president of Sony Computer Entertainment Europe in 2009; in a letter responding to a BBC Watchdog segment covering the "Yellow Light of Death".<ref name=":12">{{Cite web |last=BBC |date=18 Sep 2009 |title=Sony rebuts BBC PlayStation claim |url= | - Ray Maguire, managing director and senior vice president of Sony Computer Entertainment Europe in 2009; in a letter responding to a BBC Watchdog segment covering the "Yellow Light of Death".<ref name=":12">{{Cite web |last=BBC |date=18 Sep 2009 |title=Sony rebuts BBC PlayStation claim |url=https://news.bbc.co.uk/2/hi/technology/8263063.stm |url-status=live |archive-url=https://web.archive.org/web/20250219154020/http://news.bbc.co.uk/2/hi/technology/8263063.stm |archive-date=19 Feb 2025 |access-date=4 Jun 2025 |website=BBC NEWS}}</ref><ref name=":13">{{Cite web |last=Martin |first=Matt |date=17 Sep 2009 |title=Sony tackles BBC over 'PS3 failure' report |url=https://www.gamesindustry.biz/sony-tackles-bbc-over-ps3-failure-report |url-status=live |access-date=10 Jun 2025 |website=Gamesindustry.biz}}</ref></blockquote>In November 2006, Sony released the PlayStation 3. Similar to the Xbox 360, the early models of PlayStation 3 had what consumers believe to be a Bumpgate-related defect with its Nvidia-based Reality Synthesizer (RSX) GPU. Many consumers who had purchased early "phat" models with a 90nm GPU experienced critical system failures. PlayStation 3 systems that were exhibiting the issue may freeze suddenly during gameplay and shut off. However, all users affected would find that when attempting to turn their PlayStation 3 back on, the LED power indicator would momentarily turn green, then the system would beep three times while very briefly flashing to a yellow light, then it would continuously blink red without booting the system. This failure was coined by consumers as the "Yellow Light of Death". | ||
A 2009 segment by BBC Watchdog covered the "Yellow Light of Death".<ref name=":1">{{Cite web |last=HelpForPS3 (Reuploader) |last2=BBC |date=17 Dec 2009 |title=Sony PS3 Yellow Light of Death - BBC |url=https://www.youtube.com/watch?v=E_ef8bDQktI |url-status=live |access-date=3 Jun 2025 |website=YouTube}}</ref> The presenters stated that when viewers contacted Sony about their console malfunctioning, the company simply stated that the issue could have resulted from a variety of possible failures, and pinpointing the cause would require disassembly of the affected system to analyze error codes. This is true; the LED indicators only indicate a general hardware failure, and a proper diagnosis can only be made after checking the PS3's System Controller (Syscon) and checking for error codes. | A 2009 segment by BBC Watchdog covered the "Yellow Light of Death".<ref name=":1">{{Cite web |last=HelpForPS3 (Reuploader) |last2=BBC |date=17 Dec 2009 |title=Sony PS3 Yellow Light of Death - BBC |url=https://www.youtube.com/watch?v=E_ef8bDQktI |url-status=live |access-date=3 Jun 2025 |website=YouTube}}</ref> The presenters stated that when viewers contacted Sony about their console malfunctioning, the company simply stated that the issue could have resulted from a variety of possible failures, and pinpointing the cause would require disassembly of the affected system to analyze error codes. This is true; the LED indicators only indicate a general hardware failure, and a proper diagnosis can only be made after checking the PS3's System Controller (Syscon) and checking for error codes. | ||
| Line 42: | Line 53: | ||
When the 65nm RSX was released in Fall 2008,<ref name=":6" /> the defect that seems to have caused the "Yellow Light of Death" was resolved for the late "phat" PS3 models. PS3 "slim" models and newer revisions were completely unaffected. The timing of this led some consumers<ref name=":4" /> to suspect that Sony had known about the defect and quietly fixed it. This was at the same time that Microsoft was addressing the problems with the Xbox 360, as well as while Nvidia was being confronted for their Bumpgate defect in some of their GPUs (particularly, the G84 and G86). | When the 65nm RSX was released in Fall 2008,<ref name=":6" /> the defect that seems to have caused the "Yellow Light of Death" was resolved for the late "phat" PS3 models. PS3 "slim" models and newer revisions were completely unaffected. The timing of this led some consumers<ref name=":4" /> to suspect that Sony had known about the defect and quietly fixed it. This was at the same time that Microsoft was addressing the problems with the Xbox 360, as well as while Nvidia was being confronted for their Bumpgate defect in some of their GPUs (particularly, the G84 and G86). | ||
Notably, the RSX is a customized version of Nvidia's 256MB GeForce 7800 GTX.<ref>{{Cite web |last=Shimpi |first=Anand |last2=Wilson |first2=Derek |date=24 Jun 2005 |title=Microsoft's Xbox 360, Sony's PS3 - A Hardware Discussion |url=https://www.anandtech.com/show/1719/9 |url-status=live |access-date=16 Jun 2025 |website=AnandTech}}</ref> The GeForce 7 series does have some defective units with low-T<sub>g</sub> underfill, but it is unclear if the 7800 GTX is among them. There are no known sources of consumer complaints about the 7800 GTX, and none of the notebook laptops covered by the Nvidia class action lawsuit settlement<ref name=":16" /> seem to have that GPU or a mobile version of it. However, this does not necessarily indicate that there was no defect with the 90nm RSX, that Sony wasn't aware that it was defective, or that the 90nm RSX wasn't at all affected by Bumpgate. Console repair and modding enthusiasts have performed extensive research<ref name=":4" /><ref name=":18">{{Cite web |last=Derf |last2=Nadaman |last3= | Notably, the RSX is a customized version of Nvidia's 256MB GeForce 7800 GTX.<ref>{{Cite web |last=Shimpi |first=Anand |last2=Wilson |first2=Derek |date=24 Jun 2005 |title=Microsoft's Xbox 360, Sony's PS3 - A Hardware Discussion |url=https://www.anandtech.com/show/1719/9 |url-status=live |access-date=16 Jun 2025 |website=AnandTech}}</ref> The GeForce 7 series does have some defective units with low-T<sub>g</sub> underfill, but it is unclear if the 7800 GTX is among them. There are no known sources of consumer complaints about the 7800 GTX, and none of the notebook laptops covered by the Nvidia class action lawsuit settlement<ref name=":16" /> seem to have that GPU or a mobile version of it. However, this does not necessarily indicate that there was no defect with the 90nm RSX, that Sony wasn't aware that it was defective, or that the 90nm RSX wasn't at all affected by Bumpgate. Console repair and modding enthusiasts have performed extensive research<ref name=":4" /><ref name=":18">{{Cite web |last=Derf |last2=Nadaman |last3=Alex3107 |display-authors=2 |date=14 Jun 2025 |title=PlayStation 3 - Buying Guide |url=https://consolemods.org/wiki/PS3:Buying_Guide#PlayStation_3_%22Fat/Phat%22_(2006-2009) |url-status=live |access-date=14 Jun 2025 |website=ConsoleMods Wiki}}</ref> and testing<ref name=":14" /> strongly supports the hypothesis that the "Yellow Light of Death" was caused by Bumpgate, and the only way to permanently repair an affected system is to replace the 90nm RSX with a working non-defective unit. | ||
As of 2025, Sony has never made an official statement confirming that the "Yellow Light of Death" was a widespread issue in early PlayStation 3 consoles, nor have they definitively explained that it was linked to Bumpgate. | As of 2025, Sony has never made an official statement confirming that the "Yellow Light of Death" was a widespread issue in early PlayStation 3 consoles, nor have they definitively explained that it was linked to Bumpgate. | ||
| Line 48: | Line 59: | ||
There were a variety of Dell and HP-Compaq notebook laptops that were affected by the Bumpgate defect, as evidenced by the Nvidia class action lawsuit.<ref name=":16" /> Upon being informed of the defect by Nvidia in 2008, both companies distributed BIOS updates for affected systems with Nvidia GPUs that according to The Inquirer, "[ran] the fan all the time".<ref name=":5" /> The purpose of this was to attempt to prevent the problem from occurring so that consumers wouldn't have to get their systems repaired. However, both companies also provided free repairs for systems already exhibiting symptoms of a failing GPU, such as no video output to the monitor or the computer failing to boot. It's implied in an SEC report that Nvidia filed in 2008 that the companies were compensated for providing this service.<ref name=":10" /> | There were a variety of Dell and HP-Compaq notebook laptops that were affected by the Bumpgate defect, as evidenced by the Nvidia class action lawsuit.<ref name=":16" /> Upon being informed of the defect by Nvidia in 2008, both companies distributed BIOS updates for affected systems with Nvidia GPUs that according to The Inquirer, "[ran] the fan all the time".<ref name=":5" /> The purpose of this was to attempt to prevent the problem from occurring so that consumers wouldn't have to get their systems repaired. However, both companies also provided free repairs for systems already exhibiting symptoms of a failing GPU, such as no video output to the monitor or the computer failing to boot. It's implied in an SEC report that Nvidia filed in 2008 that the companies were compensated for providing this service.<ref name=":10" /> | ||
Both Dell and HP-Compaq also offered varying limited warranties after impacted devices were repaired. HP-Compaq offered a limited warranty for 24 months (two years) after the start of customers' original limited warranty or 90 days (approximately three months) after the affected notebook was repaired- whichever was later.<ref name=":7">{{Cite web |last=HP |date=2008 |title=HP Pavilion dv2000/dv6000/dv9000 and Compaq Presario v3000/v6000 Series Notebook PCs - HP Limited Warranty Service Enhancement |url= | Both Dell and HP-Compaq also offered varying limited warranties after impacted devices were repaired. HP-Compaq offered a limited warranty for 24 months (two years) after the start of customers' original limited warranty or 90 days (approximately three months) after the affected notebook was repaired- whichever was later.<ref name=":7">{{Cite web |last=HP |date=2008 |title=HP Pavilion dv2000/dv6000/dv9000 and Compaq Presario v3000/v6000 Series Notebook PCs - HP Limited Warranty Service Enhancement |url=https://h10025.www1.hp.com/ewfrf/wc/document?lc=en&cc=us&dlc=en&docname=c01087277 |url-status=dead |archive-url=https://web.archive.org/web/20080710172852/http://h10025.www1.hp.com/ewfrf/wc/document?lc=en&cc=us&dlc=en&docname=c01087277 |archive-date=10 Jul 2008 |access-date=8 Jun 2025 |website=HP Customer Care}}</ref> Dell extended limited warranties for systems with these issues for 12 months (one year) from the original purchase date, with a maximum of up to 60 months (five years). In addition, they even offered this to customers whose original warranties already expired- making the new warranty valid from the date the original warranty expired.<ref>{{Cite web |last=Menchaca |first=Lionel |date=18 Aug 2008 |title=NVIDIA GPU Update: Dell to Offer Limited Warranty Enhancement to All Affected Customers Worldwide |url=https://en.community.dell.com/blogs/direct2dell/archive/2008/08/18/nvidia-gpu-update-dell-to-offer-warranty-enhancement-to-all-affected-customers-worldwide.aspx |url-status=dead |archive-url=https://web.archive.org/web/20081219131311/http://en.community.dell.com/blogs/direct2dell/archive/2008/08/18/nvidia-gpu-update-dell-to-offer-warranty-enhancement-to-all-affected-customers-worldwide.aspx |archive-date=19 Dec 2008 |access-date=23 Jun 2025 |website=Direct2Dell}}</ref><ref name=":8">{{Cite web |last=Meyer |first=David |date=19 Aug 2008 |title=Dell extends warranties after GPU fault |url=https://www.zdnet.com/article/dell-extends-warranties-after-gpu-fault/ |url-status=live |access-date=8 Jun 2025 |website=ZDNet}}</ref> | ||
====Apple's Response - Macbook Pro, May 2007 - September 2008==== | ====Apple's Response - Macbook Pro, May 2007 - September 2008==== | ||
In May 2007, Apple released a version of the aluminum Macbook Pro that used the Nvidia GeForce 8600M GT GPU, and manufactured the computers with this GPU until September 2008. They stopped manufacturing them with this GPU because they discovered it was one of the models affected by Bumpgate. Unlike the issues with the PlayStation 3 and Xbox 360 where the system wouldn't even boot, consumers ran into distorted video or no video output on their devices. Nvidia had assured Apple that the graphics processors were not defective, so Apple initially ignored reports expressing that possibility.<ref name=":9">{{Cite web |last=Foresman |first=Chris |date=10 Oct 2008 |title=Apple: NVIDIA chips to blame for MacBook Pro video problems |url=https://arstechnica.com/gadgets/2008/10/apple-nvidia-chips-to-blame-for-macbook-pro-video-problems/ |url-status=live |access-date=8 Jun 2025 |website=Ars Technica}}</ref><ref>{{Cite web |last=Buchanan |first=Matt |date=10 Oct 2008 |title=Apple Confirms Failing Nvidia Graphics Cards in MacBook Pros, Offers Free Repairs and Refunds |url=https://gizmodo.com/apple-confirms-failing-nvidia-graphics-cards-in-macbook-5061605 |url-status=live |access-date=7 Jun 2025 |website=Gizmodo}}</ref> However, after doing their own investigation, Apple had found that the processors actually were defective. Because of this, Apple offered extended repair coverage adding up to four years from the date of original purchase, and refunded customers who already paid to repair systems affected by this defect.<ref>{{Cite web |last=Apple Support |date=18 Nov 2014 |title=MacBook Pro: Distorted video or no video issues |url= | In May 2007, Apple released a version of the aluminum Macbook Pro that used the Nvidia GeForce 8600M GT GPU, and manufactured the computers with this GPU until September 2008. They stopped manufacturing them with this GPU because they discovered it was one of the models affected by Bumpgate. Unlike the issues with the PlayStation 3 and Xbox 360 where the system wouldn't even boot, consumers ran into distorted video or no video output on their devices. Nvidia had assured Apple that the graphics processors were not defective, so Apple initially ignored reports expressing that possibility.<ref name=":9">{{Cite web |last=Foresman |first=Chris |date=10 Oct 2008 |title=Apple: NVIDIA chips to blame for MacBook Pro video problems |url=https://arstechnica.com/gadgets/2008/10/apple-nvidia-chips-to-blame-for-macbook-pro-video-problems/ |url-status=live |access-date=8 Jun 2025 |website=Ars Technica}}</ref><ref>{{Cite web |last=Buchanan |first=Matt |date=10 Oct 2008 |title=Apple Confirms Failing Nvidia Graphics Cards in MacBook Pros, Offers Free Repairs and Refunds |url=https://gizmodo.com/apple-confirms-failing-nvidia-graphics-cards-in-macbook-5061605 |url-status=live |access-date=7 Jun 2025 |website=Gizmodo}}</ref> However, after doing their own investigation, Apple had found that the processors actually were defective. Because of this, Apple offered extended repair coverage adding up to four years from the date of original purchase, and refunded customers who already paid to repair systems affected by this defect.<ref>{{Cite web |last=Apple Support |date=18 Nov 2014 |title=MacBook Pro: Distorted video or no video issues |url=https://support.apple.com/en-us/HT203254 |url-status=dead |archive-url=https://web.archive.org/web/20141202230527/http://support.apple.com/en-us/HT203254 |archive-date=2 Dec 2014 |access-date=7 Jun 2025 |website=Apple}}</ref> | ||
====Nvidia's Response - Inquirer Accusations and SEC Report==== | ====Nvidia's Response - Inquirer Accusations and SEC Report==== | ||
[[File:Nvidia G80, G84 and G86 size comparison.jpg|alt=Three GPU dies are lined up in a row, from largest to smallest, left to right: the Nvidia G80, then the G84, then the G86. The internals of the dies are visible.|thumb|Image of the Nvidia G80, G84, and G86's dies (left-to-right). The G84 and the G86 GPUs are known to have been impacted by the Bumpgate defect.<ref name=":5">{{Cite web |last=Demerjian |first=Charlie |date=9 Jul 2008 |title=All Nvidia G84 and G86s are bad |url= | [[File:Nvidia G80, G84 and G86 size comparison.jpg|alt=Three GPU dies are lined up in a row, from largest to smallest, left to right: the Nvidia G80, then the G84, then the G86. The internals of the dies are visible.|thumb|Image of the Nvidia G80, G84, and G86's dies (left-to-right). The G84 and the G86 GPUs are known to have been impacted by the Bumpgate defect.<ref name=":5">{{Cite web |last=Demerjian |first=Charlie |date=9 Jul 2008 |title=All Nvidia G84 and G86s are bad |url=https://www.theinquirer.net/gb/inquirer/news/2008/07/09/nvidia-g84-g86-bad |url-status=dead |archive-url=https://web.archive.org/web/20080710121746/http://www.theinquirer.net/gb/inquirer/news/2008/07/09/nvidia-g84-g86-bad |archive-date=10 Jul 2008 |access-date=7 Jun 2025 |website=The Inquirer}}</ref>]] | ||
''See also: Lawsuit(s)''<blockquote>''[As of July 2, 2008, all] newly manufactured products and all products currently shipping in volume have a different and more robust material set. [...] We intend to fully support our customers in their repair and replacement of these impacted MCP and GPU products that fail.'' | ''See also: Lawsuit(s)''<blockquote>''[As of July 2, 2008, all] newly manufactured products and all products currently shipping in volume have a different and more robust material set. [...] We intend to fully support our customers in their repair and replacement of these impacted MCP and GPU products that fail.'' | ||
| Line 62: | Line 73: | ||
On July 2nd, 2008- a few days before Demerjian's article was published- Nvidia filed a report with the Securities and Exchange Commission (SEC).<ref name=":10" /> The report explained that the corporation would pay a $150-200 million one-time charge to cover customer warranties, repairs, returns, replacements, and other notable expenses caused by poor packaging material in some of their media and communications processors (MCPs) and GPUs exclusively used in laptops. This report also stated that all of their newly manufactured products from that point forward would have a more suitable material set. | On July 2nd, 2008- a few days before Demerjian's article was published- Nvidia filed a report with the Securities and Exchange Commission (SEC).<ref name=":10" /> The report explained that the corporation would pay a $150-200 million one-time charge to cover customer warranties, repairs, returns, replacements, and other notable expenses caused by poor packaging material in some of their media and communications processors (MCPs) and GPUs exclusively used in laptops. This report also stated that all of their newly manufactured products from that point forward would have a more suitable material set. | ||
On the same day, EE Times published an article where Nvidia explained more about what the cause of the problem was.<ref>{{Cite web |last=LaPedus |first=Mark |date=2 Jul 2008 |title=Nvidia takes charge for bad chips, but who is to blame? |url= | On the same day, EE Times published an article where Nvidia explained more about what the cause of the problem was.<ref>{{Cite web |last=LaPedus |first=Mark |date=2 Jul 2008 |title=Nvidia takes charge for bad chips, but who is to blame? |url=https://eetimes.com/electronics-products/processors/4105543/Nvidia-takes-charge-for-bad-chips-but-who-is-to-blame- |url-status=dead |archive-url=https://web.archive.org/web/20121117035658/http://eetimes.com/electronics-products/processors/4105543/Nvidia-takes-charge-for-bad-chips-but-who-is-to-blame- |archive-date=17 Nov 2012 |access-date=23 Jun 2025 |website=EE Times}}</ref> Nvidia had stated in an email that one of their many packaging partners, Taiwan Semiconductor Manufacturing Co. Ltd. (TSMC), was responsible. However, a subsequent email backpedaled on this response. In the next email, Nvidia stated that they "worked closely" with TSMC on the packaging and material, and hence took full responsibility. DigiTimes attempted to ask TSMC and some of Nvidia's other packaging partners about the defective chips, but TSMC declined a response, citing "client confidentiality", and the other partners did not know anything about the issue because the chips were from an older generation by that point.<ref>{{Cite web |last=Lee |first=Ingrid |last2=Shen |first2=Steve |date=4 Jul 2008 |title=Nvidia contract makers in Taiwan low-key over defective chip reports |url=https://www.digitimes.com/bits_chips/a20080704PD210.html |url-status=dead |archive-url=https://web.archive.org/web/20080704211726/http://www.digitimes.com/bits_chips/a20080704PD210.html |archive-date=4 Jul 2008 |access-date=23 Jun 2025 |website=DIGITIMES}}</ref> In August 2008, Nvidia's CEO Jen-Hsun Huang explained how the company would handle the defect in an interview during Nvision 2008, claiming that Nvidia would take some responsibility for the defect and pay manufacturers to help consumers fix their devices.<ref>{{Cite web |last=Scath (Reuploader) |date=25 Aug 2008 |title=Nvision: Nvidia CEO Talks About Chip Failure |url=https://www.youtube.com/watch?v=QZB6kxxgnOQ&t=275 |url-status=live |access-date=27 Jun 2025 |website=YouTube}}</ref> | ||
Joel Hruska from Ars Technica explained that if Nvidia ''was'' trying to cover up the defect- as Demerjian claimed- with this report to the SEC, they not only attempted to avoid responsibility and accused their suppliers of causing the problem, they also committed financial fraud by intentionally lowballing their expected financial losses. This is a major accusation that could have had severe consequences for Nvidia, and could have been dangerous for the company.<ref name=":11">{{Cite web |last=Hruska |first=Joel |date=16 Jul 2008 |title=NVIDIA denies rumors of faulty chips, mass GPU failures |url=https://arstechnica.com/gadgets/2008/07/nvidia-denies-rumors-of-mass-gpu-failures/ |url-status=live |access-date=8 Jun 2025 |website=Ars Technica}}</ref> However, it is difficult to verify if Nvidia was lying or simply not fully aware of the scale of Bumpgate. Nvidia's public acknowledgement of the defect in the SEC report is consistent with when Dell<ref name=":8" /> and HP<ref name=":7" /> discovered the problem. Although Sony never made a statement on the "Yellow Light of Death", this is even consistent with when they switched the PlayStation 3 to the non-defective 65nm RSX.<ref name=":6" /> The only company it does not appear consistent with is Apple, who discovered the defect in their Macbook Pro systems after their own investigation in September 2008.<ref name=":9" /> Even so, it is possible that Nvidia did not know that the GeForce 8600M GT GPUs in the 2007-2008 Macbook Pro were impacted yet when Apple asked them about it, because the situation was still developing. | Joel Hruska from Ars Technica explained that if Nvidia ''was'' trying to cover up the defect- as Demerjian claimed- with this report to the SEC, they not only attempted to avoid responsibility and accused their suppliers of causing the problem, they also committed financial fraud by intentionally lowballing their expected financial losses. This is a major accusation that could have had severe consequences for Nvidia, and could have been dangerous for the company.<ref name=":11">{{Cite web |last=Hruska |first=Joel |date=16 Jul 2008 |title=NVIDIA denies rumors of faulty chips, mass GPU failures |url=https://arstechnica.com/gadgets/2008/07/nvidia-denies-rumors-of-mass-gpu-failures/ |url-status=live |access-date=8 Jun 2025 |website=Ars Technica}}</ref> However, it is difficult to verify if Nvidia was lying or simply not fully aware of the scale of Bumpgate. Nvidia's public acknowledgement of the defect in the SEC report is consistent with when Dell<ref name=":8" /> and HP<ref name=":7" /> discovered the problem. Although Sony never made a statement on the "Yellow Light of Death", this is even consistent with when they switched the PlayStation 3 to the non-defective 65nm RSX.<ref name=":6" /> The only company it does not appear consistent with is Apple, who discovered the defect in their Macbook Pro systems after their own investigation in September 2008.<ref name=":9" /> Even so, it is possible that Nvidia did not know that the GeForce 8600M GT GPUs in the 2007-2008 Macbook Pro were impacted yet when Apple asked them about it, because the situation was still developing. | ||
| Line 77: | Line 88: | ||
Throughout the lawsuit, Nvidia continually denied allegations of intentional wrongdoing. <!-- Expand with the response of Nvidia or counterclaims. -V --> | Throughout the lawsuit, Nvidia continually denied allegations of intentional wrongdoing. <!-- Expand with the response of Nvidia or counterclaims. -V --> | ||
===Outcome - Settlement and Class Member Appeal=== | ===Outcome - Settlement and Class Member Appeal=== | ||
Nvidia opted for a settlement<ref>{{Cite web |date=2010 |title=Frequently Asked Questions - What can I get from the settlement? |url= | Nvidia opted for a settlement<ref>{{Cite web |date=2010 |title=Frequently Asked Questions - What can I get from the settlement? |url=https://www.nvidiasettlement.com/faq.html |url-status=dead |archive-url=https://web.archive.org/web/20101001080625/http://www.nvidiasettlement.com/faq.html |archive-date=1 Oct 2010 |access-date=13 Jun 2025 |website=The NVIDIA GPU Litigation}}</ref>- though, still asserted that the settlement was not an admission of wrongdoing. Consumers who participated as settlement class members were presented with three options to be compensated, with all benefits paid for by Nvidia: | ||
#A replacement GPU inside their affected notebook; | #A replacement GPU inside their affected notebook; | ||
| Line 91: | Line 102: | ||
In addition, the release of the information in ''Power On'' helped enthusiasts learn more about how to fix the issue in early Xbox 360s themselves, if necessary. Many consumers who had purchased an Xbox 360 from this era<ref>{{Cite web |last=Enever |first=Liam |date=1 Oct 2017 |title=Why has my got the red ring of death (sic) |url=https://www.ifixit.com/Answers/View/430460/Why+has+my+got+the+red+ring+of+death |url-status=live |access-date=4 Jun 2025 |website=iFixIt Answers Forum}}</ref> and even some independent repair technicians<ref>{{Cite web |date=8 Apr 2014 |title=Xbox 360 Red Ring of Death: Why Lead-Free Solder or Solder Failure Are the Problem |url=https://electronicfix.com.au/console-repairs/what-does-the-rrod-mean/ |url-status=live |access-date=4 Jun 2025 |website=The Electronic Fix}}</ref> had initially assumed that the issue was caused by Microsoft using lead-free solder balls- which are used to connect the Xenos GPU to the motherboard- in order to meet new-at-the-time RoHS standards in the European Union. These individuals alleged that the brittler nature of non-leaded solder made the connections weaker compared to traditional leaded solder, causing the defect. Other consumers had a similar assumption, and thought it was related to the solder balls' melting point, and that the console was getting too hot and "desoldering" the GPU from the motherboard as a result. These misconceptions largely faded into obscurity when Microsoft released the non-defective revisions of the Xbox 360 and announced their extended warranty on their defective consoles, but they were dispelled entirely once Chapter 5 of ''Power On'' was released. | In addition, the release of the information in ''Power On'' helped enthusiasts learn more about how to fix the issue in early Xbox 360s themselves, if necessary. Many consumers who had purchased an Xbox 360 from this era<ref>{{Cite web |last=Enever |first=Liam |date=1 Oct 2017 |title=Why has my got the red ring of death (sic) |url=https://www.ifixit.com/Answers/View/430460/Why+has+my+got+the+red+ring+of+death |url-status=live |access-date=4 Jun 2025 |website=iFixIt Answers Forum}}</ref> and even some independent repair technicians<ref>{{Cite web |date=8 Apr 2014 |title=Xbox 360 Red Ring of Death: Why Lead-Free Solder or Solder Failure Are the Problem |url=https://electronicfix.com.au/console-repairs/what-does-the-rrod-mean/ |url-status=live |access-date=4 Jun 2025 |website=The Electronic Fix}}</ref> had initially assumed that the issue was caused by Microsoft using lead-free solder balls- which are used to connect the Xenos GPU to the motherboard- in order to meet new-at-the-time RoHS standards in the European Union. These individuals alleged that the brittler nature of non-leaded solder made the connections weaker compared to traditional leaded solder, causing the defect. Other consumers had a similar assumption, and thought it was related to the solder balls' melting point, and that the console was getting too hot and "desoldering" the GPU from the motherboard as a result. These misconceptions largely faded into obscurity when Microsoft released the non-defective revisions of the Xbox 360 and announced their extended warranty on their defective consoles, but they were dispelled entirely once Chapter 5 of ''Power On'' was released. | ||
Although the majority of Xbox 360 consoles affected by Bumpgate were repaired by Microsoft as part of their extended warranty program, there were some that managed to slip through the cracks, so consumers today still need to be informed. It is generally recommended by retro console enthusiasts to purchase Xbox 360 consoles manufactured after May 2008 (or marked "Q2 2008") and avoid consoles manufactured before this point, but the Tonasket (AKA "Jasper Kronos" or "Jasper V2") motherboard revision is generally considered to be the most reliable of the original "phat" model Xbox 360 consoles.<ref>{{Cite web |last=Nadaman | | Although the majority of Xbox 360 consoles affected by Bumpgate were repaired by Microsoft as part of their extended warranty program, there were some that managed to slip through the cracks, so consumers today still need to be informed. It is generally recommended by retro console enthusiasts to purchase Xbox 360 consoles manufactured after May 2008 (or marked "Q2 2008") and avoid consoles manufactured before this point, but the Tonasket (AKA "Jasper Kronos" or "Jasper V2") motherboard revision is generally considered to be the most reliable of the original "phat" model Xbox 360 consoles.<ref>{{Cite web |last=Nadaman |display-authors=etal |date=8 Jun 2025 |title=Xbox 360 - Buying Guide |url=https://consolemods.org/wiki/Xbox_360:Buying_Guide |url-status=live |access-date=14 Jun 2025 |website=ConsoleMods Wiki}}</ref> | ||
===Sony (PlayStation 3) Consumer Response=== | ===Sony (PlayStation 3) Consumer Response=== | ||
Consumers who experienced the "Yellow Light of Death" were upset with Sony for their poor response. Some, such as PS3 modding and repair enthusiast "RIP Felix", described Sony's response as "gaslighting"<ref name=":2" />- saying that Sony had manipulated consumers into thinking that there was no widespread defect. The six-page letter from Ray Maguire to the BBC following their Watchdog segment takes a tone that supports Felix's claim<ref name=":12" /><ref name=":13" />; By 2009, Sony had stopped producing PS3s with the defective 90nm RSX- which left many consumers suspicious that Sony was trying to cover up the problem to avoid taking responsibility for it. | Consumers who experienced the "Yellow Light of Death" were upset with Sony for their poor response. Some, such as PS3 modding and repair enthusiast "RIP Felix", described Sony's response as "gaslighting"<ref name=":2" />- saying that Sony had manipulated consumers into thinking that there was no widespread defect. The six-page letter from Ray Maguire to the BBC following their Watchdog segment takes a tone that supports Felix's claim<ref name=":12" /><ref name=":13" />; By 2009, Sony had stopped producing PS3s with the defective 90nm RSX- which left many consumers suspicious that Sony was trying to cover up the problem to avoid taking responsibility for it. | ||
In 2006, when the PlayStation 3 was new, the console costed $599 USD for the 60GB model ($499 USD for the 20GB model).<ref name=":3" /> This was a major price to pay for a game console at the time; the Nintendo Wii launched at $249.99 USD in November 2006,<ref>{{Cite web |last=Sanders |first=Kathleen |last2=Casamassina |first2=Matt |date=14 Sep 2006 |title=US Wii Price, Launch Date Revealed |url=https://www.ign.com/articles/2006/09/14/us-wii-price-launch-date-revealed |url-status=live |access-date=10 Jun 2025 |website=IGN}}</ref> and the Xbox 360 launched in late November 2005 at $399 USD ($299 USD for the Core System version).<ref>{{Cite web |last=Surette |first=Tim |date=17 Aug 2005 |title=Xbox 360 pricing revealed: $299 and $399 models due at launch |url= | In 2006, when the PlayStation 3 was new, the console costed $599 USD for the 60GB model ($499 USD for the 20GB model).<ref name=":3" /> This was a major price to pay for a game console at the time; the Nintendo Wii launched at $249.99 USD in November 2006,<ref>{{Cite web |last=Sanders |first=Kathleen |last2=Casamassina |first2=Matt |date=14 Sep 2006 |title=US Wii Price, Launch Date Revealed |url=https://www.ign.com/articles/2006/09/14/us-wii-price-launch-date-revealed |url-status=live |access-date=10 Jun 2025 |website=IGN}}</ref> and the Xbox 360 launched in late November 2005 at $399 USD ($299 USD for the Core System version).<ref>{{Cite web |last=Surette |first=Tim |date=17 Aug 2005 |title=Xbox 360 pricing revealed: $299 and $399 models due at launch |url=https://www.gamespot.com/news/xbox-360-pricing-revealed-299-and-399-models-due-at-launch-6131245 |url-status=dead |archive-url=https://web.archive.org/web/20130605013951/http://www.gamespot.com/news/xbox-360-pricing-revealed-299-and-399-models-due-at-launch-6131245 |archive-date=5 Jun 2013 |access-date=10 Jun 2025 |website=GameSpot}}</ref> Because of the extremely high price compared to competitors, consumers felt that they were making a major investment in a high quality system that would be well-supported if there was a defect. Unfortunately, Sony failed to ever acknowledge the "Yellow Light of Death", other than denying how widespread it was in their only official rebuttal from Ray Maguire, and made consumers pay for an issue that was never their fault. | ||
Some consumers who dealt with the "Yellow Light of Death" paid Sony to repair their systems, but others decided to consider it a loss. For consumers whose systems failed after the "Slim" models came out, some of them chose to re-purchase the cheaper, allegedly more reliable Slim models. At the time, it was assumed that these models were not prone to "Yellow Light of Death" (they have since been confirmed completely non-defective), but there was still some uncertainty among the community at the time. However, others gave up on Sony entirely, switching to other platforms- particularly, the Xbox 360. | Some consumers who dealt with the "Yellow Light of Death" paid Sony to repair their systems, but others decided to consider it a loss. For consumers whose systems failed after the "Slim" models came out, some of them chose to re-purchase the cheaper, allegedly more reliable Slim models. At the time, it was assumed that these models were not prone to "Yellow Light of Death" (they have since been confirmed completely non-defective), but there was still some uncertainty among the community at the time. However, others gave up on Sony entirely, switching to other platforms- particularly, the Xbox 360. | ||
| Line 130: | Line 141: | ||
==References== | ==References== | ||
{{reflist}} | {{reflist}} | ||
[[Category:Nvidia]] | |||
{{Placeholder box|[[mw:Help:VisualEditor/User_guide#Editing_categories|Add a category]] with the same name as the product, service, website, software, product line or company that this article is about. | {{Placeholder box|[[mw:Help:VisualEditor/User_guide#Editing_categories|Add a category]] with the same name as the product, service, website, software, product line or company that this article is about. | ||
The "Incidents" category is not needed.}} | The "Incidents" category is not needed.}} | ||