Bumpgate: Difference between revisions

Vindicator4021 (talk | contribs)
Added info on Nvidia's response (and a little bit of consumer response for them), Apple's response, a very brief description of Dell's response, some pictures, and some citations.
Vindicator4021 (talk | contribs)
Added Dell's response to the appropriate section, detailed much more of Nvidia's response and situation, renamed the "Lawsuit(s)" section to "Nvidia Lawsuit (2010)", and of course, added relevant citations.
Line 35: Line 35:
- Ray Maguire, managing director and senior vice president of Sony Computer Entertainment Europe in 2009; in a letter responding to a BBC Watchdog segment covering the "Yellow Light of Death".<ref>{{Cite web |last=BBC |date=18 Sep 2009 |title=Sony rebuts BBC PlayStation claim |url=http://news.bbc.co.uk/2/hi/technology/8263063.stm |url-status=live |archive-url=https://web.archive.org/web/20250219154020/http://news.bbc.co.uk/2/hi/technology/8263063.stm |archive-date=19 Feb 2025 |access-date=4 Jun 2025 |website=BBC NEWS}}</ref></blockquote>In November 2006, Sony released the PlayStation 3. Like the Xbox 360, the early models of PlayStation 3 had what consumers believe to be Bumpgate-related issues with their Nvidia RSX GPU. Many consumers who had bought launch models- what are now commonly known as "PlayStation 2 backwards compatible" models- found that their systems were malfunctioning. PlayStation 3 systems that were exhibiting the issue may freeze suddenly during gameplay and shut off, but all users who dealt with the defect had a common experience: when turning their PlayStation 3 on, the LED power indicator would turn green for a moment, then the system would beep three times, very briefly flash to a yellow light, then it would blink red. This was coined by consumers as the "Yellow Light of Death".
- Ray Maguire, managing director and senior vice president of Sony Computer Entertainment Europe in 2009; in a letter responding to a BBC Watchdog segment covering the "Yellow Light of Death".<ref>{{Cite web |last=BBC |date=18 Sep 2009 |title=Sony rebuts BBC PlayStation claim |url=http://news.bbc.co.uk/2/hi/technology/8263063.stm |url-status=live |archive-url=https://web.archive.org/web/20250219154020/http://news.bbc.co.uk/2/hi/technology/8263063.stm |archive-date=19 Feb 2025 |access-date=4 Jun 2025 |website=BBC NEWS}}</ref></blockquote>In November 2006, Sony released the PlayStation 3. Like the Xbox 360, the early models of PlayStation 3 had what consumers believe to be Bumpgate-related issues with their Nvidia RSX GPU. Many consumers who had bought launch models- what are now commonly known as "PlayStation 2 backwards compatible" models- found that their systems were malfunctioning. PlayStation 3 systems that were exhibiting the issue may freeze suddenly during gameplay and shut off, but all users who dealt with the defect had a common experience: when turning their PlayStation 3 on, the LED power indicator would turn green for a moment, then the system would beep three times, very briefly flash to a yellow light, then it would blink red. This was coined by consumers as the "Yellow Light of Death".


BBC Watchdog aired a segment in 2009 covering the "Yellow Light of Death".<ref name=":1">{{Cite web |last=HelpForPS3 (Reuploader) |last2=BBC |date=17 Dec 2009 |title=Sony PS3 Yellow Light of Death - BBC |url=https://www.youtube.com/watch?v=E_ef8bDQktI |url-status=live |access-date=3 Jun 2025 |website=YouTube}}</ref> The hosts stated that when viewers called Sony because their console was malfunctioning, Sony told them that it could have been for a variety of reasons, and they couldn't determine exactly what the problem was without disassembling the console having the issue. While all of this is true- the LED indicators do simply indicate a general hardware failure that requires detailed troubleshooting- it is worth noting that by August 2008, Sony was building consoles with the 65nm RSX<ref>{{Cite web |last=Miller |first=R. |date=26 Jun 2008 |title=PS3 graphics chip going 65nm this Fall |url=https://www.engadget.com/2008-06-26-ps3-graphics-chip-going-65nm-this-fall.html |url-status=live |archive-url=https://web.archive.org/web/20210924101930/https://www.engadget.com/2008-06-26-ps3-graphics-chip-going-65nm-this-fall.html |archive-date=24 Sep 2021 |access-date=3 Jun 2025 |website=Engadget}}</ref><ref>{{Cite web |title=NVIDIA Playstation 3 GPU 65nm Specs |url=https://www.techpowerup.com/gpu-specs/playstation-3-gpu-65nm.c1682 |url-status=live |access-date=3 Jun 2025 |website=TechPowerUp GPU Database}}</ref>, which has been confirmed by enthusiasts to not suffer from the likely Bumpgate-related fault.<ref name=":2" />
BBC Watchdog aired a segment in 2009 covering the "Yellow Light of Death".<ref name=":1">{{Cite web |last=HelpForPS3 (Reuploader) |last2=BBC |date=17 Dec 2009 |title=Sony PS3 Yellow Light of Death - BBC |url=https://www.youtube.com/watch?v=E_ef8bDQktI |url-status=live |access-date=3 Jun 2025 |website=YouTube}}</ref> The hosts stated that when viewers called Sony because their console was malfunctioning, Sony told them that it could have been for a variety of reasons, and they couldn't determine exactly what the problem was without disassembling the console having the issue. While all of this is true- the LED indicators do simply indicate a general hardware failure that requires detailed troubleshooting- it is worth noting that by August 2008, Sony was building consoles with the 65nm RSX<ref name=":6">{{Cite web |last=Miller |first=R. |date=26 Jun 2008 |title=PS3 graphics chip going 65nm this Fall |url=https://www.engadget.com/2008-06-26-ps3-graphics-chip-going-65nm-this-fall.html |url-status=live |archive-url=https://web.archive.org/web/20210924101930/https://www.engadget.com/2008-06-26-ps3-graphics-chip-going-65nm-this-fall.html |archive-date=24 Sep 2021 |access-date=3 Jun 2025 |website=Engadget}}</ref><ref>{{Cite web |title=NVIDIA Playstation 3 GPU 65nm Specs |url=https://www.techpowerup.com/gpu-specs/playstation-3-gpu-65nm.c1682 |url-status=live |access-date=3 Jun 2025 |website=TechPowerUp GPU Database}}</ref>, which has been confirmed by enthusiasts to not suffer from the likely Bumpgate-related fault.<ref name=":2" />


Sony never issued a recall, or extended warranty for these systems; consumers had to pay out of their own pocket to get them repaired. According to one of the individuals interviewed in BBC's Watchdog segment, if the system was outside of its one year warranty period, they would have to pay £128 (~$173) to get the system fixed, and they would only provide customers with a three-month post-repair warranty, and if it failed again after that point, they would have to pay out of pocket again.<ref name=":1" /> Keeping in mind that the fault was very likely caused by a GPU defect, this possibility was very likely unless Sony's repair technicians replaced the defective 90nm GPU with a non-defective one, which they were able to do for some consoles.<!-- Felix also mentions this in the video, but there's a primary source for it that I need to find. -->
Sony never issued a recall, or extended warranty for these systems; consumers had to pay out of their own pocket to get them repaired. According to one of the individuals interviewed in BBC's Watchdog segment, if the system was outside of its one year warranty period, they would have to pay £128 (~$173) to get the system fixed, and they would only provide customers with a three-month post-repair warranty, and if it failed again after that point, they would have to pay out of pocket again.<ref name=":1" /> Keeping in mind that the fault was very likely caused by a GPU defect, this possibility was very likely unless Sony's repair technicians replaced the defective 90nm GPU with a non-defective one, which they were able to do for some consoles.<!-- Felix also mentions this in the video, but there's a primary source for it that I need to find. -->
Line 43: Line 43:
As of 2025, Sony has still never made a statement confirming that "Yellow Light of Death", their likely Bumpgate-related issue, was a widespread problem in early PlayStation 3 models. The only reason that consumers know that the RSX was likely afflicted by Bumpgate is thanks to console repair and modding enthusiasts- who had to do extensive research to find out the true cause of the problem.<ref name=":4">{{Cite web |last=RIP Felix |first= |date=23 Dec 2022 |title=A PS3 Story: The Yellow Light of Death |url=https://www.youtube.com/watch?v=Za7WTNwAX0c |url-status=live |access-date=2 Jun 2025 |website=YouTube}}</ref> It was clear from the timeline- what systems have been observed to be reliable or unreliable revisions- as well as tests performed by console repair and modding enthusiasts (such as "poking" the underfill on suspected bad GPUs with a soldering iron<ref name=":2">{{Cite web |last=RIP Felix |date=15 Jan 2024 |title=A 360 Story - The RED Ring of Death & the 7th Generation Console War |url=https://www.youtube.com/watch?v=3qKtS_uxdcU |url-status=live |access-date=2 Jun 2025 |website=YouTube}}</ref>) that it was a Bumpgate-related defect.
As of 2025, Sony has still never made a statement confirming that "Yellow Light of Death", their likely Bumpgate-related issue, was a widespread problem in early PlayStation 3 models. The only reason that consumers know that the RSX was likely afflicted by Bumpgate is thanks to console repair and modding enthusiasts- who had to do extensive research to find out the true cause of the problem.<ref name=":4">{{Cite web |last=RIP Felix |first= |date=23 Dec 2022 |title=A PS3 Story: The Yellow Light of Death |url=https://www.youtube.com/watch?v=Za7WTNwAX0c |url-status=live |access-date=2 Jun 2025 |website=YouTube}}</ref> It was clear from the timeline- what systems have been observed to be reliable or unreliable revisions- as well as tests performed by console repair and modding enthusiasts (such as "poking" the underfill on suspected bad GPUs with a soldering iron<ref name=":2">{{Cite web |last=RIP Felix |date=15 Jan 2024 |title=A 360 Story - The RED Ring of Death & the 7th Generation Console War |url=https://www.youtube.com/watch?v=3qKtS_uxdcU |url-status=live |access-date=2 Jun 2025 |website=YouTube}}</ref>) that it was a Bumpgate-related defect.


==== Dell's Response - Driver Update ====
====Dell's and HP's Responses - BIOS Updates and Free Repairs====
In an attempt to mitigate the problem without having to replace the GPU, Dell sent out a driver update for their laptops with Nvidia GPUs.
HP and Dell both distributed BIOS updates for affected systems with Nvidia GPUs that according to The Inquirer, "[ran] the fan all the time".<ref name=":5" /> The purpose of this was to attempt to prevent the problem from occurring so that consumers wouldn't have to get their systems repaired. However, both companies also offered free repairs for systems already exhibiting symptoms of a failing GPU, such as no video output to the monitor or the computer failing to boot. Both companies also offered varying limited warranties after impacted devices were repaired. HP offered a limited warranty for 24 months (two years) after the start of customers' original limited warranty or 90 days (approximately three months) after the notebook was repaired- whichever was later.<ref name=":7">{{Cite web |last=HP |date=2008 |title=HP Pavilion dv2000/dv6000/dv9000 and Compaq Presario v3000/v6000 Series Notebook PCs -  HP Limited Warranty Service Enhancement |url=http://h10025.www1.hp.com/ewfrf/wc/document?lc=en&cc=us&dlc=en&docname=c01087277 |url-status=dead |archive-url=https://web.archive.org/web/20080710172852/http://h10025.www1.hp.com/ewfrf/wc/document?lc=en&cc=us&dlc=en&docname=c01087277 |archive-date=10 Jul 2008 |access-date=8 Jun 2025 |website=HP Customer Care}}</ref> Dell extended limited warranties for systems with these issues for 12 months (one year) from the original purchase date, with a maximum of up to 60 months (five years). In addition, they offered this even to customers whose warranties already expired, from the date the initial warranty expired.<ref>{{Cite web |last=BlueScreenDeath |first= |last2=Menchaca |first2=Lionel |date=23 Sep 2008 |title=Dell Warranty Extension Due to Nvidia Defect |url=https://www.dell.com/community/en/conversations/locked-topics-laptops-general/dell-warranty-extension-due-to-nvidia-defect/647e9a01f4ccf8a8de2c999e |url-status=live |access-date=8 Jun 2025 |website=Dell Community}}</ref><ref name=":8">{{Cite web |last=Meyer |first=David |date=19 Aug 2008 |title=Dell extends warranties after GPU fault |url=https://www.zdnet.com/article/dell-extends-warranties-after-gpu-fault/ |url-status=live |access-date=8 Jun 2025 |website=ZDNet}}</ref><!-- Quick note on the Dell Warranty citation (from the Dell Community forums): I could not find the real original source. The links to the original are dead, and not archived- so I had to make do. -->


==== Apple's Response - Macbook Pro, May 2007 - September 2008 ====
====Apple's Response - Macbook Pro, May 2007 - September 2008====
In May 2007, Apple released a version of the aluminum Macbook Pro that used the Nvidia GeForce 8600M GT GPU, and manufactured the computers with this GPU until September 2008. It just so happened that this was one of the GPUs affected by Bumpgate. Unlike the issues with the PlayStation 3 and Xbox 360 where the system wouldn't even boot, consumers ran into distorted video or no video output on their devices. According to Gizmodo, Nvidia had told Apple that the graphics processors were not defective, so Apple initially ignored reports expressing that possibility.<ref>{{Cite web |last=Buchanan |first=Matt |date=10 Oct 2008 |title=Apple Confirms Failing Nvidia Graphics Cards in MacBook Pros, Offers Free Repairs and Refunds |url=https://gizmodo.com/apple-confirms-failing-nvidia-graphics-cards-in-macbook-5061605 |url-status=live |access-date=7 Jun 2025 |website=Gizmodo}}</ref> However, after doing their own investigation, Apple had found that the processors were indeed defective. Because of this, Apple offered extended repair coverage adding up to four years from the date of original purchase, and refunded customers who already paid to repair their systems affected by this defect.<ref>{{Cite web |last=Apple Support |date=18 Nov 2014 |title=MacBook Pro: Distorted video or no video issues |url=http://support.apple.com/en-us/HT203254 |url-status=dead |archive-url=https://web.archive.org/web/20141202230527/http://support.apple.com/en-us/HT203254 |archive-date=2 Dec 2014 |access-date=7 Jun 2025 |website=Apple}}</ref>
In May 2007, Apple released a version of the aluminum Macbook Pro that used the Nvidia GeForce 8600M GT GPU, and manufactured the computers with this GPU until September 2008. It just so happened that this was one of the GPUs affected by Bumpgate. Unlike the issues with the PlayStation 3 and Xbox 360 where the system wouldn't even boot, consumers ran into distorted video or no video output on their devices. Nvidia had assured Apple that the graphics processors were not defective, so Apple initially ignored reports expressing that possibility.<ref name=":9">{{Cite web |last=Foresman |first=Chris |date=10 Oct 2008 |title=Apple: NVIDIA chips to blame for MacBook Pro video problems |url=https://arstechnica.com/gadgets/2008/10/apple-nvidia-chips-to-blame-for-macbook-pro-video-problems/ |url-status=live |access-date=8 Jun 2025 |website=Ars Technica}}</ref><ref>{{Cite web |last=Buchanan |first=Matt |date=10 Oct 2008 |title=Apple Confirms Failing Nvidia Graphics Cards in MacBook Pros, Offers Free Repairs and Refunds |url=https://gizmodo.com/apple-confirms-failing-nvidia-graphics-cards-in-macbook-5061605 |url-status=live |access-date=7 Jun 2025 |website=Gizmodo}}</ref> However, after doing their own investigation, Apple had found that the processors were indeed defective. Because of this, Apple offered extended repair coverage adding up to four years from the date of original purchase, and refunded customers who already paid to repair systems affected by this defect.<ref>{{Cite web |last=Apple Support |date=18 Nov 2014 |title=MacBook Pro: Distorted video or no video issues |url=http://support.apple.com/en-us/HT203254 |url-status=dead |archive-url=https://web.archive.org/web/20141202230527/http://support.apple.com/en-us/HT203254 |archive-date=2 Dec 2014 |access-date=7 Jun 2025 |website=Apple}}</ref>


====Nvidia's Response====
====Nvidia's Response - Inquirer Accusations and SEC Report====
[[File:Nvidia G80, G84 and G86 size comparison.jpg|alt=Three GPU dies are lined up in a row, from largest to smallest, left to right: the Nvidia G80, then the G84, then the G86. The internals of the dies are visible.|thumb|Image of the Nvidia G80, G84, and G86's dies (left-to-right). The G84 and the G86 GPUs are known to have been impacted by the Bumpgate defect.<ref name=":5">{{Cite web |last=Demerjian |first=Charlie |date=9 Jul 2008 |title=All Nvidia G84 and G86s are bad |url=http://www.theinquirer.net/gb/inquirer/news/2008/07/09/nvidia-g84-g86-bad |url-status=dead |archive-url=https://web.archive.org/web/20080710121746/http://www.theinquirer.net/gb/inquirer/news/2008/07/09/nvidia-g84-g86-bad |archive-date=10 Jul 2008 |access-date=7 Jun 2025 |website=The Inquirer}}</ref>]]
[[File:Nvidia G80, G84 and G86 size comparison.jpg|alt=Three GPU dies are lined up in a row, from largest to smallest, left to right: the Nvidia G80, then the G84, then the G86. The internals of the dies are visible.|thumb|Image of the Nvidia G80, G84, and G86's dies (left-to-right). The G84 and the G86 GPUs are known to have been impacted by the Bumpgate defect.<ref name=":5">{{Cite web |last=Demerjian |first=Charlie |date=9 Jul 2008 |title=All Nvidia G84 and G86s are bad |url=http://www.theinquirer.net/gb/inquirer/news/2008/07/09/nvidia-g84-g86-bad |url-status=dead |archive-url=https://web.archive.org/web/20080710121746/http://www.theinquirer.net/gb/inquirer/news/2008/07/09/nvidia-g84-g86-bad |archive-date=10 Jul 2008 |access-date=7 Jun 2025 |website=The Inquirer}}</ref>]]
''See also: Lawsuit(s)''
''See also: Lawsuit(s)''<blockquote>''[As of July 2, 2008, all] newly manufactured products and all products currently shipping in volume have a different and more robust material set. [...] We intend to fully support our customers in their repair and replacement of these impacted MCP and GPU products that fail.''


Nvidia processors were not the only ones affected by the defect (i.e., the Xbox 360's ATI Xenos GPU), but they seem to be the most heavily impacted. There were a wide variety of Nvidia graphics processors across multiple architectures impacted by this defect, but the earliest confirmed system with the defect appears to have been manufactured in December 2005, and the latest systems were manufactured in late February 2010. However, the defects were being noticed broadly amongst consumers around July 2008, particularly when Gizmodo and The Inquirer reported on the problems.<ref>{{Cite web |last=Buchanan |first=Matt |date=3 Jul 2008 |title=Lots of Nvidia Laptop Graphics Cards Are Overheating, Dying |url=https://gizmodo.com/lots-of-nvidia-laptop-graphics-cards-are-overheating-d-5021713 |url-status=live |access-date=7 Jun 2025 |website=Gizmodo}}</ref><ref name=":5" />
- Martin Burkett, Nvidia Chief Financial Officer, Form 8-K report to SEC.<ref name=":10">{{Cite web |last=Burkett |first=Marvin |last2=United States Securities and Exchange Commission |date=2 Jul 2008 |title=Form 8-K |url=https://www.sec.gov/Archives/edgar/data/1045810/000119312508145974/d8k.htm |url-status=live |access-date=8 Jun 2025 |website=SEC}}</ref></blockquote>Nvidia processors were not the only ones affected by the defect (i.e., the Xbox 360's ATI Xenos GPU), but they seem to have been the most heavily impacted. There were a wide variety of Nvidia graphics processors across multiple architectures impacted by this defect, but the earliest confirmed system with the defect appears to have been manufactured in December 2005, and the latest systems were manufactured in late February 2010.


[Discuss initial response. Briefly touch on the lawsuit; there's a dedicated section below]
The defects were being noticed broadly amongst consumers around July 2008, particularly when ''The Inquirer'' published reports that drew attention to the problems.<ref>{{Cite web |last=Buchanan |first=Matt |date=3 Jul 2008 |title=Lots of Nvidia Laptop Graphics Cards Are Overheating, Dying |url=https://gizmodo.com/lots-of-nvidia-laptop-graphics-cards-are-overheating-d-5021713 |url-status=live |access-date=7 Jun 2025 |website=Gizmodo}}</ref><ref name=":5" /> Charlie Demerjian, reporter for ''The Inquirer'', firmly presented claims and evidence that every G84 and G86 GPU was defective- including desktop GPUs- even accusing Nvidia of attempting to cover up the problems.


<!-- I'm aware that these are not the only companies that had GPUs affected by Bumpgate; I'm just naming off the ones I personally know the most about for now. If someone comes across this and knows more about another affected company's response, please feel free to add it! -->
On July 2nd, 2008- a few days before Demerjian's article was published- Nvidia filed a report with the Securities and Exchange Commission (SEC).<ref name=":10" /> The report explained that the corporation would pay a $150-200 million one-time charge to cover customer warranties, repair, returns, replacements, and other notable expenses caused by poor packaging material in some of their multi-chip packages (MCPs) and GPUs exclusively used in laptops. This report also confirmed that all of their newly manufactured products from that point forward would have a more suitable material set.
==Lawsuit(s)==
{{Placeholder box|If applicable, add any information regarding litigation around the incident here.


===Claims===
As Joel Hruska from Ars Technica explained, if Nvidia was trying to cover up the defect- as Demerjian claimed- with this report to the SEC, they not only attempted to avoid responsibility and accused their suppliers of causing the problem, they also committed financial fraud by intentionally lowballing their expected financial losses- a major accusation that could have had severe consequences for Nvidia.<ref name=":11">{{Cite web |last=Hruska |first=Joel |date=16 Jul 2008 |title=NVIDIA denies rumors of faulty chips, mass GPU failures |url=https://arstechnica.com/gadgets/2008/07/nvidia-denies-rumors-of-mass-gpu-failures/ |url-status=live |access-date=8 Jun 2025 |website=Ars Technica}}</ref> However, it is difficult to verify if Nvidia was lying or simply not fully aware of the scale of Bumpgate. Nvidia's public acknowledgement of the defect in the SEC report is consistent with when Sony switched the PlayStation 3 to the non-defective 65nm Nvidia RSX,<ref name=":6" /> and it also seems to be consistent with when Dell<ref name=":8" /> and HP<ref name=":7" /> discovered the problem. The only company it does not appear consistent with is Apple, who discovered the defect in their Macbook Pro systems after their own investigation in September 2008.<ref name=":9" /> Even so, it is possible that Nvidia did not know that the GeForce 8600M GT GPUs in the 2007-2008 Macbook Pro were impacted yet when Apple asked them about it.
 
Regardless of if Nvidia was truthful in their report or not, they denied the claims that individuals like Demerjian were making- that their GPUs were defective and failing en masse.<ref name=":11" /> However, by 2010, consumers' trust in Nvidia had eroded to the point that a class action lawsuit was filed because of the defect.
 
==Nvidia Lawsuit (2010)==
 
=== Claims ===
Main claims of the suit.
Main claims of the suit.


===Rebuttal===
=== Rebuttal ===
The response of the company or counterclaims.
The response of Nvidia or counterclaims.


===Outcome===
=== Outcome ===
The outcome of the suit, if any.}}
The outcome of the suit. (Settlement)


===Nvidia Lawsuit===
==Consumer response==
==Consumer response==
{{Placeholder box|Summary and key issues of prevailing sentiment from the consumers and commentators that can be documented via articles, emails to support, reviews and forum posts.}}[General Consumer Response (frustration at/satisfaction with Microsoft, Sony, etc.)]
{{Placeholder box|Summary and key issues of prevailing sentiment from the consumers and commentators that can be documented via articles, emails to support, reviews and forum posts.}}[General Consumer Response (frustration at/satisfaction with Microsoft, Sony, etc.)]
Line 80: Line 83:


===Nvidia Consumer Response===
===Nvidia Consumer Response===
Based on the fact that this problem affected the Xbox 360's ATI graphics processor, Nvidia was likely not fully responsible for the defect. However, their poor response to the defect ultimately left consumers very angry with the company- enough to prompt a class-action lawsuit.
Based on the fact that this problem affected the Xbox 360's ATI graphics processor, Nvidia was likely not fully responsible for the defect. However, their appearance of a poor response to the defect ultimately left consumers very angry with the company- enough to prompt a class-action lawsuit.


===Misdiagnosis/Poor Repair of Faults===
===Misdiagnosis/Poor Repair of Faults===