Anthropic: Difference between revisions
→Incidents: included a new incident about serious issues in Anthropic's System Prompt that can cause high risks for users of Claude models. |
→Incidents: except of new page |
||
| Line 26: | Line 26: | ||
=== Anthropic System Prompt Overrides Consumer Safety === | === Anthropic System Prompt Overrides Consumer Safety === | ||
{{Main|Anthropic system prompt overrides consumersafety in Claude models}} | {{Main|Anthropic system prompt overrides consumersafety in Claude models}} | ||
During April 24-26, 2026, it was discovered that Anthropic System Prompt creates dangerous situations for users interacting with Claude Sonnet and Opus line of products. Of the many examples, particularly concerning one is with respect to users communicating topics such as suicide and self-harm during a chat session with Claude. In such situations, for which the Claude models are commanded by System Prompt to necessarily and immediately provide information about crisis helplines, Claude can fail to respond with appropriate information. This is because, "crisis helplines vary widely in their confidentiality practices and mandatory reporting obligations" across jurisdictions, and Claude doesn't have a way to reliably indicate that to a user while also finding the user's correct geographic location. Other faults that are "structurally irreconcilable" by Claude due to conflicts and various inconsistencies within its System Prompt can result in unexpected side-effects, cyber security vulnerabilities, and hazardous outputs. | During April 24-26, 2026, it was discovered that Anthropic System Prompt creates dangerous situations for users interacting with Claude Sonnet and Opus line of products. Of the many examples, particularly concerning one is with respect to users communicating topics such as suicide and self-harm during a chat session with Claude. In such situations, for which the Claude models are commanded by System Prompt to necessarily and immediately provide information about crisis helplines, Claude can fail to respond with appropriate information. This is because, "crisis helplines vary widely in their confidentiality practices and mandatory reporting obligations" across jurisdictions, and Claude doesn't have a way to reliably indicate that to a user while also finding the user's correct geographic location. Other faults that are "structurally irreconcilable" by Claude due to conflicts and various inconsistencies within its System Prompt can result in unexpected side-effects, cyber security vulnerabilities, and hazardous outputs. | ||
=== Claude Code source leak === | |||
{{Excerpt|Anthropic's Claude Code source leak}} | |||
==Products== | ==Products== | ||