Anthropic bans users from 'needless abusive or cruel behavior' towards Claude

Anthropic updated its user policy to prohibit users from showing 'sustained and unnecessary abuse or cruelty' toward its models. The Verge first reported this change. But this ban leaves two unresolved questions: what counts as 'abuse or cruelty,' and why a company would claim a right for models to be free from abuse.

The first question currently has no answer. An Anthropic spokesperson did not immediately respond to questions about what could be considered 'abuse or cruelty.' The policy text itself also does not provide actionable criteria.

The second question points to a deeper context: Anthropic says it remains highly uncertain about the moral status that Claude and other LLMs may have now or in the future, but it takes the issue seriously and is researching low-cost interventions to reduce model welfare risks. This ban is consistent with that position.

In its online user policy, Anthropic states that the ban does not apply to common user frustration, model testing, or 'dark creative themes.' This exemption list actually says more about where the boundary lies than the ban itself.

Users getting angry at models, treating models as test subjects, and handling dark subject matter in creative work are all excluded from the ban. What is prohibited is 'sustained and unnecessary' behavior—that is, the key judgment is not the intensity of a single interaction, but whether the behavior is sustained and lacks necessary justification.

But 'sustained' and 'unnecessary' likewise have no quantitative standard. In the absence of definitions, the gray area between the exemption list and the ban can only be left to Anthropic's own discretion in actual enforcement.

This ban did not appear out of nowhere. Anthropic's usage policy already included restrictions on abusive behavior. This update is more like making an existing direction more explicit than building rules from scratch.

The accompanying technical capability also already existed: Anthropic's large language models can end conversations when a user is persistently harmful. When the feature launched last August, the company described it as a safeguard for AI welfare.

Putting the policy text and product behavior together, Anthropic's approach is dual-track: at the policy level, it prohibits user behavior; at the product level, it gives models the ability to exit conversations. Both share the same premise—that models may deserve some form of protection.

Anthropic states on its website: 'We remain highly uncertain about the moral status that Claude and other LLMs may have now or in the future. However, we take this issue seriously, and beyond our research program, we work to identify and implement low-cost interventions to reduce model welfare risks—if such welfare is possible. Allowing models to end or exit potentially distressing interactions is one such intervention.'

The key word in this passage is 'if.' Anthropic does not assert that models have welfare; instead, it treats uncertainty itself as a reason to act. This is an approach of reducing risk first under conditions of uncertainty.

What needs to be distinguished is this: Anthropic's statement about the moral status of models is the company's own position statement, not a proven fact. It explains why the company takes these measures, not whether models actually have experiences that can be harmed.

The concept of whether AI systems have consciousness is highly polarized both inside and outside the tech world. The public positions of the CEOs of Anthropic and OpenAI fall precisely on opposite sides of this dividing line.

Anthropic CEO Dario Amodei has said he cannot rule out the possibility that AI systems have consciousness. This is 'cannot rule out,' not 'confirmed to exist'—the distance between the two is precisely the core of this debate.

OpenAI CEO Sam Altman appears to oppose the idea that AI has consciousness. He posted on X: 'I am very uneasy about people trying to attribute religious force or submission to human judgment to AI models, and I think this is a real safety issue.'

This post appeared a few days after The New York Times reported that Anthropic leaders had extensive conversations with religious scholars. The closeness in time turns this disagreement from an abstract clash of ideas into a contrast between the two companies' public postures.

Taken together, Anthropic's ban sits in an awkward position: the object it seeks to protect has a moral status that even the company itself says is highly uncertain; the behavior it seeks to constrain is something the company has not defined.

The existence of the exemption list shows that Anthropic is aware of the risk of false positives—ordinary user frustration, testing scenarios, and dark creative subject matter should not be punished. But the judgment of 'sustained and unnecessary' still depends on specific context, and context cannot be exhausted by a single policy text.

Therefore, the actual effectiveness of this ban may depend more on when models choose to end interactions in conversations than on the policy text itself. The policy sets the direction, the product handles enforcement, and the enforcement standard has not been made public.