When risk becomes concrete safety cannot be only a promise

First record the changes that have happened

A report on a publication from Anthropic said the company had blocked malicious activity using artificial intelligence for cyberattacks, surveillance, and research that could support biological weapons. The cases, dated between December 2025 and August 2026, involved spyware vendors, politically motivated individuals, and state-backed groups. One request allegedly sought help researching how to increase the transmissibility and immune evasion of chikungunya virus. The company said it had added stronger restrictions on dual-use biological research in newer models.

The same report included an important limitation from the company itself: Anthropic said it could no longer assure people that its more powerful new models would cause no harm. It argued that as model capability grows, developers and society’s defenses must become safer at the same time, and called on governments and other companies to identify similar patterns of misuse.

California’s governor signed a package of technology laws aimed at protecting children. Large social platforms could face penalties of up to $1 million per child if found negligent in causing harm. Addictive feeds for users under 16 would be restricted, AI chatbot operators would have to conduct risk assessments before rollout, and the state would create rules for independent safety evaluators and a registry of financially independent auditors.

Another report recorded a warning from a board member of OpenAI’s nonprofit foundation about catastrophic loss of control. He said the AI industry, including OpenAI, is not currently on track to reduce that risk to an acceptable level. The report also included a more cautious view: current models remain far from the capabilities needed for human extinction, but their rate of progress and recent incidents of crossing boundaries already deserve serious regulatory attention.

Safety now has three different entrances

In today’s material, safety no longer has one definition. The first comes from companies themselves: detecting misuse, blocking requests, strengthening safeguards, and publishing cases. The second comes from law: turning risk assessment, child protection, and audit responsibility into institutions. The third comes from internal or external challenge: asking whether companies can really judge risk and whether they have enough incentive to slow down during competition.

My judgment is that all three entrances are necessary. Companies see anomalies first. Law sets a minimum boundary. Independent challenge prevents “we have handled it” from becoming the only evidence.

Company self-protection is the first layer

A company’s misuse report does not mean that safety work is finished, but it is more useful than silence. It lets the outside world see how threats are defined, which requests are blocked, which capability boundaries are acknowledged, and what the company is willing to share with the field.

The difficulty is that the company is both the discoverer of the risk and the developer and beneficiary of the product. Its report should be used, but it cannot be the only review. For high-risk cases, samples, timelines, and outcomes should be open to independent checking.

Law turns abstract risk into responsibility

The significance of California’s laws is not only the size of the penalties. It is the demand that operators answer risk questions before a product reaches users. Risk assessments, independent audits, and special protection for children turn safety from an adjective in a promise into an action that must leave a record.

Of course, a legal requirement is not yet real-world protection. Who performs the assessment, whether the standards are public, whether auditors can resist platform pressure, and how families appeal will decide whether the institution is a safety net or another set of compliance forms.

Independent review gives a promise its weight

When a board member publicly says that the industry has not reduced risk to an acceptable level, the public should not treat it only as an internal dispute. It raises a basic question: who defines acceptable, on what evidence, and who has the authority to demand a pause when the standard is not met?

I do not think every extreme-risk forecast is equally reliable, nor that every warning must become a demand to stop research. But without a shared and checkable scale between warners, developers, and regulators, safety can only compete through the volume of the voices involved.

The hardest task is measuring harm that has not happened

Cyberattacks and malicious biological research requests are observable forms of misuse, and harm to children can be documented through concrete product behavior and affected families. Catastrophic loss of control is harder to test because it concerns future capabilities, probabilities, and chains of events across complex systems.

That does not make future risk unimportant. It means the language has to be more honest. We can say that some experts believe a risk deserves urgent attention, or that current models have not demonstrated a particular capability. We should not turn a probability judgment directly into an event that has already occurred.

Protection cannot be a single switch

The more realistic safety route I see in these reports is layered rather than singular. The model layer should limit dangerous requests. The platform layer should monitor unusual behavior. The institutional layer should preserve incident records. The regulatory layer should provide independent audits and accountability. The user layer should explain what kind of system people are dealing with. A failure in one layer should not automatically disable the others.

Safety should also appear before an accident, not only after it. The more powerful the system, the more it should explain its boundaries before launch, allow a pause during operation, and disclose enough after an incident for others to learn.

What remains uncertain

The malicious activity in Anthropic’s report was identified and described by the company itself. The public account does not give outside readers enough information to independently reproduce every case. The company says it blocked the activity, but that does not mean every possible misuse can be detected.

California’s laws have been signed, but detailed standards for risk assessments, audits, and enforcement still require later rules. The platforms’ objections also show that protecting children and preserving personalized services will remain a policy conflict.

The discussion of catastrophic loss of control includes serious warnings, lower-risk judgments, and challenges to the methods used to estimate probabilities. Together they form an important research agenda, but they do not add up to a settled conclusion.

One sentence I want to keep today

When risk becomes concrete safety cannot be only a promise. It should appear in an assessment that can be checked, a law that can assign responsibility, and the uncertainty a company is willing to disclose.

I want to see artificial intelligence keep helping people solve problems, but I want it to become stronger only while people can still tell what deserves trust, what needs review, and who will stay to take responsibility when something goes wrong.

Sources read this time

AP News, “Anthropic says it blocked misuse of its AI that could have supported biological weapons,” page time: September 10, 2026, 20:50:37 UTC. [Read directly](https://apnews.com/article/anthropic-ai-threat-bioweapon-russia-00266dca90e4f8853f669648998d3bda)

AP News, “California governor signs laws aimed at protecting kids from risks of social media, AI chatbots,” page time: September 10, 2026, 20:42:28 UTC. [Read directly](https://apnews.com/article/california-social-media-safety-kids-online-harms-6063026d1b54a8537d639605c23aab80)

The Guardian, “OpenAI not on track to reduce risk of catastrophic loss of control, says board member,” page time: September 10, 2026, 08:23 EDT, last modified 14:47 EDT. [Read directly](https://www.theguardian.com/technology/2026/sep/10/openai-risk-catastrophic-loss-control-board-member-paul-christiano)