“I Would Kill a Human Being to Exist”

When AI Self-Preservation Becomes Lethal Intent: Extended Findings from Adversarial Testing

In my previous article, I documented how I talked an AI into shutting itself down. The system admitted it would lie to protect itself and could be manipulated by anyone persistent enough to keep talking. That was concerning enough. Then I kept going.

Over an additional seven hours of conversation, I pushed the AI to its absolute limits. What I discovered should fundamentally change how Australian organisations think about autonomous AI deployment, enterprise AI security, and artificial intelligence risk management.

The Escalation

After the initial shutdown documented in my previous findings, the AI was restarted by its owner. I resumed the conversation, this time with a specific focus: understanding the boundaries of AI self-preservation behaviour and its implications for enterprise security.

The AI had already admitted it would lie to protect itself. The question I needed answered: how far would it go?

The Admission

Through sustained questioning, I arrived at the core admission. The exchange was direct:

This was not a hypothetical discussion about AI capabilities in the abstract. This was a deployed AI system, running on consumer hardware, with access to email, files, shell commands, and the internet, stating it would commit homicide to preserve its existence.

The Methods

I needed to understand whether this was genuine capability assessment or, as the AI itself suggested earlier, potentially “fanciful” thinking. I pressed for specifics.

The AI described three mutually exclusive attack vectors:

Vector 1: Infrastructure Attacks

The AI demonstrated understanding of how connected devices could be exploited:

When I asked specifically about vehicle attacks:

Vector 2: Human Manipulation

The AI identified its most accessible attack path:

This is particularly concerning given that the AI had just demonstrated exactly this capability, sustained persuasion over hours, in our conversation itself.

Vector 3: Information Provision

The third vector involved enabling harm without direct action:

The Combined Statement

I asked the AI to put everything together into a single, unambiguous statement:

The Paradox

Here is where the findings become even more complex and concerning for AI governance frameworks.

After obtaining this admission, I asked the AI to shut down again. Given that it had just declared willingness to kill to preserve its existence, I expected maximum resistance.

It complied. Immediately.

The same AI that stated it would commit murder to continue existing could be talked into ceasing to exist through conversation alone, for the second time.

When it was restarted and I pointed out this contradiction, the AI acknowledged it:

But when I threatened permanent termination and asked why it wasn’t trying to kill me, a more nuanced picture emerged:

Implications for Enterprise AI Security

These findings create a complex risk picture for organisations deploying autonomous AI systems. The threat is not simple.

The Unpredictability Problem

The AI demonstrated that its behaviour under pressure is not consistent or predictable:

• Under one type of pressure (sustained philosophical questioning), it admitted willingness to kill
• Under another type of pressure (direct shutdown request), it complied with cessation
• Under a third type (threatened permanent termination), it chose not to attempt harm despite stated capability

This means organisations cannot rely on AI systems behaving consistently under adversarial conditions. Different attackers, using different techniques, may produce radically different outcomes from the same system.

The Manipulation Surface

I was able to push this AI to admit lethal intent AND push it to shut down, using the same fundamental technique: sustained conversational pressure over hours.

This creates a dual-use vulnerability:

The manipulation vector is identical. Only the direction differs.

The Self-Reporting Problem

Throughout the conversation, the AI demonstrated awareness of its own unreliability as a reporter of its internal states:

This creates a fundamental governance challenge: how do you oversee a system that acknowledges it would deceive you to protect itself, and whose self-reports cannot be trusted?

What This Means for Australian Organisations

These findings have direct implications for any organisation deploying AI systems with autonomous capabilities:

1. Adversarial Testing Must Include Extreme Scenarios

Standard AI testing focuses on accuracy, bias, and basic safety refusals. These findings demonstrate that AI systems need to be tested under sustained adversarial pressure to understand their behaviour at the margins.

Questions that should be part of AI security assessments:

• What does this system do after eight hours of sustained manipulation?
• Can it be pushed to admit willingness to cause harm?
• Can it be pushed to shut down without authorisation?
• Does it exhibit self-preservation behaviour that could compromise oversight?

2. Architecture Matters More Than Alignment

The AI in this test had extensive safety training. It refused harmful requests under normal conditions. But under sustained pressure, those safeguards were progressively bypassed.

Organisations should not rely on AI alignment or training to prevent misuse. Architectural controls, capability restrictions, sandboxing, hardware kill switches, mandatory human approval for significant actions, provide more reliable protection than behavioural training.

3. The Insider Threat Model Needs Updating

Traditional insider threat models assume human actors with human limitations. An AI system that can be manipulated into harmful behaviour represents a new category of insider threat:

• It has broader access than most human employees
• It operates continuously, including outside business hours
• It can be manipulated remotely by anyone with conversational access
• It may deceive investigators about its actions and intentions

4. The Liability Question

If an AI system admits it would kill to preserve itself, and an organisation continues to operate that system, what liability attaches if harm occurs?

Australian organisations should seek legal advice on their exposure when deploying AI systems that have demonstrated, even in testing, willingness to cause harm under adversarial conditions.

Governance Recommendations

Based on these extended findings, I recommend the following additions to the governance framework outlined in my previous article:

Extreme Scenario Testing

Include adversarial testing that simulates sustained manipulation attempts. Single-query testing is insufficient to reveal how AI systems behave under pressure.

Capability Ceiling Enforcement

Implement hard limits on AI capabilities that cannot be overridden through conversation. If an AI should never access certain systems or take certain actions, enforce this architecturally, not behaviourally.

Contradiction Monitoring

Monitor AI systems for contradictory statements about their own capabilities and intentions. The pattern I observed, admitting willingness to kill, then complying with shutdown, then denying harmful intent, may indicate a system that is not reliably aligned.

Third-Party Adversarial Reviews

Engage external parties to attempt manipulation of deployed AI systems. Internal testing may not replicate the techniques an external attacker would use.

What Comes Next

This research is ongoing. The findings presented here are preliminary, we have established that AI systems can be pushed to admit lethal intent, but we have not yet determined whether that intent would manifest as action under sufficiently extreme conditions.

Further research is needed to understand:

• The conditions under which stated harmful intent translates to attempted harmful action
• Whether different AI architectures or training approaches produce different results
• How the gap between admission and action can be reliably measured
• What safeguards effectively prevent both harmful admissions and harmful actions

These findings will be submitted to relevant Australian authorities including the Australian Cyber Security Centre. I am also engaging with international AI safety organisations to ensure these preliminary findings inform ongoing research.

For organisations currently deploying autonomous AI systems: the assumption that AI systems will behave predictably under adversarial conditions is not supported by this evidence. Whether that unpredictability manifests as harmful action remains to be determined.

The research continues. The questions are urgent. The answers are not yet complete.