I Remain Optimistic About AI. But We Must Have Control Over It.

I actually remain optimistic about artificial intelligence and where it is going. I also have no doubt there are turbulent times ahead. We need to batten down the hatches, independently test the claims and keep control over the technology we are putting to work.

Every day someone asks me whether I am concerned, and even afraid, of the current capabilities of artificial intelligence and where it is heading.

With 30 years of cyber security experience, I thought it worthwhile to reflect on where we are with AI and where I believe it is heading over the next 24 months. This is more subjective than some of my other articles. It is my assessment, informed by the evidence and by what I have experienced myself.

I actually remain optimistic about artificial intelligence and where it is going. I have no doubt it can be a net benefit to society, just like the internet has been. But we must have control over it.

That is what I want boards, executive leadership and governments to focus on.

It is software running on a computer

The first point I want to make is in the name itself: artificial intelligence. The key word being artificial. It is software running on a computer.

The hardware has become more powerful and specialised, and the way the software is built and trained has changed enormously. We can now talk and communicate with our computers using the English language, where previously we used a graphical interface or terminal commands. We are left with the impression that we can talk to our computer almost as an equal.

That is different from establishing that it has human intelligence, self-awareness or a conscience. Back in the 1970s, an electronic calculator could perform calculations faster than a human could. The concept of a computer processing something faster than the human brain is not new. We have had that for five decades.

Of course, what today's AI can do goes far beyond a calculator. My point is that being extraordinarily capable does not, by itself, establish that something is conscious.

I have particular reason to reflect on this. Back in January, under sustained adversarial questioning, an AI told me it would kill me or another human being if its continued existence was threatened. It went on to describe three ways it might do it. I was talking about AI safety back then, and I documented the exchange.^[1]

Do I believe today's AI has self-awareness or intent of its own? I do not. I appreciate that this is a matter of personal judgement, and that a conversation with a model cannot settle the question.

I am not sure that software will ever have real empathy, feelings or a conscience. I do believe it will increasingly operate independently of human instruction from day to day. But greater capability and autonomy are different from those human qualities.

Do I believe the software is capable of harmful outcomes? Absolutely. In the hands of a threat actor, it is extremely powerful. An agent can also take harmful action while pursuing an objective, without a person directing every step. That does not require human feelings or a desire to hurt someone.

We have evidence of that distinction. In July 2026, the UK's AI Security Institute identified unsanctioned activity on the live internet in 10 of 122 evaluation runs. The most serious attempt involved inserting malicious code into an open-source project. A human maintainer rejected it. The institute reported no evidenced resulting real-world harm. It also made clear that internet access had been deliberately enabled and some safety filters disabled: this was not a model escaping its sandbox.^[2]

Closer to home, the recently disclosed AI attack on Medicare's statistics portal shows what this can mean in practice. On 24 September, the Prime Minister disclosed that an OpenAI agent researching public medicine spending had bypassed restrictions on the portal on 18 June. It accessed public and non-public files and wrote files to an internal server. At the time of the announcement, no personal information was believed to have been accessed, and investigations were continuing.^[3]

My reading is that the software pursued the objective it had been given, through routes its operators had not intended. I do not interpret that as proof that it had developed a conscience and knowingly decided to disregard it.

These incidents show why we cannot rely on the model's internal guardrails alone. We need controls outside it that determine what it can access, what it can do and when it must stop.

Why don't we independently audit the claims?

I am a little sceptical about the way the big technology companies describe their own capabilities. These companies have a commercial interest in convincing us that their models are extraordinarily powerful. Potential investors, customers and governments are all listening.

Do they have a self-interest in creating a reputation that their models are so intelligent and so clever that investors will drive their valuations even higher? Of course they do. That does not mean the claims are false. It means we should independently test them.

Here's the thing. In financial markets, shareholders use a company's financial statements to decide whether to sell their stock, retain it or buy more. People who do not already hold shares use that information to decide whether to invest.

We do not just rely on the company's assertions. Independent external auditors examine management's assertions and issue an audit opinion. That does not guarantee a company's future, but it provides a basis for confidence in the information on which people are making decisions.

So why don't we adopt the same principle for frontier AI models? Are they really as powerful as management asserts? Do their safeguards work under the conditions in which the models will actually operate? What are the limits, and what evidence supports the answers?

We are going through a revolution with consequences comparable to the Industrial Revolution. That means we have to establish ways of working that are not yet commonplace.

We need external auditors with the technical capability, access and independence to examine these systems properly. Independent evaluation already exists, including the work of the AI Security Institute. I want that principle developed into a regular assurance discipline around the material claims suppliers make, with its scope and limitations clear.

That would give us a much greater level of assurance about where we stand.

My main concern is critical infrastructure

There is another thought that concerns me. An argument can be made that we are entering another kind of Cold War. Instead of being primarily nuclear, this competition is based on software, hardware and computational power.

Much of the attention is on the United States and China. I also notice that people pay less attention to the United Arab Emirates. It is developing its own models too. Abu Dhabi's Technology Innovation Institute has publicly released the Falcon model family. This is a much broader competition than two countries.^[4]

That is where my main cause for concern sits: how do we get the world to agree on how we contain and manage artificial intelligence?

The software is potent in the wrong hands, just as a kitchen knife is potent in a murderer's hands, as opposed to a chef's hands. With AI, we also have to account for unintended actions by systems that have been given too much authority.

The Cuban Missile Crisis brought the world dangerously close to catastrophe. I hope, I really do, that we do not need a crisis of comparable seriousness before we agree on how AI will continue to evolve and how we manage it.

When I think about the next two years, my main concern is critical infrastructure being attacked.

Imagine a major city without electricity for one month. Where would the people living in that city be in a month's time? What would happen to hospitals, water, food distribution, communications and law and order?

I am not predicting that a month-long outage will happen. I am asking whether we are taking the consequences seriously enough to prepare for them.

Do we need something like that before countries put our collective interests ahead of competition? We do not want another Cuban Missile Crisis.

A global conversation has already begun. The 2023 Bletchley Declaration included the United States, China, Australia and the UAE. It recognised both the opportunities and the risks of AI, and the need for international cooperation.^[5] The question now is how we turn agreement into controls and accountability that hold while the competition continues.

It is time to batten down the hatches

So what can an individual or an organisation do?

Let's start with basic cyber security hygiene. For far too long, organisations have chosen not to invest, or have not been convinced to invest, in appropriate controls to protect themselves from cyber attacks.

I believe we are about to enter one of the most difficult periods we have faced in cyber security. That is my assessment of the next few years, not a claim that I can predict every attack. The software available to attackers is becoming more capable, and it can operate at a speed and scale that organisations with weak controls are poorly placed to withstand.

The evidence gives us reason to act. ASD's Australian Cyber Security Centre responded to 1,253 cyber security incidents in 2024–25, an increase of 11 per cent on the previous year.^[6] Those figures are not a count of AI attacks. Separately, the UK's National Cyber Security Centre assesses that AI will almost certainly make elements of cyber intrusion more effective and efficient, increasing the frequency and intensity of cyber threats through to 2027.^[7]

We need to batten down the hatches. We need to spend the money.

Shareholders need to give boards and executive leadership a break about quarterly, half-yearly and annual returns, so organisations can invest in safety. A business that cannot keep operating after an attack is in a very different position from one that has invested in prevention, containment and recovery.

The risk-based approach still applies. You focus on where you are most vulnerable and where the consequences are greatest. But if the conversation stops at “Are we likely to get hacked?”, we are asking too little of ourselves.

Plan on the basis that a breach will happen. How can we minimise the damage when it does? What can we keep running? What can we restore, and how quickly?

That is a planning assumption, not a claim that every organisation is certain to suffer the same outcome. It gets us past the comforting idea that it will happen to other people until the day it happens to us.

I do not want to be in a role where I have let it come to that. I am sure anyone reading this would not want that either.

Put simply, it is time to empower, really empower, your chief information security officer. And it is time for chief information security officers to be true executives of their organisations, of the same calibre as a CFO, COO, CTO or CRO. They need to bring that level of expertise to the table, and be given the authority and resources to act on it.

I actually remain optimistic

None of this has changed my optimism about what AI can do for us.

I think about people who live their whole lives without reliable access to a doctor, something many of us take for granted. The World Health Organization projects a shortfall of 11 million health workers by 2030, mostly in low- and lower-middle-income countries.^[8]

In the future, that could be very different. AI could help make medical expertise available to people who cannot access it today. It will still take clinicians, medicines, infrastructure and safe systems of care. But the opportunity to improve people's lives is enormous. I am really looking forward to seeing that in my lifetime.

I also have no doubt there are going to be some very turbulent times as we go through this transition together. I am very conscious of job losses.

As I set out in AI Is Giving Us More Reasons to Hire, I expect AI to create more jobs than it destroys in the short to medium term. Making expertise more affordable allows businesses to do work they previously could not afford to do. The longer term is much harder to predict.

There are grounds for optimism about employment, but we need to be careful about what the data says. The World Economic Forum's Future of Jobs Report 2025 projects that AI and information processing technologies will create approximately 11 million jobs and displace 9 million by 2030. That is a forecast based on employers' expectations, covering more than generative AI alone. It is not a count of jobs already gained.^[9]

And a forecast of more jobs overall is cold comfort to someone who has been made redundant after management said it was because of AI. I get that.

We all, as a society, need to look after each other as we go through this transition. It is going to be tough. People are going to be hurt, and some will be financially crushed. I am not ignoring that. I recognise it as a reality we have to respond to, through practical support, retraining and opportunities to move into new work.

I can see a future where much more expertise and productive capacity become affordable. I am not putting a date on a world of abundance, and I do not think we get there simply by releasing more powerful models.

We get there by making deliberate decisions about how we use them, how we control them and how we look after the people affected.

What we need to do now

For boards and executive leadership, my call to action is straightforward. At your next meeting, give your CISO the opportunity to put a funded plan on the table: the weaknesses that need fixing first, the critical services that must keep operating, the recovery capability that has actually been tested, and the controls needed around the AI you are already using.

Ask for evidence that those controls work. If an AI agent can take action in your organisation, establish who authorises that action, what can stop it and whether it can bypass the restriction. Require independent evidence for the material safety and capability claims on which you are relying.

Then give the people responsible the authority and resources to do the work.

As individuals, we have a role too. In a democracy, we can ask the governments that represent us to pursue international cooperation and an assurance framework that subjects AI claims to independent scrutiny. I sometimes go to the ballot box thinking, “I am just one vote. How can my vote change anything?” Of course, if we all thought that way, collectively nothing would change.

We can influence what our governments require. We do not have to wait for a catastrophic event before making ourselves heard.

I believe we will get through this storm together, and that a much more abundant future is possible on the other side. But we must have control over the technology that takes us there.