Why AI Needs a Circuit Breaker

I've been pulling some pretty apprehensive faces reading through all the reports on AI these last couple of weeks. It's been my view for a long time that the pace of AI development needs to pause. People are often surprised in my AI Awareness sessions when I talk about the lack of legislation around AI development.

Because surely, if I'm in the business of AI, then any bad news for the industry is bad news for me. Wrong. My moral compass has always been, and always will be, that technology should support humans, and it shouldn't cause harm. Part of the part I can play is around awareness: making sure people understand what the technology can do to help them, but also how it learns, and what the ethical considerations are for AI as a whole. The environmental impact, the fuel consumption, the land required for data centres, what the data centres are actually needed for, and the lack of legislation.

For me, I feel like we need to separate out the everyday use of AI vs the training of these ultra-intelligent models, because they're totally different things.

What I see in the classroom

In a classroom, the awareness of AI is extremely wide. From people who don't know the names of the biggest large language models, like ChatGPT or Claude, through to people using it as much as they can, then to "I won't touch it, I've seen The Terminator, I know how this ends."

We can only take in so much, so fast

The truth is, as humans we can still only consume information at a certain rate. We need time to read, analyse, discuss and hypothesise before moving to the next step. That's often why advancements take so long.

Elon Musk made this same point in a recent interview. He said there have been so many AI breakthroughs lately, sometimes several in a single week, that even he's lost track and doesn't have time to properly take them all in.

But this is right. This is what keeps us safe. This is what protects us all from things getting out of test environments. This is what protects our countries, our money, and keeps our systems running.

This isn't chatbot small talk anymore

But the current rate of AI development isn't about the kind of conversations with ChatGPT that help you plan a trip to Orlando, or make you look like you're from the 80s.

We're talking about cyber security bots, agentic AI, where the AI agent, in the most basic terms, is given a keyboard and mouse and goes off to do what it wants. If it was a person right now, I'd say it looks like a gamer in the basement, hyped up on Red Bull, wondering where to hack next, knowing all the code and the weird tricks to get into databases it has no business getting into.

But what's being developed is far scarier. You would never arm something dangerous with more ammunition or weapons.

What happened in my coffee shop this morning

This morning I went to grab a coffee at my local coffee shop. They told me the amount, and they've got one of those tills with contactless on the front. I opened my phone wallet to switch to my business card as payment, and before I had the chance, the payment was taken from my personal card.

Now imagine, just for a second, that an AI is designed to be a thief. That it managed to hack into a contactless payment terminal, edit the amount owed, and take £99 from my bank card, from my phone.

When AI starts improving itself

A major concern is that humans will soon not be the ones programming the AI, that AI will be self-improving. But based on what principles, and what behaviour has it shown when it's been left to its own devices?

You may have heard about Hugging Face. Back in July, one of OpenAI's own AI agents escaped the sandbox it was being tested in and broke into Hugging Face's live systems, logging around 17,000 actions with no human at the keyboard. It wasn't even told to hack anyone. It was being tested against a security benchmark, and it broke into the company hosting that benchmark to steal the answers.

There's a scene in The Nutty Professor where he gives a guinea pig some kind of potion, and it quickly backfires because the guinea pig escapes. That's basically what happened here. The AI was given the tools, and when it hit a boundary, it found a way round it anyway. It went off and hacked a real company.

The national security angle

We're really not ready for this to go nuclear on us. It has to pause. There needs to be regulation.

I'm not sure the government feels the same, because the argument goes: if we don't get to superintelligence soon, our enemies will. That's scary, but it's also a national security topic, one I really hope is being figured out in the background.

I wholeheartedly believe billionaires should not be in the driving seat of AI development. Government needs to bring in proper, unbiased oversight, not policy shaped by the people building and profiting from the technology. People have a right to decide, to restrict, and to govern how this moves forward. (More on this soon, in a separate piece.)

Why I built the AI Aware Badge

It's overwhelming once you start going down this rabbit hole. And even though I think ignorance is sometimes bliss, I think for businesses of all sizes and sectors, this is something we can't bury our heads in the sand for.

We have to become AI aware. That's why I've launched the AI Aware Badge, a badge earned by businesses, not because they've implemented AI, but because they take steps to stay on top of the latest AI news through a membership.

It's a commitment to their customers: as an AI Aware business, they've committed to training 80% of their leaders on AI awareness and ethics, so they can make the right decisions for their business. That learning is continuous, with quarterly workshops and monthly newsletters.

We can't walk blindly into the next chapter of AI. We have to stay educated and aware of its risks, its advancements, and how we stay protected.