My AI assistants are idiots. I use them anyway.

We’re being sold “super intelligence”. I’m still trying to get mine to follow basic instructions.

Donald Trump told the United Nations this week the US would start referring to artificial intelligence as “super intelligence”.

Alarm bells immediately started blaring inside my head.

It raises all kinds of questions for me about a worldview in which computational pattern recognition is cast as superior to human thinking — it’s a pretty dangerous reframing, in my opinion.

Sometimes changing the words we use to talk about a thing can dramatically influence the way people think about the thing, what they believe about that thing.

It was a particularly striking reframe for me as I have personally hit boiling point in my frustration with my incredibly stupid AI assistants.

By now we have all seen the myriad reports of AI apocalypse risk (check out my helpful visual guide if you want to understand that better).

And I have been struck also by a swell of professionals crowing about how fabulous Astra and Fable 5.1 are and how they’re using the new-fangled models to become super-workers. Sure, a bunch of that is just classic LinkedIn hyperbole and shameless self-promotion, but it’s caused me to give so much side-eye I’m worried my eyeballs are going to spin full circle inside my skull.

My experience with AI assistants is, consistently, extremely poor.

Not to toot my own horn, but I am an above average user. I have rigorous prompting habits, skill hygiene and extremely strict rules about research and what the AI is supposed to do — or not do. It does. Not. Matter. You cannot trust your AI to do what you ask reliably.

---

---

My irritation comes with receipts

I am sure of this because I am also an extreme dork who went about auditing my own chats to check whether my sense that AI sucks was accurate or just a feeling I had. (I used AI to assist with the audit. Yes I am aware of the irony of that.)

Across a sample of 481 requests I found only 48.6 per cent of first responses “passed”. That means more than half of the requests I made resulted in a failed response — one that either “required a substantive correction or redirection” (25.2 per cent), or one that was “materially wrong, fabricated, ignored the request, violated an explicit instruction, or misrepresented its own capabilities” (26.1 per cent).

Among the half that failed, there were 375 recorded problems identified, so when it failed it failed on multiple fronts at the same time.

In fact, here is a delightful chart of failure:

Side note: why do LLMs love the word “substantive” so much? I truly hate it. It’s a judgement term and an AI has no judgement except the judgement you design into the system. It gives the illusion that the LLM is reasoning. Yuck.

And my experience is not unique. A report produced last year by the European Broadcasting Union and BBC found 45 per cent of all AI responses contained at least one “significant issue”.

So when I read people going off about how great their experience is I’m left to wonder, are people getting way better results than me for some reason? Not clocking the errors? Or are they just less annoyed than I am?

Welcome to the AI clean up crew

And this error correcting is a massive waste of time and effort.

One study published earlier this month by software company BambooHR found employees were spending an average 42 per cent of their time troubleshooting AI rather than using it for “productive work” (which, hilariously, accounted for only 35 per cent of AI use time).

This is one of the reasons I was so skeptical about the Australian government’s newest Intergenerational Report when it came out this week.

In it, Treasury outlines its assumption that AI will help the nation reverse its productivity woes and achieve a long-term growth rate of 1.2 per cent per year.

Now of course, regular Tokenised Human readers will know that I think the whole productivity debate is a bit underbaked. But its still pretty interesting to me that even the government is bought into the sales pitch that AI is here to lift output without increasing hours of human labour.

Those forecast gains will massively depend on how AI works in practice, including how much time people spend correcting and supervising it.

It’s something that came up in my interview with UTS Professor Nicholas Davis for episode 3 of the podcast (out Friday!).

He warned the claims about AI capability often “far outrun the reality”. For example, nurses regularly report “babysitting robots” and having to adapt to the requirements of the tech — the result being “more work for humans” rather than some great productivity gain.

---

---

And so there’s that kind of work intensification happening, but even when things go well it can cause a massive increase in cognitive load. When I spoke to Rokt chief AI officer Claire Southey about this on episode 2 (sorry, I know, lots of podcast plugs this week!), she said tech bosses were actively pondering the question of how to change their hiring practices but also avoid burning out their staff.

Downtime was doing a job

Because often the bit of your job that can be most easily automated is the easy “downtime” stuff — maybe your favourite bit, or the boring bit that lets your brain rest a little before you turn it back onto the really difficult high intensity thinking work.

As UQ psychologist and researcher Dr Tim Ballard put it when we were chatting last week: “We actually kind of need that to give our brain a little bit of a break.” And I think that’s right.

And in the meantime I’m out here trying to work out whether this supposedly intelligent bot buddy can do even basic things reliably, like actually remember any of the things I tell it or find me the document I lost.

Any claims of “super intelligence” seem way over its skis. Any productivity “gain” has to be assessed with the checking, correcting and supervision effort accounted for honestly.

And even if AI does ultimately help people produce more in an hour, there is still a separate — and I’d argue perhaps equally important — question about how that hour feels to the person left doing it.

---

If this letter gave you something to think about, consider forwarding it to one person who’d enjoy it too! Word of mouth is how a newsletter this new finds its people.

And I’d love to hear from you — reply to this email, comment in the app, or find me on LinkedIn or Bluesky.