The real risk of AI: we stop trying

Sarah O’Connor is a columnist, reporter and associate editor at the Financial Times, who writes a weekly column on the world of work. Her book, We Are Not Machines: The Fight for the Future of Work (Allen Lane), investigates how AI and robotics are changing the nature of our jobs, and asks: what if the real risk is not that we are robotising our work, but that we are robotising ourselves?

Caitlin Allen (CA) – You’ve spent over a decade writing about the world of work. Why did you choose to write this specific book now, and how did you decide on which workplaces to focus your reporting?

Sarah O’Connor (SO’C) – I was feeling an itch to get out into the world and figure out what was really happening in workplaces on the ground, because I, like a lot of journalists, have an inbox that was being constantly flooded by AI predictions, economic modelling, bold statements from technology companies, offering utopian or dystopian messages that weren’t particularly illuminating. The journalism that I’ve always liked to do is to get out of the office and talk to people who are on the receiving end of this and see how it’s affecting their work in day-to-day life, and I felt like that was a bit missing from the conversation.

I chose to look at places where it felt as if the disruption was already taking place, because obviously it’s happening faster in some jobs than others. In the white-collar world, software and translation were two areas that I knew were already becoming quite disrupted. In terms of blue-collar work, I wanted to go and see this brand new Amazon warehouse, which is becoming highly automated with robots, because when I started to write about the world of work, I had gone to see one of the first ever Amazon warehouses when it opened in the UK. I was curious to know how that transition [to automation] was going because I’d always thought, this is an example of when surely it makes sense to automate. A lot of people who I interviewed there almost 15 years ago had said things like, ‘I feel as if I’m treated like a robot’. So my view had been, why not bring on the actual robots then? Not every job has to be mourned. Sometimes it’s a good idea to do away with dull jobs that treat people badly. So I wanted to know whether things were going as I’d hoped they would be.

CA – You begin the book by talking about a professional translator, Petr. What struck me was the level of passion with which he spoke about his job and viewed it as an incredibly creative role. Why do translators such as Petr feel as though the parts of their job they derive the most pleasure from are being lost?

SO’C – Petr translates subtitles for things like TV shows and video games from English into Czech, his first language. It was only by talking to him that I realised how much human judgement and care and craft goes into a job like that. 

One of the things Petr told me which I had never thought of is that when you’re translating subtitles from English into Czech, English only has one form of ‘you’, whereas in lots of languages, including Czech, there’s a formal and informal version. As the translator, you have to make judgement calls about which form the characters would be using in a situation. How well do they know each other? At some point, that might shift. They might start out a series or episode not knowing each other that well then by the end they’ve become more familiar. So there are all kinds of calls which require your own human discretion. The other thing he talked about was jokes. Jokes are so hard to translate, particularly puns or ones that rely on cultural knowledge. But he said that also makes them a really satisfying creative challenge. If you figure out a clever way of making a joke land in a different culture in a different language so that it’s still funny, you really feel like you’ve nailed it. My favourite example of the ingenuity of translators is the Asterix books. Obelix’s dog, in the original French book, is called Idéfix, which means ‘obsession’ in French, but the English translators called him Dogmatix, which, if anything, is an even better joke. There are lots of reasons why Petr loves translation and described it to me as his dream job. 

In terms of what’s happened to these jobs, it’s not that the entire act of being a translator has been wiped out. But agencies that mediate translation work have come up with a new model called ‘Machine Translation Post-Editing’. An agency puts subtitles through AI, gives the machine translation to someone like Petr and asks him to check it for accuracy, tidy it up and make it sound a bit more ‘human’. The problem is that the agency expects translators to do this at twice the pace for half the price, meaning they have to rattle through this work to maintain their average income. But also, a lot of translators have said that it’s very hard to be creative in your own solutions if you’ve got a solution sitting in front of you. It’s difficult to think from scratch about a different way of doing things. You end up with a job that feels faster, but also less creative, more mechanical, and more like being on a production line. 

What struck me when talking to Petr and some of his fellow translators is that this is the precise opposite of the promise of AI and automation technology, which was that it would do away with the dull and mechanical parts of our jobs to free us up to spend more time on the human bits, the creative bits, the relational bits. For them, it’s going precisely the opposite way. 

CA – Another example from your book of a highly counterproductive use of AI that stuck with me, perhaps because of my own line of work, is the copywriter who is forced to put her writing through an AI tool called a ‘humanizer’. Could you explain this case and what we can draw from it? 

SO’C – This was truly depressing to me, too, as a fellow professional writer, because you might think that AI detectors are the professional writer’s friend. It should be good for us that there is technology out there that purports to be able to tell the difference between AI writing and human writing. We know from all kinds of studies, and just from general market demand, that a lot of people really do care about whether a human wrote something that they’re reading, which should be good news for us as writers that demand for what we do still exists. The problem is distinguishing it from the AI copy.

This interviewee of mine works as a copywriter for a biotech company in Georgia. And she, again, really likes this job. She has to go out and research complicated scientific ideas then write them up in a clear way. What happened to her was that she got called in by a manager who said: ‘We have a problem. Google rankings of our webpages seem to be dropping and we think the issue is that Google is down-ranking things which it thinks that AI wrote.’ So her manager put her and her fellow copywriters’ articles through an AI detection system and they all came up as AI-written. She, my interviewee, was furious because she said it wasn’t true. She had written them herself and put a lot of effort into it. So she did something which I thought was quite funny. In the meeting with her boss, she sat while he was talking and typed out what he was saying in real-time, then turned her screen to him and put what he had just said into the same AI detector. It came back as substantially AI-written. This did make him laugh, apparently. 

But the problem remained, which was that Google still thought the pieces were AI-written. The solution presented to her was that she should write articles, put them through the AI detector, and if the percentage score on that was too high, she would have to put them through another system called a ‘humanizer’, which is itself an AI system. And that will change what she’s written in order to get past the AI detector. It’s already bleak that a human who’s written something is having to put her words through this AI sausage-maker called a humanizer. But what she found really depressing was that the humanizer made the language worse, adding spelling mistakes, grammar errors and just deforming it really. Even when demand for your skill exists, AI can still mess with your job in unexpected and not necessarily positive ways. 

I want to add that I was confronted with a dilemma towards the end of writing my book because I had this line in the conclusion which I’d written a long time ago and thought was a really elegant, pithy sentence that summed up the argument of the book. But, as I was reading one of the final edits, I looked at it again and, having spent the last year immersed in AI slop on places like LinkedIn, I thought to myself, oh God, that sounds like something ChatGPT or Claude would write. It was a good sentence but it used one of those structures which AI models have overused to the point of saturation that now we’re all sick of, plus everyone’s AI antenna goes off. I had a real dilemma over whether to keep it. Because I did write it and it seemed wrong to change my words to get out of the way of the machine, or should I change it because it might make people accuse me of using AI when I haven’t? I deliberated about it and, in the end, I deleted it and wrote a less elegant sentence which said the same thing. I still don’t know if that was the right call. 

CA – You mentioned that a lot of our predictions about the latest wave of automation have already proven to be wrong. Building machines that can perform basic, physical tasks has, in many cases, proven surprisingly difficult and yet we have machines that can mimic human cognition to quite a startling degree. A decade ago, we thought creative jobs would be the most shielded. Are they now the most vulnerable?

SO’C – Sadly, I do think they are among the more vulnerable ones, despite the fact that demand for human-created stuff – whether it’s music, novels, performances, film – is still strong. My concern is twofold. 

One is the issue we’ve just discussed, that figuring out the provenance of stuff is going to get increasingly hard. That’s going to make it difficult for creators, particularly freelance creators, to find a market and prove that they deserve to be there. There have been people coming up with labelling schemes for a ‘human-created’ product, a bit like fair-trade but for humans versus AI, though I’m sceptical that will work because it’s such a grey area. Everyone has their own view of how much AI has to be used or not used in order for it to qualify as human-made. And, as these systems become more integrated into Google, it’s going to be hard to be super purist about. 

The other part that worries me is that there are lots of jobs in creative industries that are somewhere in the middle of the process. AI so far can’t really come up with good new ideas, and that’s great, but a lot of people who do creative work and really value their jobs are somewhere along the production process. Whether it’s special effects in film or storyboards in advertising, lots of jobs are primed for some form of automation, with incentives for employers at the top to compress costs and increase speed. We’re starting to see this in Hollywood. We’re not seeing AI come up with new film scripts or replace actors, but there’s a lot happening behind the scenes, with different parts of the production process being sped up and automated. There’s a flip side as well: if it becomes quicker and easier to experiment and make films with fewer people, that might present more opportunities for people who have great ideas. I don’t think it’s wholly negative and I certainly don’t think that demand for creativity will end, but I do think it might be a rocky road for people in creative professions. 

CA – In your book, you remind us that anxiety about the impact of technological change in the workplace has a deep history. The term ‘Luddite’ is used today as an insult to suggest somebody is ‘anti-progress’, and yet the Luddites didn’t object to new technology in and of itself. They took issue with the way it was degrading their craft by producing poorer quality goods. How does that resonate with what’s happening today? 

SO’C – I thought about the Luddites a lot when I was talking to some of my interviewees because many of their complaints were quite similar. As you say, the Luddites were watching a job that they knew how to do well being replaced by people with less skill who, with the help of machines, were producing something of a lower standard. 

The framework knitters, who were among the Luddites, had a way of knitting stockings all in one piece so that they were robust. Then, employers came up with this new way of using machines and less qualified workers to make something called cut-ups, which is basically where you cut two pieces of cloth into the right shape and stitch them together. The Luddites argued that these were terrible quality, would fall apart after they’d been worn a few times and were dragging their trade into disrepute. Their complaint wasn’t just that they were being undercut by people with less skill who could do the job for less money than them. It was also that the product was worse. 

You can hear the same thing being said today by lots of people, including translators. There was a big study in Sweden recently, which compared the whole corpus of subtitles for a particular TV show, before and after the introduction of this Machine Translation Post-Editing system. Across every metric of quality, it found that the subtitles have got worse. There are more errors in them, the language is thinner and doesn’t reflect the richness of the original dialogue as much. But if you’re watching that show with subtitles, it’s probably because you don’t speak the original language, so this is a loss that we might not even realise we’re experiencing. A lot of translators told me that they were worried we would get used to that and just accept a world of lower-quality stuff. 

A few years ago, I was at my parents’ house and I found my old portable CD player that was given to me for my 13th birthday. It still worked. I put some headphones in and I couldn’t believe how good the sound was, just the richness of the audio. I did some research and realised that makes sense because on Spotify, particularly the free version, the quality of the sound is terrible. You might say that’s fine; plenty of people over the years have traded in a high-quality but expensive product for one that is lower quality but much cheaper. That’s a perfectly reasonable choice for consumers to make and often might be the right one. After all, the Luddites complained about those poor quality stockings but ultimately mechanisation allowed for lots more people to have more than one outfit to wear in a week. But what worried me with the CD player was that I hadn’t realised that I had made a trade-off. I’d just forgotten how much better music used to sound. I do worry that we might sleepwalk into a world in which we forget how good purely human-written prose used to be, or music without the touch of machines. Or that it might just become a niche interest for those still willing and able to afford it.

CA – There are people within your book who are more positive about the changes that AI and robotics have brought to their work, whether that’s software engineers who can work faster or miners who feel safer at work because they can operate machinery from a control room. The best outcome, you write, ‘is not necessarily for workers to reject the new tools but to have control over when and how to use them’. Could you elaborate? 

SO’C – This was a pattern I noticed again and again as I went to different workplaces and talked to different workers. What seems to make the difference as to whether the transition is going badly or well is less the technology itself and more this question of: ‘do you feel that these machines are working for you or do you feel that you are working for the machines?’. 

The people who were really unhappy, who felt that they’d lost something and that they were having to work more like machines themselves, were people whose employers had basically tried to get them to plug the gaps in machine-led systems, to be the ‘human-in-the-loop’. That was true of the Amazon workers in these new warehouses where robots are doing some of the job but not all of it, meaning the workers felt like they needed to keep up with the pace that the robots were setting. It was true for the translators who were basically having to tidy up the output of machines.

Those who felt more positive talked about it differently. The miners that you mentioned were overseeing these self-driving trucks and loaders but would say things like, ‘I’m going to send the truck there’, and they had this sense that they were the ones in control. Sometimes that’s partly about the nature of the job and which tasks have been automated and why, but it’s also to do with who had a seat at the table when some of these decisions were made. For the miners, they are in Sweden, a country which has a system called co-determination. When technology enters the workplace, and an employer wants to do something with it that will change the nature of the job, they have to sit down with the workers and negotiate over it first. The miners had sat down and talked through all kinds of implications that might come from this change in the way they were working, such as where the control room might be and what it would mean that everyone in the mine could now be seen at all times, and they came to a series of compromises with the employer.

A lot of employers are messing up here. What lots of companies and board members are saying to their workforces, I think because they’re hearing it from tech companies, is: ‘This change is coming whether you like it or not. We have to adapt. You have to get on board with it. You have to start using these tools.’ But once people feel like they’re being forced to use something, particularly if it’s a tool that’s a bit of a square peg in a round hole and can create more work in some ways, they start to feel resistant, which isn’t what employers want. If there was a different approach in which workers were invited to experiment with these tools and figure out what they’re good for, what parts of the job could be automated and what still needs to be done by humans, then I think people would feel more positive about the future. 

CA – Where do the Hollywood writers that you spoke to fit in? They seem like a group who could have encountered similar challenges to the translators but, for now at least, have negotiated an agreement in which they feel happier about the way AI has been introduced to their jobs? 

SO’C – What’s interesting about the Hollywood writers is that they picked the fight before the damage had started to be done. Because they’re imaginative people by nature, I think they could see where this new technology might go. 

Hollywood is a heavily unionised place. It’s one of the few areas in America where there’s sector-wide collective bargaining, so the Writers Guild of America union negotiates these three- or four- year deals with all of the studios simultaneously. Back in 2022, they were just about to start the next round of negotiations when ChatGPT came out. Very quickly, the writers thought, ‘this could be really bad for us’. Similar to the translators, they didn’t think they would be completely replaced, but they did think something could happen where they’re given a draft script written by AI, particularly if it’s a franchise film or something where you can feed in a load of training data off a studio’s IP and then get AI to come up with some remix with the same characters. And then the writers might be asked to do a polish. They knew that would make them much less money and would be a much less fulfilling job. So they made it part of that big negotiation. 

They ended up going on strike over it – not just over this issue, there was stuff about pay, too – but it was a long strike and actors joined in too so it caused big production holdups in Hollywood. In the end, they came to an agreement with the studios which was quite good for the writers because it basically said: ‘We can use AI but we don’t have to, so we will be the ones that get to decide when and how to use it. And anything that we create, has to be rewarded as such,’ to the extent that, if the studios invite them to use AI to speed up the process, it basically couldn’t be done as a cost-cutting measure. So it incentivises the use of AI only where it would improve quality. One of the things the writers said was, if you do have bargaining power, it makes sense to negotiate early while the technology is still nowhere near as good as you. I should add that a lot of people don’t see that Hollywood strike as having been a good thing overall for the industry. While it’s not just because of the strikes, a lot of production has been moved out of LA to lower-cost locations. But for the writers themselves, the inside story of that fight was quite interesting. 

CA – You scrutinise the use of tsunami metaphors in discussions about AI, pointing out that this is useful language for tech companies hoping to resist regulation because it leaves us feeling powerless. Which of your interviewees most reassured you that we have control over how this latest phase of automation will affect our working lives? 

SO’C – To be honest, I felt a bit reassured by almost everyone. I think we often assume we can look at the labour market and map out who the winners and losers [of automation] are going to be. But, in the process of going out and talking to people, it becomes very clear that’s not really how people respond to change. 

We’re all in control of our own lives, and everyone that I met was doing something. Some were pivoting their skills or business away from areas they thought were under threat, some were changing jobs altogether. Others were leaning in within their workplace and trying to form unions or talk to their employers about different ways of doing things. The market is not static, and no-one is this passive individual that just sits there waiting for a wave to hit them and does nothing. 

The column that I wrote today is about anticipatory labour market decisions, when people see technology coming and make decisions ahead of any disruption. I was writing about the original teamsters who drove groups of horses that pulled wagons, so they were the precursors to motorised trucks. It was clear from about the turn of the 20th century that motorised trucks were coming, though it wasn’t until after the First World War that they became widespread. But, in that period in between, the writing seemed to be on the wall and the profession got older because younger people decided to go elsewhere. We’re seeing this in the kinds of subjects that young people are choosing to do at university now. It’s hard to predict the future and some of our anticipatory decisions might be wrong, but we’re all making choices to try and find our own way through this. That’s what left me more heartened and optimistic than when I went into it. 

CA – In your conclusion, you say there’s a danger that machines will achieve their goal of human-like intelligence, not because of the pace of technological change but because we, as humans, retreat from all that we’re capable of. What do you mean by this?

SO’C – I think we’ve started to lose faith in ourselves a bit because we’re constantly surrounded by predictions that machines can already do so many things better than humans and soon they’ll be able to do everything better than us. We discount how good we actually are, but worse than that, we might become worse at doing things because we stop using those muscles. A number of the translators that I spoke to said to me completely independently that they’ve noticed the language that people use in general with each other, or their written language, is becoming a bit flatter and more machine-like, because we’re absorbing so much machine copy every day that we’re almost starting to imitate it.

Or, in the emotional realm, I can see a plausible future where a lot of people start to prefer the company of AI companions because they’re available 24/7, they have endless patience, they don’t need anything for themselves, they’re always just devoted to you. This could really reduce our tolerance for the messy imperfections of real humans. We might drift away from being able to form deep, complicated human relationships that require give-and-take and negotiation, and we could retreat into a worse, more needy version of ourselves. This is my biggest fear, I think, that as humans we will just retreat, and that’s how the machines catch up with us.

CA – You say that you used to be a techno-optimist. At the same time, you’ve said that writing this book left you feeling more hopeful than when you began it. What is the message you’d most like readers to take away from your book? 

SO’C – I’ve spent a lot of my career writing about bad jobs and people who are treated poorly by their employers. I started off as a techno-optimist in the sense that I thought technology in and of itself might just get rid of a bunch of bad work. But I grew to realise that was hopelessly naive. Technology can be used to improve the quality of work, but not unless a lot of people make a lot of effort to secure that outcome.

When we think of the Industrial Revolution, society as a whole did benefit from it, but that wasn’t automatic. Generations of workers formed trade unions to make sure that workers got a fairer share of the productivity gains, and parliamentarians fought for protections on health and safety and the end of child labour. Society managed to figure out how to shave off technology’s sharpest edges and ensure that everyone had some share of the benefits. I think we’re entering a phase where we’re going to have to do that all over again. 

The message that I really want to get across is please don’t cede the fight before it’s even begun. Don’t imagine that the technology itself, or the people making it, are these powerful forces that can’t be stopped. Technological change has always required society writ large to step in and say: ‘We have these new tools, now we need to figure out how to make the best of them, without costing us things that we don’t want to lose.’ The worst-case scenario would be that all those worst-case things do happen because we all just think that the battle’s already been lost. It’s only just beginning.