How do Humans Differentiate in the AI age
This edition profiles a fascinating conversation about the impact and implications of AI breakthroughs and what it means to be human. It’s an unsettling one…
An update on the Framing book: I have just completed the first draft, and now face a tough period of editing. As always, I have overloaded the book with too many examples, too many words. Cutting it out will be painful but necessary. Wish me luck!
Thanks for reading The Story Rules Newsletter! Subscribe for free to receive new posts and support my work.
Welcome to the one hundred and ninety-second edition of ‘3-to-1 by Story Rules‘.
A newsletter recommending good examples of storytelling across:
- 3 tweets, and
- 1 long-form content piece
Let’s dive in.
𝕏 3 Tweets of the week
Superb visual. Also love how they made the distances relatable (to a US audience).
Paul Graham making a strong statement on the importance of reading and writing.
Simple but important advice.
🎧 1 long-form listen of the week
Plain English: Ethan Mollick on the Paradoxes of AI
Few people are better qualified to talk about AI than these two: Ethan Mollick is the Wharton professor who’s written not one but two books on working with AI, while Derek Thompson is one of the finest observers and storytellers we have of popular trends and culture.
Thompson opens with the framing of AI as an omniscient home tutor… but with poor results:
Thompson: Let’s say you’re in high school. Your parents say, ‘We’ve decided to get you a private tutor.’ And boy, do we have the perfect guy for the job. He knows everything. He has literally memorised every fact and finding in every discipline. He can explain every concept to you with whatever metaphor or analogy you want. And best of all, he’s cheap. He’s going to live in the basement. He never has to eat or sleep. He’s available 24/7.
And the tutor says, ‘There’s something else I can do for you as well. In fact, I’ve done it for most of my past clients. If you want me to just take the biology test for you, I can do it. If you want to skip the geography lesson and have me write the geography essay for you, I can do that too.’ He’s the best teacher ever. And he’s the worst tutor in the world. A tutor so terrible, in fact, that the more you rely on him, the dumber you get.
A team of researchers from California recently studied the effect of AI chatbots on math scores on a popular learning website. On problems that students could copy into their chatbots, performance improved since 2022. But when students were given tests away from their computers, their scores plummeted. In fact, their scores after the invention of AI were worse than they had been before. The tutor who could teach them anything with any metaphor taught them to be dumber.
A brilliant quote from media theorist, Marshall McLuhan—”Every technology is both an extension and an amputation”:
Thompson: The second lesson comes from the media theorist Marshall McLuhan, who smartly said that every technology is both an extension and an amputation. A machine that extends human intelligence is also a machine that cuts off the need for humans to be intelligent. Like many technologies, AI forces us to ask a hard question. When you’re in possession of a tool that can reduce struggle and friction, you have to ask yourself – what is the value of friction? When is the struggle the point, and when is the solution the point?
Humans don’t think very well in exponentials—which is why we have consistently under-forecasted AI’s capabilities:
Thompson: Your book opens with forecasters trying to predict the future of AI and math, and they gave AI a 2.3% chance of winning math Olympiad gold by 2025. Last year super forecasters gave a Millennium Prize problem a 1.7% chance of being solved. Now OpenAI has said it has already solved one.
Mollick: It teaches a lesson that people have heard since Covid – humans don’t think in exponentials very well, no matter how hard we try. Even the world’s best forecasters, and they are really good. It’s a straight line on a logarithmic graph. The ability level has not actually varied very much over the last four years – it’s actually ticked up a little bit. And it’s really hard for us to grasp what that means.
If you worry that big companies are taking a lot of time to adopt AI, Mollick reframes it as saying that ‘big companies are always slow… and in fact with AI, they are moving very fast!’:
Mollick: This is where I’ll argue I’m surprised how fast it is. I’m a professor at a business school, an economic sociologist, and the lesson you learn in entrepreneurship is that entrepreneurs should never win. They should never win, because all the resources, all the money, all the customers are with large companies. Why do they win? Because big companies can’t change.
So the fact that we’re actually seeing high levels of ROI in surveys already – even if $30 million isn’t enough to move the needle for a lot of these large companies, they are seeing positive gains over a very short period of time, and some of them are starting to reconfigure how they work. To me that’s fast, not slow.
AI empowers individuals—but the hard part is still working with other humans to get stuff done:
Mollick: The current phase of AI we’re in is single player. It is about individuals getting performance gains. I can vibe code an entire game overnight. I can write academic papers faster than ever before. But when I start writing systems – multiplayer, many people working together – there are still huge gaps. How does that even work? How do we work together in this?
Mollick uses the O-ring analogy from the Challenger Space Shuttle disaster to talk about the need for human intervention, when AI is bad even at a few small parts of the process:
Mollick: The O-ring is named after the space shuttle Challenger, where 16,000 parts worked great that day in 1986, but the O-ring failed and caused a catastrophe. I talk about the jagged frontier a lot – the idea that AI is good at some stuff you wouldn’t expect and bad at some stuff you wouldn’t expect. It only takes one or two places for AI to be bad at a job for you to still need a human to sub in, because you can’t have everything work great and then one thing fail.
It’s not additive. So the O-ring means jobs are not easily replaced one for another with AI, because if AI can’t do human tasks exactly, you still have value accruing to human work. You see that happening even in coding, where AI now writes code but you still need human engineers to manage it. Maybe that will change in the next couple of months, and then we’ll see another part of the O-ring get stuck.
Mollick reflects on the pace of change; two years back GPT-4 was the best model. Now it is #181:
Mollick: I was trying to figure out how good the best model was when my last book came out, which was two years ago. Both when I started writing it and when I finished it and for several months afterwards, the best model in the world was GPT-4. It was the best model for about a year. And GPT-4 is now the 181st best model in the world right now. There are 19 models that can run on your phone that are better than GPT-4. And that was what everyone was freaking out about. Now the average best model changes every 70 days or less.
If you use AI as a tutor who is pushing you, you will do well:
Mollick: If on the other hand AI is used as a tutor, where it’s not giving you the answers but it’s pushing you, we now have enough studies showing positive effects – meta-analyses, RCTs at Harvard, at Stanford, in Kenya. We know AI can be a very powerful learning tool. So the answer isn’t that hard. We need to use AI as a learning tool outside of class, and inside of class do what we’ve known for the last 30 years of pedagogy works best – active learning experiences, discussions, cases. We have to test people in class without laptops nearby.
The old prompting techniques don’t work anymore:
Mollick: We’ve been studying this at Wharton, and all of the prompting techniques I used to teach people – none of them work anymore. If you learned AI last year, and you’re a recent college grad and you had a formal class on this, none of that works anymore. ‘Think step by step’ is not a useful thing to tell an AI model. Personas no longer change AI performance levels. Almost all the stuff you’ve learned is now useless. So I think we need higher order thinking. AI looks a lot like project management right now – so maybe everyone has to learn a project management class.
Mollick writes his first drafts himself:
Mollick: I write every draft myself. And it’s tempting sometimes, because I’m like, I don’t really want to do another post, another article. But I write the drafts myself, and I absolutely turn to AI afterwards. And the most depressing thing is there’ll be two lines of AI writing in a piece of work and everyone’s like, that was a great line. And I’m like – we wrote everything else. This is the hardest challenge I have, because we will absolutely deskill ourselves every chance we get. But part of the message of the book is: be thoughtful about AI use.
We will need 3 “gyms”—for physical, social, and now mental fitness:
Thompson: Initially, the gym was unnecessary because we had to be hunter-gatherers. We had to walk for 12 hours a day. Then we invented a specialised labour economy that meant we didn’t have to walk that much. And so if we wanted to remain fit, we had to go to this special place and simulate some ancient processes in order to get back that fitness. That’s gym 1.0.
Then when I wrote The Antisocial Century, I was really interested in people who said we’ve thought of friendship and community and neighbourliness as something that naturally comes out of us. But now that the smartphone is here, and you can sit on your couch all day and watch Netflix and go on TikTok, we need to think about hanging out as a means of social fitness – get out of your house and practise being around people.
And now what I’m hearing from you is there’s a third gym that we need – because AI is so good at doing our homework and our work and our jobs for us, we need to remember that sometimes the friction is the point. The struggle is the point. We need to think of aspects of intellectual endeavour as a kind of gym – a special place or time to go to work on skills that might feel hard in the moment, that we know we don’t have to do, but we think might be good for us in the long run..
Mollick talks about human advantages over AI–taste, expertise, agency:
Mollick: AI produces a lot of things. A lot of too many things is slop. The differentiator between slop and not slop is a human deciding something is worthy and tasteworthy, and canonising it or building on it. So that’s valuable also.
Mollick: What is taste? Taste is the idea that I as a human can only work on so many problems, and I need people to care about the problem I’m working on. So I’m almost p-hacking myself by selecting topics that I think will be interesting. The fact that AI can produce an infinite number of papers, most of which are trivially true, starts to challenge the very idea of classic Mertonian science. What do we do in science? Do we look at everything? Do we look at problems that are interesting to us? Does taste matter?
He believes that agency and taste will persist longer as a human advantage:
Mollick: And finally, agency – the ill-defined term. Entrepreneurial action. The idea that you are not waiting for clear instructions or rules, because they’re not forthcoming, and you’re going to try and experiment and do things with AI on your own. I think expertise will probably be the biggest under attack in the short term, because AI can reach expert levels in many fields. But then applying that expertise to other fields is where wide knowledge comes in. And I think agency and taste will persist longer.
Thompson states that (oral) presentation would remain a human advantage … but Mollick isn’t so sure:
Thompson: Let me offer one more – it’s presentation. Ten Ethans could have written Coexistence and then gone on ten podcasts. It matters that one of those Ethans is the best at presenting the information in the book. The text is the same whether it was written by a human or a large language model. But the ability to present ideas is a skill that ought to become relatively more valuable in a world where jobs are emulsifying in the background, where the coders are the managers and the managers are the coders. It’s hard to imagine a world where the ability to present knowledge becomes less valuable – in a world where the underlying knowledge itself is somewhat anonymised, because no one knows exactly where it came from.
Mollick: Says the podcaster and the professor – we’re both heavily vested in the idea that presentation matters. I actually do feel the bitter lesson, the idea that AI trained on enough stuff will eventually beat most fields, nipping at my heels. I made explainer videos at various points, and they were very expensive and hard to make and took a team of very talented people. And I could get a pretty damn good explainer video with a musical theatre number built in out of one-shot prompting. So I think there’s value in knowing how to steer it. At the highest level of expertise, human experts are very good, and AI does hit boundaries in this sort of stuff – especially with that jaggedness built in, where even if it’s a better explainer than humans, it tends to explain things the same way. So that variation doesn’t matter. There is something about presentation that matters. But I almost would consider it expertise and presentation, if you’re very good at that. Meanwhile a whole bunch of people who were bad explainers have gotten much better. The general quality of explanations is going upwards.
That’s all from this week’s edition.
Photo by Diana Polekhina on Unsplash
