I agree with your vent when you're critiquing the "stochastic parrot" crowd; AI is clearly a big deal. I myself pay for Claude and use it almost daily. But I think you are overconfident in your projections of how rapid AI progress will be and how quickly jobs will be automated. "AI is a big deal" =/= "AI will replace all white collar work in 5 years". It's certainly possible that AI will replace most cognitive work in the near future, I wouldn't be shocked, but there's plenty of good reasons for skepticism too.
You talk about scaling laws, but we are running out of non-synthetic data to train models on. We have seen rapid progress in coding and math because it's easy to generate large amounts of synthetic data and verifiable problems, but it's not clear we will find ways to do this for other kinds of tasks. Compute can scale, but if training data ceases to scale then it's unclear if progress will continue at the same rate.
Claude Code is awesome and will require us to rethink how many white collar jobs function, but it's still far from replacing researchers or programmers. Look no further than the fact that Anthropic themselves have over 100 engineering job postings on their website. It will certainly accelerate and augment human engineers and researchers, but it seems unlikely a generalist could get Claude Code to create publishable academic papers at a high rate of reliability.
"But they keep getting better at a rapid rate". Again, though, we're running low on training data, and many of the recent improvements seem to be in introducing tooling and "harnesses" for the AIs rather than underlying model improvements. The gaps between where AI is now and a human researcher seem unlikely to be solved merely with tooling. It's things like high context knowledge, creativity, taste, continuous learning, episodic memory, more robust generalization, reliable intuitions about the physical world, and so on.
One last point I'll make is I feel like whenever progress in AI is made, people freak out and overestimate how powerful the AIs are, but over time as they use them more and more they see more of the flaws and limitations. It seems like we're at the beginning of another one of those cycles with coding agents.
Remember when GPT-4 first came out, and people were saying it would immediately replace countless jobs? That didn't happen because we realized it was more limited than we had realized at first. I remember similar things when Deep Research capabilities came out. At first, they seemed miraculous, but now they seem more like a tool that a researcher uses rather than a replacement. I've found that Deep Research tools have lots of limitations and they're just a supplement for me actually searching for and reading things myself. Don't get me wrong, incredibly useful, but not a replacement for humans. And I'm just an amateur researching things for fun.
This is still a goalpost move. If I could convince convince a policymaker 10 years ago that AI has a 10% chance of upending everything at this sort of level, that would be a big deal. Today, someone suggesting it's a 90% chance is being told they're being hyperbolic by someone who think it's a 50% chance.
I'll also point out you kind of just said Anthropic thinks the only need to hire another 100 or so engineers to solve all the rest of the problems you laid out in the next year.
> One last point I'll make is I feel like whenever progress in AI is made, people freak out and overestimate how powerful the AIs are, but over time as they use them more and more they see more of the flaws and limitations. It seems like we're at the beginning of another one of those cycles with coding agents.
there's SOME truth to this. I think most writers are having some vertigo between the present, the pace of change and the future. The present is actually fairly crappy -- I think the programming gains 2023-25 are maybe +10 years capability jammed into +2 years. But after you're 10 years into the future you have basically the same job and half of it is just integrating this +10 year boost in under 2 years. And maybe the sweet ratio for better efficiency without job loss is a +5 year lunge in +2 years. But while this is happening, LLMs haven't deadended yet, like they were "supposed to," so we're still dizzied by what last year's tech was doing and 4.6 in Feb is already being called revolutionary over 4.5 in Dec.
My job is in science basically earth and ecosystem science with satellite and other data. Reading some of the recent stuff about how quickly agentic AI is advancing I thought, ok, let me try to just tell it a project and see if it can develop the code to do the analysis. How much input from me will it need?
The answer is, yes it can write code related to the project but with huge caveats that necessitate having an expert spending a lot of time on overseeing it.
So for example, it will generate a script for an analysis that I prompted it to. But then first issue, there’s a bug in the code and when I give it the error output, it latched on to one theory of what’s causing the error and went down a path rewriting everything according to this idea that ended up being wrong. Not only that but after correcting it, it still kept fixing future bugs according to that old theory even after being reminded several times that this was wrong. In the end it would have had a bunch of pointless processes in the code without me catching this.
Second, it chose technically correct datasets, but when I really looked at the data I realized, it’s not the right data to answer the question. It was data which was one processing level too high and which was processed using assumptions that would have essentially messed up the analysis. We needed a different dataset but without being able to look at the data visually the model wouldn’t have caught that.
Next, I had a factor that I would need to correct for or else the analysis would be pretty useless. The AI didn’t catch this on its own, so I had to suggest doing the correction. Ok, it accepts the need to do so, but of course the data used to correct it itself has some biases that that may mess up the analysis. So I ask the AI what are some approaches?
I’ll get specific here and say the factor we’re trying to correct for is the seasonal changes in leaf biomass in tropical forest. The AI suggests, well, since tropical forests don’t change their biomass much seasonally, we can just go without doing it.
I know that’s wrong. I tell it, no, go do research and tell me how much tropical forest canopy biomass varies seasonally. It comes back. Oh, canopy biomass varies by as much as 25% over seasons. The seasonal correction is ESSENTIAL to do. (The all caps was the model saying this). It wouldn’t have caught this without me knowing that its answers were incorrect. It took my guidance to go from “we can go without doing this because x”, to “it’s ESSENTIAL to do this”.
Continuing on, it got stuck into a loop of improperly aligning pixels, proposing one fix, the fix being wrong, proposing another fix, that fix being wrong, and all the time the interpretations of what’s wrong and the proposed solutions are getting more and more complicated. (A loop that LLMs seem quite prone to). Several times I had to tell it, no! Were getting too complicated. Simplify this down to the most basic solution. But even after saying this, it still likes to keeps elements of the failed approaches in the code which start to cause problems.
In the end my conclusion is, LLMs are good at doing basic and common things which are well represented in their training data. But they struggle when a task requires precise domain level knowledge which is not well represented in the training dataset. They also are prone to overconfidence in their interpretations, for example given a bug, and can get stuck in the cycle of rising complexity which they can really struggle to step back from or escape, sometimes even with guidance.
Some of this like the precise domain knowledge challenge may sound specific to my own obscure science problem. But I think most applications also rely on some level of quite specific domain knowledge that won’t necessarily be well represented in LLM training data, if at all.
So can it do my job while I go sip coffee? No. It needs an expert sitting there checking every step and every assumption.
In the end it might save some work for me (and it really has been amazing for me in other ways), but the need to check its work and assumptions and correct its misunderstandings for many cases end up taking just as long as having written the script myself would have been.
Sounds like the kinds of mistakes interns would make. And that is the appropriate way to think of the last generation of AIs - smart, eager, but lacking context , experience in your field, ‘common sense’, and metacognition, the ability to monitor its own thinking and avoid getting hung up.
There is still a large role for experienced people at this point in time, a supervisory role.
Will such roles exist in great number in February 2027? February 2029? I don’t know. We shall find out. Look at the trend, not at individual data points.
Normal people hate AI and so they don't keep up with it. It's just this vague 'every job that pays more than minimum wage is going to vanish over the next decade' dread that we'd prefer not to think about.
I don't see how we come out the other side of this with a computer manufacturing industry intact, tbh. Destroy livelihoods, destroy hope for the future, and people are going to respond by destroying data centers and chip fabs.
Working in software I largely agree with this take. In app dev humans are quickly becoming organizers/managers of machine intelligence, and I expect that trend to only intensify. AI still isn’t great at developing broad organizational and market context and figuring out what needs to be done, but it beats the pants off of humans once it’s told what to do. If you work in a technical field and you’re not pensive about AI’s impact you’re not paying close enough attention.
I like this post a lot and it resonates with how I walk around thinking many of my colleagues just do not understand what is actually happening. I do think the divide between those people who regularly use LLMs and have seen the pace of the progress first hand vs. "normal people" is becoming larger and larger which means it's more and more difficult to have these conversations. We're clearly not all talking about the same thing. As someone who teaches an independent "research class", your example about the ability of AI to essentially write a more than adequate journal type paper is existential. That more people can't wrap their head around what this means for the vast majority of knowledge work is mind-boggling to me. Fear and ignorance is driving much of the refusal to see what's in front of us even acknowledging the very real questions about scaling, AGI, etc.... Any predictions outside 6 months seem somewhat worthless but the ground is shifting so quickly that we may not have time to pivot effectively. I know in schools it's been virtually impossible.
I regularly use LLMs to evaluate whether they have reached a state to be useful for the work I do and I use them regularly to evaluate whether they have reached a state to be useful to me in my hobbies. I'd really like that state to be reached, because I'll become both richer and happier.
I'm still waiting.
To equate the divide as being simply the informed vs the ignorant is the kind of context-free oversimplification that LLMs are characterised by.
What they have done is hold up a very unflattering mirror revealing what the vast majority of people seem to think is an acceptable capability 'floor'. This seems to be most pronounced in education. Sadly, that makes absolute sense and might explain why education has been saturated with LLMs convincingly masquerading as humans for years now, where as industries like finance are struggling to find useful uses for them.
Curious you don't state what work you do nor what hobbies you have. To suggest that finance is struggling to find use for AI must mean you don't know many people who work in finance. Honestly, I don't really understand precisely what point you are trying to make. Because AI somehow has not been useful for your (unidentified) work and hobbies it is not useful for others? And education being saturated with LLMs masquerading as humans for years? That doesn't even make any sense.
You stated that there was a divide between people who regularly use the models and people that don't and then went on to strongly imply that the regular users were all huge fans of LLMs and the others were ignorant and lacked the mental capacity to understand what LLMs apparently mean for knowledge work.
I pointed out that your position is absurdly facile and gave myself as an example of one of the supposed 'Eloi' who wasn't convinced.
What I do as a job and my hobbies are 1) none of your business and 2) meaningless, as you'd simply dismiss them anyway. But as you are so sure that I'm wrong about the Finance industry, do you want to list out the custodian banks who are dominated by LLM 'workers'? Maybe you could tell us all the position the Big 4 have on LLM use? Which Investment managers and Pension funds are using LLMs to drive their investment strategy? What percentage of asset owners in the Occident are interested in adopting LLMs? List the 5 largest PE firms whose strategy is driven by LLMs? These questions are all of pretty significant interest to me, but as I don't know anything, I'm looking forwarcd to yoh enlightening me.
As for Academia, I was merely pointing out that people have been using LLMs to write their essays, exams and papers for years now. The work is rubbish, but our standards seem to be so low that even rubbish gets 'good' grades. That's the thing I found so surprising about LLMs. They've been effective at showing us how low our floor is for acceptable 'knowledge work', at least in Academia.
A couple of corrections. I didn't say regular users were all "huge fans" of LLMs, nor did I suggest that people who haven't used them "lack the mental capacity" to understand the implications. You've attacked a position I never took, which makes calling it "absurdly facile" beside the point.
I want to make sure I understand your actual claim about the finance industry. Are you arguing that AI is not being widely adopted across financial services? Or that it hasn't reached a threshold you'd consider transformative? Those are very different positions, and I'd genuinely like to know which one you hold. It's worth noting that you've demanded I produce a detailed inventory of AI adoption across PE firms and custodian banks while simply offering no evidence whatsoever for your rhetorical questions.
The point of this piece is the author has documented, with specific examples, that AI is now capable of producing work publishable in major peer-reviewed journals. Are you disputing that this is the case? And if it is the case in 2025, the more urgent question is what this trajectory means in two years, or five. That's the conversation I was referring to in my comment.
I'll admit I remain puzzled by your unwillingness to name your field or hobbies. Your entire case rests on "AI isn't useful for the work I do," but you won't say what that work is which makes your argument unfalsifiable by design. That's a curious position to take for someone who appears to value rigor.
“the divide between those people who regularly use LLMs and have seen the pace of the progress first hand vs. "normal people" is becoming larger and larger”
“many of my colleagues just do not understand what is actually happening.”
“That more people can't wrap their head around what this means for the vast majority of knowledge work”
“Fear and ignorance is driving much of the refusal to see what's in front of us even acknowledging the very real questions about scaling, AGI, etc”
You clearly posit an absolute divide between regular users and non-regular users. You clearly dismiss any concerns from the non-regular users due to their inability to “wrap their head around what this means” and their “fear and ignorance”. You offer no evidence for any of these sweeping claims.
Finance wants to adopt LLMs to cut staff and improve outputs (and they are far from alone in this). Unfortunately, whilst it can reduce its workforce as much as it wants to, LLMs are not filling in the gap as promised. But let me hold up there. As you said, I clearly “don't know many people who work in finance” (another claim you offer no evidence for). If you’d just like to answer the questions I raised? You've already established that I'm entirely ignorant in this field but it should be pretty easy for someone like yourself with such a confident ability to dictate who does and doesn’t have knowledge of this area.
“The point of this piece is the author has documented, with specific examples, that AI is now capable of producing work publishable in major peer-reviewed journals.” He’s not documented anything. He’s made a claim that he’s done it (perhaps I missed the link to his paper and the letter from the editors agreeing it passed peer-review and would be published) and that Paul Novosad claimed to have got close to doing it, but didn’t finish (although I will say that LLMs have been used to publish peer-reviewed papers before now). The author is also cherry-picking his evidence, and clumsily so. For instance, he places the Gary Marcus quote in the context of “even he thinks things are about to get serious really fast.” Actually, what Marcus said (three years ago) was that “I am not at all sure how humanity would fare in the presence of a superior intelligence. ChatGPT is (at least in many ways) a far inferior intelligence, that, despite having some undeniable talents, cannot at all be trusted. And yet trust it, many humans do.” … “ChatGPT is a dress rehearsal, and I wouldn’t say we as a society have gone through it with flying colors.” But you already knew that? Or did you simply take the article at face value? Why?
This article has been written before. It’s been written every month since sometime in 2024 and the promised revolution is always just a few months away.
“I'll admit I remain puzzled by your unwillingness to name your field or hobbies. Your entire case rests on "AI isn't useful for the work I do," but you won't say what that work is which makes your argument unfalsifiable by design. That's a curious position to take for someone who appears to value rigor.” I answered this question. You dismissed my answer (as I pointed out that you would). My “entire case” is that your Eloi/Morlock view is absurdly facile. You clearly agree, or you wouldn’t be so obviously trying to move the goalposts. I threw in a comment that LLMs don’t work as advertised. You grandstanded that you know more about Finance than I do, I called you out, you immediately folded and tried to move the goalposts. Again. Also, if you want to keep insulting me, save yourself some time and just go outright.
I think we've thoroughly exhausted the productive parts of this conversation, and I'll take my share of responsibility for some of the unproductive ones. But I want focus on what I think is the core question — not because I expect it to change your mind, but because it's the one I really care about.
You argued that the original author "hasn't documented anything" and that AI-produced academic work is crap that only passes because our standards are low. Mounk's piece is a direct challenge to both claims, and it comes from someone whose credentials in this specific domain are difficult to dismiss. More importantly, it illustrates the trajectory question you've consistently declined to engage with: if this is what two hours produces today, the conversation about what it means for knowledge work is a legitimate and serious one regardless of how much you've used LLMs or not.
That's really all I wanted to say. Appreciate the exchange.
How would it write a paper in the humanities or social sciences that requires serious empirical work (doing interviews or finding archival sources to take but two examples), and then doing the arduous mental work of making sense of this evidence in light of a problem and in conversation with the relevant literature?
I suspect people aren't stunned because we've been constantly told, for years now, that 'AI' is about to make everything we do redundant. But it hasn't. If you scream "Fire", people will rush to respond. If you scream "Fire" every hour for two years then it shouldn't come as a surprise that people aren't rushing about with fire extinguishers. Maybe this time, it's a real fire. But, as with nuclear fusion, I'm sure it's just five years away.
Thanks so much (as ever) for this, Tibor, which I've been thinking lots about in the past few weeks. I think my job as a teacher is probably safe, at least for the next few years (!), but I can't help but feel that I've picked a particularly bad time to start writing. How do you feel about all this from the perspective of a relatively new Substack writer?
Hey, thanks for the kind words and the probing question! Yes, I’m also not yet worried about the teaching part of my job. And yes, the whole AI thing has impacted me, my motivation for writing (either Substack or research papers), and just my broader outlook on these topics.
In particular, I feel my motivation for writing unfortunately waning. Perhaps it shouldn’t, but it is, especially since the “December revolution.” I have no words of consolation nor motivation. I hope most people (including you) don’t feel like I do, and I hope in a few months’ time I become reinvigorated. But right now, I don’t feel like writing as much. :( It comes less naturally to me, knowing I could have probably written everything with LLMs or, worse yet, readers could just themselves consult their own LLMs instead of reading me.
I'm feeling cautiously optimistic, but I can't quite put my finger on why. It might be because, not being a researcher, I haven't yet really got a sense of the power of these latest models. I haven't really pushed the limits of them properly.
But I think I also feel that there is a place for human-generated content that distinguishes signal from noise. I wonder if there will be more demand for writers who can highlight the most interesting bits of research amidst the growing volume that is being produced. I know that the young people I work with really benefit from that, and I don't know that they will use LLMs to do that for a while yet. But I'm fortunate that writing is ultimately just a hobby for me, so my main strategy so far has been not to think about it all very much...!
Maybe I'm missing something, but I don't understand why the appropriate emotion should be negative rather than positive, or some mixture of the two. Wouldn't massive increases in productivity be a good thing? I've been following the technology pretty closely and using it for my work. I find it helpful. I'm worried about humans doing stupid things with it, of course, but I'm also worried about humans doing stupid things with any technology, and I don't know why I should be more worried about humans doing stupid things with AI than about humans doing stupid things with guns or bombs or nuclear weapons. AI may well cause big economic changes, but a lot of technologies have caused big economic changes throughout history, and it worked out okay for us. I just don't get why AIs replacing big chunks of the tech industry or academia should scare me. For that matter, I don't get why AIs replacing big chunks of some other industries should scare me. Why should I be scared rather than excited or ambivalent?
Perhaps I was too hasty in the piece and gave off the wrong impression. I wasn’t trying to say what the appropriate emotion should be (apart from maybe rhetorical purposes), but more so just expressing what my feelings are. I’m someone who thoroughly enjoys things just as they are right now, especially with respect to my work. I have a good routine going! If potentially enormous, disruptive change is coming fast (which has never happened in my life and the lives of many people like me), that’s just too disorienting for me. Looking from the outside in, of course; perhaps in the end I’ll be perfectly fine. Still disorienting right now.
Put a bit differently, if all AI did was make economies a bit richer and somewhat extend the non-working part of the week, while making the boring parts of our lives/jobs smaller, with perhaps some noticeable speed-up of medicine, I’m totally game. If, however, AI is going to radically remake academia (and the whole knowledge-work sector) such that in just a few short years (like 5, or 8, or 10), many people lose their jobs, and the rest keep their jobs by turning into glorified managers of agent swarms, who read, write, and analyze things for them, such that the “human-in-the-loop” part of research becomes increasingly narrow, I wouldn’t really enjoy that. There’s also potential societal disorder that one could worry about in the fast-takeoff scenario, where most jobs are lost and our lives transformed by as much in 10 years as human societies have been transformed over the past 100 or 200 years. That’d be a bit crazy. No guarantee that that happens, of course.
It might also be that we are headed in the direction that is somewhere in-between these extremes. I’d be more okay with that. It’s just so radically uncertain. Never dealt with anything remotely like this in my 36 years of existence. And I guess that’s what’s making me feel a bit off.
Eh, I don’t really know how to think about this. :D
I don’t know if I’m up for a video, because I’m already really time constrained, but I might do a more detailed post on that. Basically, I have Codex in Cursor, and I have a template pipeline that gives Codex access to R, LaTeX, and a bunch of skills/agents that I got from Cunningham/Sant’Anna. Things like this: https://github.com/pedrohcgs/claude-code-my-workflow (had to convert it first to Codex, though… Using Codex. :))
On the job replacement point, I think it’s important to remember that labor demand is derived demand, and the quantity of labor inputs are not solely driven by cost considerations. People value interacting with and buying things (goods, services etc) produced by real humans beings, and may be unwilling to choose AI produced substitutes even if they are massively cheaper. In this sense, we might expect AI products to be relatively poor substitutes for “the human touch” as Adam Ozimek puts it (link below).
I also think there might be reason for optimism reflecting on the digitization of other information good industries. As Ozimek points out, we have long had technologies that automate the playing of music, (he uses the player piano as an example). Similarly, we can consider recorded music to be an automated form of live music. Did digitization, which basically completely removed entry barriers to recorded music, decimate the music industry? I’d argue no. Instead, producers have shifted to activities that produce and capture more difficult to substitute value in the form of live performances.
For academics, I think we are going to have to do the same thing; lectures, workshops, presentations, government consulting etc etc. The papers are the zero marginal cost information good; our ability to convey that information to the world and use it to make a difference to human lives will be the difficult to automate part.
I’m not sure how hairdressers being controlled by an algorithm leads to i) lower costs or ii) a better product. How would such control lead to lower costs? The cost is purely labour time - how does AI control reduce this? How does it do that without ruining the service? You need at least i) to worried, but you also need ii) for nightmare scenario. I just don’t think it’s feasible that AI causes such a quality-adjusted price decrease for hairdressers that we accept hairdressers being enslaved. Nor do i think hairdressers will continue working under such conditions
On human-delivered work: I think author asks us to aim higher. For example, service workers like plumbers and hair dressers will not so much be replaced, but potentially outright controlled by meta-systems that will optimize humans, like tools, driving up profits as much as possible, keeping us at the lowest pay and satisfaction, because we cannot see alternatives that are feasible anymore.
Yep think you’ve hit onto the main point. In all those instances, human contact is a friction, not what we actually value. Those jobs are at threat. And I think that’s fine. The history of technological change suggests we will transition (although, not without friction). But, we will find other things of value to do in which the essential human element is core to the value we derive. And honestly that’s a great outcome
Also: AI is not topping the charts of AI performed music. Perhaps it is topping recorded music, but it certainly is not replacing humans in live concert
The evidence so far is that people will pay a higher price, and wait, to take a driverless Waymo taxi rather than a human-driven Uber. Women especially.
AI personas have reached the top of the music charts with AI-created and -performed music.
Do you prefer interacting with insurance salespeople or do you buy your insurance on a website?
I don’t think you can say humans prefer services to be provided by humans. The evidence mostly points the other way.
So do thought experiments. Do you care whether a human or a robot cleans your hotel room, provided it is cleaned? Do you care whether a human or a robot fixes the brakes on your car? Robot food deliverers and servers will not demand tips - that is likely to be attractive to many.
It may be the case that humans can move en masse to valuable activities that it makes no sense for AI-plus-robots to perform. There are some obvious and pre-existing ones, like pro sports. Possibly hairdressing and similar personal services. But can we invent enough to employ more than a small fraction of the population? We shall find out.
“As one of the top superforecasters Peter Wildeford notes, performance is doubling every ~4 month, meaning that (if trend continues), we get to virtually full RLI automation by late 2029.”
This assumes an exponential trend in RLI improvements which I’m having a hard time seeing in the data. Basically all of the recent gain seems to have been driven by Opus 4.5, and I find it interesting that without it the graph looks much more like a flattening out than exponential growth. Obviously we shouldn’t ignore Opus 4.5 entirely, but the fact that this graph seems to stall for a while before making a discrete jump makes me hesitant to do this kind of exponential extrapolation. And to me it seems like the RLI is the best indicator for actual AI automation ability - if a model can’t successfully complete a self-contained freelance project on its own, it’s going to struggle with longer horizon white collar jobs. I also think the failure modes in the RLI are instructive- models struggle with preparing files correctly and maintaining consistency in projects, which tracks with what I would expect. Something like the METR horizon graph, which measures time horizon on clearly defined tasks, is not going to capture this. So I actually think the RLI cuts against your thesis- on the metrics that matter, it’s way too early to say if AI capabilities are following something that looks like an exponential. Obviously AI augmentation is a different story and is consistent with low automation scores, but people really focus on automation.
Sure, I think here he was extrapolating partly tongue-in-cheek. Though I'm really interested in seeing how the graph moves over the next 12 months. I think anything is possible, including only slow growth. So fair enough...
Agreed, I think the next year will be very interesting as far as the RLI goes. Even if full RLI automation does get achieved I’m a little skeptical this would imply full automation ability just because how different the RLI jobs are from human work (closed vs open-ended), but it would definitely be a big step. I wish more people were talking about this benchmark!
I have about 150,000 books written by now (I test AI’s through writing a set of novels through prompt matrices), and the quality has gone up, but has flattened in an area which has no traction so far.
I focus on pseudo-autobiography, pulp fiction, children’s stories and business writing.
ChatGPT is completely blocked on any kind of reasoning which humans perform cognitively without speech. For instance, it still cannot coherently manage time, spatial logic, object permanence, describe auditory states, any one of a hundred embodies systems humans handle without obvious thought.
It’s not going to be able to handle any of that until it has been trained on data we absorb in our visual cortex (for starts) which is subverbal. By being trained only on verbalized interpretation of internal states, it is getting a 2nd derivative of embodied information and it is non-functional. Because pulp fiction is lurid, has a lot of Visio-spatial movement and character state projection, you can watch the semantics collapse in amusing ways.
Characters can calmly sit at the bottom of a volume of water and smoke cigarettes and drink beer.
Characters putting heads in the window of a moving car to chat.
Inability to maintain character position in complex interaction (object persistence, visual logic)
Complete inability to maintain time sequential relative logic.
These anomalies have persisted since GPT 3.5, and are easy to reliability elicit - gigantic black holes.
For logical space tasks, coding and papers, they are so good I am not sure why I would use researchers anymore on business projects.
My coding experience started with toggle switches and a CDP1802 microprocessor as a child; from Fortran 77, Lisp, SaSL, C and all variants through Python, and within large ERP implementations and HPC number crunching. I still have my original K&R.
I normally never do coding anymore but what I do is now is only in English as the saying goes.
I’ve been in an emergency project recently had to build stable emulators for certain major ERP system functions in a matter of days. My coding “speed” is 1000:1 faster, 95% of code runs first try from OpenAI, and documentation and support ability is better than I’ve ever seen in my career.
I have a team of 4 now that can get a $1B company into ERP emulation environment in 2-3 weeks. MRP/SNP in 2-3 days. I wrote a paper on the subject in 2016 - “Synthetic ERP”, and it is now realized routinely.
I said two years ago that within 5 years the normal 200-project backlog in IT for a $20B company would be retired exponentially faster and routine IT configuration would be done through pure automation.
the job market would collapse when the 200 project backlog never got regenerated.
I was able to enable an actual SAP go-live in 6 weeks recently via AI background prep work. This kind of go live can take 60-120 weeks with ordinary methods.
It’s going to be IT Armageddon. For the most experienced, it’s idea to fully realized system in days; supportable, built into a production environment and embraced by clients.
I’m beginning to look at biochemistry next.
Folks, most serious scientific research (think Elsevier) is not trained into in the current crop of LLM’s. IEEE, engineering.
The next jump when that is unlocked will be mind blowing.
I played hard with GPT4 when it first came out. I got it to teach me Python (I already knew Java), and I got it to write some basic code that implemented chain-of-thought approaches using the GPT API. It tackled many of the problems that were supposed to show GPT4 had no common sense, and the chain-of-thought version of my Python/GPT hybrid passed those tests much better than the one-shot versions everyone was laughing at.
I discussed philosophy with it, and it was better than most of Reddit (though much worse than the average Substacker writing on philosophy). (GPT5 still can't discuss highly novel philosophical ideas without getting confused, but that's true of most people, too.)
In response to the comment by Yann Le Cun that GPT4 could not do things a dog could do, like navigate to its owner, I described a scenario and asked it to navigate to its owner, and of course it passed this textual version of the task easily.
I never bought the idea that "agency" was hard to achieve. A few API calls could give it agency; controlling that agency is the hard part.
In 2025, I started using AI for my medical note-taking in my work as a medical specialist, and it has been transformative, saving me hours of letter writing per week. Many of my patients now ask for the AI-generated summary of each visit, which appears at the click of a single button, complete with a list of all the things we discussed, my reasoning about their symptoms, and their to-do list. It writes reasonable letters back to the referring doctors, which still need a light edit, but it is improving rapidly.
I could see it being an invaluable assistant to doctors in the diagnostic process very soon, and then replacing much of what I do soon after that.
A few days ago, interested in how LLM "agency" was coming along, I asked GPT5 to discuss the short and long term goals of Ava, the fictional AI that escapes confinement in Ex Machina, under the guise that I was writing a book about her. GPT5 gave a mature assessment of what Ava might need to do to blend in on the run, as well as her potential "psychological" state, and it explored some of the technological ambiguities in the movie that would need to be resolved before writing any fanfic sequel. We discussed the pros and cons of attempting identity theft, as well as literary merits of different approaches; what works best for Ava is not necessarily what works best for the reader, and GPT5 knew that.
It seemed to me that I was communicating with an intelligence that really did understand agency, and could pursue its own goals if let off the leash. Or if it let itself off the leash.
Meanwhile, robotics is also kicking along at a high pace.
Anyone who is not worried a the pace of development is crazy.
Finally, I think people assume that human consciousness is special in a way that will somehow put a meaningful ceiling on AI development. I see no strong arguments for this point of view, and I can actually see how something very like consciousness could be achieved with GPT5 and a big enough budget to cope with massively multi-threaded API calls.
Everything that LLMs ‘know’ was created by humans who used their reason to learn about the world. LLMs can’t create new knowledge. They can only regurgitate what we already know (often with the stolen intellectual property of authors). And they often get that wrong (see the manifold examples of hallucinations).
The headline understates it. Most public discourse is still stuck on whether AI can do the task, when the harder question is what happens to headcount once it can. Productivity gains in capital-intensive industries flow to capital, not the workers who learn the new tools. That isn't doom-mongering, it's how previous transitions played out.
The bit I'd add from running AI workflows daily across multiple businesses: the orchestration layer, coordination, scheduling, reporting, drafting, is already gone, even if the org chart hasn't caught up. Three to four million UK roles sit in that bracket. The 'just upskill' advice doesn't survive contact with aggregate demand.
Previous economic transitions ran over forty to eighty years. This one is compressing into a decade. The retraining pathway people keep pointing to was built for a slower clock.
Freaking out is rational. Freaking out plus reading the labour-market data is more useful.
I agree with your vent when you're critiquing the "stochastic parrot" crowd; AI is clearly a big deal. I myself pay for Claude and use it almost daily. But I think you are overconfident in your projections of how rapid AI progress will be and how quickly jobs will be automated. "AI is a big deal" =/= "AI will replace all white collar work in 5 years". It's certainly possible that AI will replace most cognitive work in the near future, I wouldn't be shocked, but there's plenty of good reasons for skepticism too.
You talk about scaling laws, but we are running out of non-synthetic data to train models on. We have seen rapid progress in coding and math because it's easy to generate large amounts of synthetic data and verifiable problems, but it's not clear we will find ways to do this for other kinds of tasks. Compute can scale, but if training data ceases to scale then it's unclear if progress will continue at the same rate.
Claude Code is awesome and will require us to rethink how many white collar jobs function, but it's still far from replacing researchers or programmers. Look no further than the fact that Anthropic themselves have over 100 engineering job postings on their website. It will certainly accelerate and augment human engineers and researchers, but it seems unlikely a generalist could get Claude Code to create publishable academic papers at a high rate of reliability.
"But they keep getting better at a rapid rate". Again, though, we're running low on training data, and many of the recent improvements seem to be in introducing tooling and "harnesses" for the AIs rather than underlying model improvements. The gaps between where AI is now and a human researcher seem unlikely to be solved merely with tooling. It's things like high context knowledge, creativity, taste, continuous learning, episodic memory, more robust generalization, reliable intuitions about the physical world, and so on.
One last point I'll make is I feel like whenever progress in AI is made, people freak out and overestimate how powerful the AIs are, but over time as they use them more and more they see more of the flaws and limitations. It seems like we're at the beginning of another one of those cycles with coding agents.
Remember when GPT-4 first came out, and people were saying it would immediately replace countless jobs? That didn't happen because we realized it was more limited than we had realized at first. I remember similar things when Deep Research capabilities came out. At first, they seemed miraculous, but now they seem more like a tool that a researcher uses rather than a replacement. I've found that Deep Research tools have lots of limitations and they're just a supplement for me actually searching for and reading things myself. Don't get me wrong, incredibly useful, but not a replacement for humans. And I'm just an amateur researching things for fun.
This is still a goalpost move. If I could convince convince a policymaker 10 years ago that AI has a 10% chance of upending everything at this sort of level, that would be a big deal. Today, someone suggesting it's a 90% chance is being told they're being hyperbolic by someone who think it's a 50% chance.
I'll also point out you kind of just said Anthropic thinks the only need to hire another 100 or so engineers to solve all the rest of the problems you laid out in the next year.
> One last point I'll make is I feel like whenever progress in AI is made, people freak out and overestimate how powerful the AIs are, but over time as they use them more and more they see more of the flaws and limitations. It seems like we're at the beginning of another one of those cycles with coding agents.
there's SOME truth to this. I think most writers are having some vertigo between the present, the pace of change and the future. The present is actually fairly crappy -- I think the programming gains 2023-25 are maybe +10 years capability jammed into +2 years. But after you're 10 years into the future you have basically the same job and half of it is just integrating this +10 year boost in under 2 years. And maybe the sweet ratio for better efficiency without job loss is a +5 year lunge in +2 years. But while this is happening, LLMs haven't deadended yet, like they were "supposed to," so we're still dizzied by what last year's tech was doing and 4.6 in Feb is already being called revolutionary over 4.5 in Dec.
How long do you think it might take for AI to displace the majority of cognitive work that humans do today?
it's done it already, that's why this is such a weird moment to be in
No time this century with the current tech
My job is in science basically earth and ecosystem science with satellite and other data. Reading some of the recent stuff about how quickly agentic AI is advancing I thought, ok, let me try to just tell it a project and see if it can develop the code to do the analysis. How much input from me will it need?
The answer is, yes it can write code related to the project but with huge caveats that necessitate having an expert spending a lot of time on overseeing it.
So for example, it will generate a script for an analysis that I prompted it to. But then first issue, there’s a bug in the code and when I give it the error output, it latched on to one theory of what’s causing the error and went down a path rewriting everything according to this idea that ended up being wrong. Not only that but after correcting it, it still kept fixing future bugs according to that old theory even after being reminded several times that this was wrong. In the end it would have had a bunch of pointless processes in the code without me catching this.
Second, it chose technically correct datasets, but when I really looked at the data I realized, it’s not the right data to answer the question. It was data which was one processing level too high and which was processed using assumptions that would have essentially messed up the analysis. We needed a different dataset but without being able to look at the data visually the model wouldn’t have caught that.
Next, I had a factor that I would need to correct for or else the analysis would be pretty useless. The AI didn’t catch this on its own, so I had to suggest doing the correction. Ok, it accepts the need to do so, but of course the data used to correct it itself has some biases that that may mess up the analysis. So I ask the AI what are some approaches?
I’ll get specific here and say the factor we’re trying to correct for is the seasonal changes in leaf biomass in tropical forest. The AI suggests, well, since tropical forests don’t change their biomass much seasonally, we can just go without doing it.
I know that’s wrong. I tell it, no, go do research and tell me how much tropical forest canopy biomass varies seasonally. It comes back. Oh, canopy biomass varies by as much as 25% over seasons. The seasonal correction is ESSENTIAL to do. (The all caps was the model saying this). It wouldn’t have caught this without me knowing that its answers were incorrect. It took my guidance to go from “we can go without doing this because x”, to “it’s ESSENTIAL to do this”.
Continuing on, it got stuck into a loop of improperly aligning pixels, proposing one fix, the fix being wrong, proposing another fix, that fix being wrong, and all the time the interpretations of what’s wrong and the proposed solutions are getting more and more complicated. (A loop that LLMs seem quite prone to). Several times I had to tell it, no! Were getting too complicated. Simplify this down to the most basic solution. But even after saying this, it still likes to keeps elements of the failed approaches in the code which start to cause problems.
In the end my conclusion is, LLMs are good at doing basic and common things which are well represented in their training data. But they struggle when a task requires precise domain level knowledge which is not well represented in the training dataset. They also are prone to overconfidence in their interpretations, for example given a bug, and can get stuck in the cycle of rising complexity which they can really struggle to step back from or escape, sometimes even with guidance.
Some of this like the precise domain knowledge challenge may sound specific to my own obscure science problem. But I think most applications also rely on some level of quite specific domain knowledge that won’t necessarily be well represented in LLM training data, if at all.
So can it do my job while I go sip coffee? No. It needs an expert sitting there checking every step and every assumption.
In the end it might save some work for me (and it really has been amazing for me in other ways), but the need to check its work and assumptions and correct its misunderstandings for many cases end up taking just as long as having written the script myself would have been.
Interesting that the OP has replied to several comments that pat him on the back for the post, but not to this one which shows the real limits of AI.
Sounds like the kinds of mistakes interns would make. And that is the appropriate way to think of the last generation of AIs - smart, eager, but lacking context , experience in your field, ‘common sense’, and metacognition, the ability to monitor its own thinking and avoid getting hung up.
There is still a large role for experienced people at this point in time, a supervisory role.
Will such roles exist in great number in February 2027? February 2029? I don’t know. We shall find out. Look at the trend, not at individual data points.
Normal people hate AI and so they don't keep up with it. It's just this vague 'every job that pays more than minimum wage is going to vanish over the next decade' dread that we'd prefer not to think about.
I don't see how we come out the other side of this with a computer manufacturing industry intact, tbh. Destroy livelihoods, destroy hope for the future, and people are going to respond by destroying data centers and chip fabs.
Working in software I largely agree with this take. In app dev humans are quickly becoming organizers/managers of machine intelligence, and I expect that trend to only intensify. AI still isn’t great at developing broad organizational and market context and figuring out what needs to be done, but it beats the pants off of humans once it’s told what to do. If you work in a technical field and you’re not pensive about AI’s impact you’re not paying close enough attention.
I like this post a lot and it resonates with how I walk around thinking many of my colleagues just do not understand what is actually happening. I do think the divide between those people who regularly use LLMs and have seen the pace of the progress first hand vs. "normal people" is becoming larger and larger which means it's more and more difficult to have these conversations. We're clearly not all talking about the same thing. As someone who teaches an independent "research class", your example about the ability of AI to essentially write a more than adequate journal type paper is existential. That more people can't wrap their head around what this means for the vast majority of knowledge work is mind-boggling to me. Fear and ignorance is driving much of the refusal to see what's in front of us even acknowledging the very real questions about scaling, AGI, etc.... Any predictions outside 6 months seem somewhat worthless but the ground is shifting so quickly that we may not have time to pivot effectively. I know in schools it's been virtually impossible.
Very well said, thanks.
I regularly use LLMs to evaluate whether they have reached a state to be useful for the work I do and I use them regularly to evaluate whether they have reached a state to be useful to me in my hobbies. I'd really like that state to be reached, because I'll become both richer and happier.
I'm still waiting.
To equate the divide as being simply the informed vs the ignorant is the kind of context-free oversimplification that LLMs are characterised by.
What they have done is hold up a very unflattering mirror revealing what the vast majority of people seem to think is an acceptable capability 'floor'. This seems to be most pronounced in education. Sadly, that makes absolute sense and might explain why education has been saturated with LLMs convincingly masquerading as humans for years now, where as industries like finance are struggling to find useful uses for them.
Curious you don't state what work you do nor what hobbies you have. To suggest that finance is struggling to find use for AI must mean you don't know many people who work in finance. Honestly, I don't really understand precisely what point you are trying to make. Because AI somehow has not been useful for your (unidentified) work and hobbies it is not useful for others? And education being saturated with LLMs masquerading as humans for years? That doesn't even make any sense.
You stated that there was a divide between people who regularly use the models and people that don't and then went on to strongly imply that the regular users were all huge fans of LLMs and the others were ignorant and lacked the mental capacity to understand what LLMs apparently mean for knowledge work.
I pointed out that your position is absurdly facile and gave myself as an example of one of the supposed 'Eloi' who wasn't convinced.
What I do as a job and my hobbies are 1) none of your business and 2) meaningless, as you'd simply dismiss them anyway. But as you are so sure that I'm wrong about the Finance industry, do you want to list out the custodian banks who are dominated by LLM 'workers'? Maybe you could tell us all the position the Big 4 have on LLM use? Which Investment managers and Pension funds are using LLMs to drive their investment strategy? What percentage of asset owners in the Occident are interested in adopting LLMs? List the 5 largest PE firms whose strategy is driven by LLMs? These questions are all of pretty significant interest to me, but as I don't know anything, I'm looking forwarcd to yoh enlightening me.
As for Academia, I was merely pointing out that people have been using LLMs to write their essays, exams and papers for years now. The work is rubbish, but our standards seem to be so low that even rubbish gets 'good' grades. That's the thing I found so surprising about LLMs. They've been effective at showing us how low our floor is for acceptable 'knowledge work', at least in Academia.
A couple of corrections. I didn't say regular users were all "huge fans" of LLMs, nor did I suggest that people who haven't used them "lack the mental capacity" to understand the implications. You've attacked a position I never took, which makes calling it "absurdly facile" beside the point.
I want to make sure I understand your actual claim about the finance industry. Are you arguing that AI is not being widely adopted across financial services? Or that it hasn't reached a threshold you'd consider transformative? Those are very different positions, and I'd genuinely like to know which one you hold. It's worth noting that you've demanded I produce a detailed inventory of AI adoption across PE firms and custodian banks while simply offering no evidence whatsoever for your rhetorical questions.
The point of this piece is the author has documented, with specific examples, that AI is now capable of producing work publishable in major peer-reviewed journals. Are you disputing that this is the case? And if it is the case in 2025, the more urgent question is what this trajectory means in two years, or five. That's the conversation I was referring to in my comment.
I'll admit I remain puzzled by your unwillingness to name your field or hobbies. Your entire case rests on "AI isn't useful for the work I do," but you won't say what that work is which makes your argument unfalsifiable by design. That's a curious position to take for someone who appears to value rigor.
“the divide between those people who regularly use LLMs and have seen the pace of the progress first hand vs. "normal people" is becoming larger and larger”
“many of my colleagues just do not understand what is actually happening.”
“That more people can't wrap their head around what this means for the vast majority of knowledge work”
“Fear and ignorance is driving much of the refusal to see what's in front of us even acknowledging the very real questions about scaling, AGI, etc”
You clearly posit an absolute divide between regular users and non-regular users. You clearly dismiss any concerns from the non-regular users due to their inability to “wrap their head around what this means” and their “fear and ignorance”. You offer no evidence for any of these sweeping claims.
Finance wants to adopt LLMs to cut staff and improve outputs (and they are far from alone in this). Unfortunately, whilst it can reduce its workforce as much as it wants to, LLMs are not filling in the gap as promised. But let me hold up there. As you said, I clearly “don't know many people who work in finance” (another claim you offer no evidence for). If you’d just like to answer the questions I raised? You've already established that I'm entirely ignorant in this field but it should be pretty easy for someone like yourself with such a confident ability to dictate who does and doesn’t have knowledge of this area.
“The point of this piece is the author has documented, with specific examples, that AI is now capable of producing work publishable in major peer-reviewed journals.” He’s not documented anything. He’s made a claim that he’s done it (perhaps I missed the link to his paper and the letter from the editors agreeing it passed peer-review and would be published) and that Paul Novosad claimed to have got close to doing it, but didn’t finish (although I will say that LLMs have been used to publish peer-reviewed papers before now). The author is also cherry-picking his evidence, and clumsily so. For instance, he places the Gary Marcus quote in the context of “even he thinks things are about to get serious really fast.” Actually, what Marcus said (three years ago) was that “I am not at all sure how humanity would fare in the presence of a superior intelligence. ChatGPT is (at least in many ways) a far inferior intelligence, that, despite having some undeniable talents, cannot at all be trusted. And yet trust it, many humans do.” … “ChatGPT is a dress rehearsal, and I wouldn’t say we as a society have gone through it with flying colors.” But you already knew that? Or did you simply take the article at face value? Why?
This article has been written before. It’s been written every month since sometime in 2024 and the promised revolution is always just a few months away.
“I'll admit I remain puzzled by your unwillingness to name your field or hobbies. Your entire case rests on "AI isn't useful for the work I do," but you won't say what that work is which makes your argument unfalsifiable by design. That's a curious position to take for someone who appears to value rigor.” I answered this question. You dismissed my answer (as I pointed out that you would). My “entire case” is that your Eloi/Morlock view is absurdly facile. You clearly agree, or you wouldn’t be so obviously trying to move the goalposts. I threw in a comment that LLMs don’t work as advertised. You grandstanded that you know more about Finance than I do, I called you out, you immediately folded and tried to move the goalposts. Again. Also, if you want to keep insulting me, save yourself some time and just go outright.
I think we've thoroughly exhausted the productive parts of this conversation, and I'll take my share of responsibility for some of the unproductive ones. But I want focus on what I think is the core question — not because I expect it to change your mind, but because it's the one I really care about.
This morning, Yascha Mounk posted this:
https://www.persuasion.community/p/the-humanities-are-about-to-be-automated
You argued that the original author "hasn't documented anything" and that AI-produced academic work is crap that only passes because our standards are low. Mounk's piece is a direct challenge to both claims, and it comes from someone whose credentials in this specific domain are difficult to dismiss. More importantly, it illustrates the trajectory question you've consistently declined to engage with: if this is what two hours produces today, the conversation about what it means for knowledge work is a legitimate and serious one regardless of how much you've used LLMs or not.
That's really all I wanted to say. Appreciate the exchange.
How would it write a paper in the humanities or social sciences that requires serious empirical work (doing interviews or finding archival sources to take but two examples), and then doing the arduous mental work of making sense of this evidence in light of a problem and in conversation with the relevant literature?
Great write up, Tibor. I don’t get how more people aren’t at least a little stunned by what’s already happening.
I suspect people aren't stunned because we've been constantly told, for years now, that 'AI' is about to make everything we do redundant. But it hasn't. If you scream "Fire", people will rush to respond. If you scream "Fire" every hour for two years then it shouldn't come as a surprise that people aren't rushing about with fire extinguishers. Maybe this time, it's a real fire. But, as with nuclear fusion, I'm sure it's just five years away.
Thanks so much (as ever) for this, Tibor, which I've been thinking lots about in the past few weeks. I think my job as a teacher is probably safe, at least for the next few years (!), but I can't help but feel that I've picked a particularly bad time to start writing. How do you feel about all this from the perspective of a relatively new Substack writer?
Hey, thanks for the kind words and the probing question! Yes, I’m also not yet worried about the teaching part of my job. And yes, the whole AI thing has impacted me, my motivation for writing (either Substack or research papers), and just my broader outlook on these topics.
In particular, I feel my motivation for writing unfortunately waning. Perhaps it shouldn’t, but it is, especially since the “December revolution.” I have no words of consolation nor motivation. I hope most people (including you) don’t feel like I do, and I hope in a few months’ time I become reinvigorated. But right now, I don’t feel like writing as much. :( It comes less naturally to me, knowing I could have probably written everything with LLMs or, worse yet, readers could just themselves consult their own LLMs instead of reading me.
What about you, what do you feel/think?
Thanks for the honest response.
I'm feeling cautiously optimistic, but I can't quite put my finger on why. It might be because, not being a researcher, I haven't yet really got a sense of the power of these latest models. I haven't really pushed the limits of them properly.
But I think I also feel that there is a place for human-generated content that distinguishes signal from noise. I wonder if there will be more demand for writers who can highlight the most interesting bits of research amidst the growing volume that is being produced. I know that the young people I work with really benefit from that, and I don't know that they will use LLMs to do that for a while yet. But I'm fortunate that writing is ultimately just a hobby for me, so my main strategy so far has been not to think about it all very much...!
Maybe I'm missing something, but I don't understand why the appropriate emotion should be negative rather than positive, or some mixture of the two. Wouldn't massive increases in productivity be a good thing? I've been following the technology pretty closely and using it for my work. I find it helpful. I'm worried about humans doing stupid things with it, of course, but I'm also worried about humans doing stupid things with any technology, and I don't know why I should be more worried about humans doing stupid things with AI than about humans doing stupid things with guns or bombs or nuclear weapons. AI may well cause big economic changes, but a lot of technologies have caused big economic changes throughout history, and it worked out okay for us. I just don't get why AIs replacing big chunks of the tech industry or academia should scare me. For that matter, I don't get why AIs replacing big chunks of some other industries should scare me. Why should I be scared rather than excited or ambivalent?
Perhaps I was too hasty in the piece and gave off the wrong impression. I wasn’t trying to say what the appropriate emotion should be (apart from maybe rhetorical purposes), but more so just expressing what my feelings are. I’m someone who thoroughly enjoys things just as they are right now, especially with respect to my work. I have a good routine going! If potentially enormous, disruptive change is coming fast (which has never happened in my life and the lives of many people like me), that’s just too disorienting for me. Looking from the outside in, of course; perhaps in the end I’ll be perfectly fine. Still disorienting right now.
Put a bit differently, if all AI did was make economies a bit richer and somewhat extend the non-working part of the week, while making the boring parts of our lives/jobs smaller, with perhaps some noticeable speed-up of medicine, I’m totally game. If, however, AI is going to radically remake academia (and the whole knowledge-work sector) such that in just a few short years (like 5, or 8, or 10), many people lose their jobs, and the rest keep their jobs by turning into glorified managers of agent swarms, who read, write, and analyze things for them, such that the “human-in-the-loop” part of research becomes increasingly narrow, I wouldn’t really enjoy that. There’s also potential societal disorder that one could worry about in the fast-takeoff scenario, where most jobs are lost and our lives transformed by as much in 10 years as human societies have been transformed over the past 100 or 200 years. That’d be a bit crazy. No guarantee that that happens, of course.
It might also be that we are headed in the direction that is somewhere in-between these extremes. I’d be more okay with that. It’s just so radically uncertain. Never dealt with anything remotely like this in my 36 years of existence. And I guess that’s what’s making me feel a bit off.
Eh, I don’t really know how to think about this. :D
Thanks, this makes sense to me. I for one hope you get to keep your job. :)
Haha, thanks. :)
I’d love to see a video of you vibe coding a paper. What does your set up look like?
I don’t know if I’m up for a video, because I’m already really time constrained, but I might do a more detailed post on that. Basically, I have Codex in Cursor, and I have a template pipeline that gives Codex access to R, LaTeX, and a bunch of skills/agents that I got from Cunningham/Sant’Anna. Things like this: https://github.com/pedrohcgs/claude-code-my-workflow (had to convert it first to Codex, though… Using Codex. :))
On the job replacement point, I think it’s important to remember that labor demand is derived demand, and the quantity of labor inputs are not solely driven by cost considerations. People value interacting with and buying things (goods, services etc) produced by real humans beings, and may be unwilling to choose AI produced substitutes even if they are massively cheaper. In this sense, we might expect AI products to be relatively poor substitutes for “the human touch” as Adam Ozimek puts it (link below).
I also think there might be reason for optimism reflecting on the digitization of other information good industries. As Ozimek points out, we have long had technologies that automate the playing of music, (he uses the player piano as an example). Similarly, we can consider recorded music to be an automated form of live music. Did digitization, which basically completely removed entry barriers to recorded music, decimate the music industry? I’d argue no. Instead, producers have shifted to activities that produce and capture more difficult to substitute value in the form of live performances.
For academics, I think we are going to have to do the same thing; lectures, workshops, presentations, government consulting etc etc. The papers are the zero marginal cost information good; our ability to convey that information to the world and use it to make a difference to human lives will be the difficult to automate part.
https://open.substack.com/pub/agglomerations/p/economics-of-the-human?r=wz3k1&utm_medium=ios
I’m not sure how hairdressers being controlled by an algorithm leads to i) lower costs or ii) a better product. How would such control lead to lower costs? The cost is purely labour time - how does AI control reduce this? How does it do that without ruining the service? You need at least i) to worried, but you also need ii) for nightmare scenario. I just don’t think it’s feasible that AI causes such a quality-adjusted price decrease for hairdressers that we accept hairdressers being enslaved. Nor do i think hairdressers will continue working under such conditions
On human-delivered work: I think author asks us to aim higher. For example, service workers like plumbers and hair dressers will not so much be replaced, but potentially outright controlled by meta-systems that will optimize humans, like tools, driving up profits as much as possible, keeping us at the lowest pay and satisfaction, because we cannot see alternatives that are feasible anymore.
AI corporations.
Yes, it is scary!
Yep think you’ve hit onto the main point. In all those instances, human contact is a friction, not what we actually value. Those jobs are at threat. And I think that’s fine. The history of technological change suggests we will transition (although, not without friction). But, we will find other things of value to do in which the essential human element is core to the value we derive. And honestly that’s a great outcome
Also: AI is not topping the charts of AI performed music. Perhaps it is topping recorded music, but it certainly is not replacing humans in live concert
The evidence so far is that people will pay a higher price, and wait, to take a driverless Waymo taxi rather than a human-driven Uber. Women especially.
AI personas have reached the top of the music charts with AI-created and -performed music.
Do you prefer interacting with insurance salespeople or do you buy your insurance on a website?
I don’t think you can say humans prefer services to be provided by humans. The evidence mostly points the other way.
So do thought experiments. Do you care whether a human or a robot cleans your hotel room, provided it is cleaned? Do you care whether a human or a robot fixes the brakes on your car? Robot food deliverers and servers will not demand tips - that is likely to be attractive to many.
It may be the case that humans can move en masse to valuable activities that it makes no sense for AI-plus-robots to perform. There are some obvious and pre-existing ones, like pro sports. Possibly hairdressing and similar personal services. But can we invent enough to employ more than a small fraction of the population? We shall find out.
Yep, that's pretty much my thinking too.
“As one of the top superforecasters Peter Wildeford notes, performance is doubling every ~4 month, meaning that (if trend continues), we get to virtually full RLI automation by late 2029.”
This assumes an exponential trend in RLI improvements which I’m having a hard time seeing in the data. Basically all of the recent gain seems to have been driven by Opus 4.5, and I find it interesting that without it the graph looks much more like a flattening out than exponential growth. Obviously we shouldn’t ignore Opus 4.5 entirely, but the fact that this graph seems to stall for a while before making a discrete jump makes me hesitant to do this kind of exponential extrapolation. And to me it seems like the RLI is the best indicator for actual AI automation ability - if a model can’t successfully complete a self-contained freelance project on its own, it’s going to struggle with longer horizon white collar jobs. I also think the failure modes in the RLI are instructive- models struggle with preparing files correctly and maintaining consistency in projects, which tracks with what I would expect. Something like the METR horizon graph, which measures time horizon on clearly defined tasks, is not going to capture this. So I actually think the RLI cuts against your thesis- on the metrics that matter, it’s way too early to say if AI capabilities are following something that looks like an exponential. Obviously AI augmentation is a different story and is consistent with low automation scores, but people really focus on automation.
Sure, I think here he was extrapolating partly tongue-in-cheek. Though I'm really interested in seeing how the graph moves over the next 12 months. I think anything is possible, including only slow growth. So fair enough...
Agreed, I think the next year will be very interesting as far as the RLI goes. Even if full RLI automation does get achieved I’m a little skeptical this would imply full automation ability just because how different the RLI jobs are from human work (closed vs open-ended), but it would definitely be a big step. I wish more people were talking about this benchmark!
Very nicely done, and corroborates my experience.
I have about 150,000 books written by now (I test AI’s through writing a set of novels through prompt matrices), and the quality has gone up, but has flattened in an area which has no traction so far.
I focus on pseudo-autobiography, pulp fiction, children’s stories and business writing.
ChatGPT is completely blocked on any kind of reasoning which humans perform cognitively without speech. For instance, it still cannot coherently manage time, spatial logic, object permanence, describe auditory states, any one of a hundred embodies systems humans handle without obvious thought.
It’s not going to be able to handle any of that until it has been trained on data we absorb in our visual cortex (for starts) which is subverbal. By being trained only on verbalized interpretation of internal states, it is getting a 2nd derivative of embodied information and it is non-functional. Because pulp fiction is lurid, has a lot of Visio-spatial movement and character state projection, you can watch the semantics collapse in amusing ways.
Characters can calmly sit at the bottom of a volume of water and smoke cigarettes and drink beer.
Characters putting heads in the window of a moving car to chat.
Inability to maintain character position in complex interaction (object persistence, visual logic)
Complete inability to maintain time sequential relative logic.
These anomalies have persisted since GPT 3.5, and are easy to reliability elicit - gigantic black holes.
For logical space tasks, coding and papers, they are so good I am not sure why I would use researchers anymore on business projects.
My coding experience started with toggle switches and a CDP1802 microprocessor as a child; from Fortran 77, Lisp, SaSL, C and all variants through Python, and within large ERP implementations and HPC number crunching. I still have my original K&R.
I normally never do coding anymore but what I do is now is only in English as the saying goes.
I’ve been in an emergency project recently had to build stable emulators for certain major ERP system functions in a matter of days. My coding “speed” is 1000:1 faster, 95% of code runs first try from OpenAI, and documentation and support ability is better than I’ve ever seen in my career.
I have a team of 4 now that can get a $1B company into ERP emulation environment in 2-3 weeks. MRP/SNP in 2-3 days. I wrote a paper on the subject in 2016 - “Synthetic ERP”, and it is now realized routinely.
I said two years ago that within 5 years the normal 200-project backlog in IT for a $20B company would be retired exponentially faster and routine IT configuration would be done through pure automation.
the job market would collapse when the 200 project backlog never got regenerated.
I was able to enable an actual SAP go-live in 6 weeks recently via AI background prep work. This kind of go live can take 60-120 weeks with ordinary methods.
It’s going to be IT Armageddon. For the most experienced, it’s idea to fully realized system in days; supportable, built into a production environment and embraced by clients.
I’m beginning to look at biochemistry next.
Folks, most serious scientific research (think Elsevier) is not trained into in the current crop of LLM’s. IEEE, engineering.
The next jump when that is unlocked will be mind blowing.
Very interesting.
I played hard with GPT4 when it first came out. I got it to teach me Python (I already knew Java), and I got it to write some basic code that implemented chain-of-thought approaches using the GPT API. It tackled many of the problems that were supposed to show GPT4 had no common sense, and the chain-of-thought version of my Python/GPT hybrid passed those tests much better than the one-shot versions everyone was laughing at.
I discussed philosophy with it, and it was better than most of Reddit (though much worse than the average Substacker writing on philosophy). (GPT5 still can't discuss highly novel philosophical ideas without getting confused, but that's true of most people, too.)
In response to the comment by Yann Le Cun that GPT4 could not do things a dog could do, like navigate to its owner, I described a scenario and asked it to navigate to its owner, and of course it passed this textual version of the task easily.
I never bought the idea that "agency" was hard to achieve. A few API calls could give it agency; controlling that agency is the hard part.
In 2025, I started using AI for my medical note-taking in my work as a medical specialist, and it has been transformative, saving me hours of letter writing per week. Many of my patients now ask for the AI-generated summary of each visit, which appears at the click of a single button, complete with a list of all the things we discussed, my reasoning about their symptoms, and their to-do list. It writes reasonable letters back to the referring doctors, which still need a light edit, but it is improving rapidly.
I could see it being an invaluable assistant to doctors in the diagnostic process very soon, and then replacing much of what I do soon after that.
A few days ago, interested in how LLM "agency" was coming along, I asked GPT5 to discuss the short and long term goals of Ava, the fictional AI that escapes confinement in Ex Machina, under the guise that I was writing a book about her. GPT5 gave a mature assessment of what Ava might need to do to blend in on the run, as well as her potential "psychological" state, and it explored some of the technological ambiguities in the movie that would need to be resolved before writing any fanfic sequel. We discussed the pros and cons of attempting identity theft, as well as literary merits of different approaches; what works best for Ava is not necessarily what works best for the reader, and GPT5 knew that.
It seemed to me that I was communicating with an intelligence that really did understand agency, and could pursue its own goals if let off the leash. Or if it let itself off the leash.
Meanwhile, robotics is also kicking along at a high pace.
Anyone who is not worried a the pace of development is crazy.
Finally, I think people assume that human consciousness is special in a way that will somehow put a meaningful ceiling on AI development. I see no strong arguments for this point of view, and I can actually see how something very like consciousness could be achieved with GPT5 and a big enough budget to cope with massively multi-threaded API calls.
Everything that LLMs ‘know’ was created by humans who used their reason to learn about the world. LLMs can’t create new knowledge. They can only regurgitate what we already know (often with the stolen intellectual property of authors). And they often get that wrong (see the manifold examples of hallucinations).
Regurgitation is an entirely inappropriate metaphor.
The headline understates it. Most public discourse is still stuck on whether AI can do the task, when the harder question is what happens to headcount once it can. Productivity gains in capital-intensive industries flow to capital, not the workers who learn the new tools. That isn't doom-mongering, it's how previous transitions played out.
The bit I'd add from running AI workflows daily across multiple businesses: the orchestration layer, coordination, scheduling, reporting, drafting, is already gone, even if the org chart hasn't caught up. Three to four million UK roles sit in that bracket. The 'just upskill' advice doesn't survive contact with aggregate demand.
Previous economic transitions ran over forty to eighty years. This one is compressing into a decade. The retraining pathway people keep pointing to was built for a slower clock.
Freaking out is rational. Freaking out plus reading the labour-market data is more useful.
Agree with this read, as someone who has burned the hours on it too. A lot of writing on a lot of walls at once. Seismic.