> Before approving construction, I would want communities of humans to understand why the design works and what justifies confidence in its safety. I would hope that we all would.

Until very recently, I pored over every single line of code Claude generated with razor sharp scrutiny. I would usually catch issues with every response. I'm catching fewer problems these days. Maybe the model is just getting better, and maybe I'm being less careful while under pressure to ship more and more often. But model capability is obviously growing. Even back in March, you could tell it "give me a function that adds two numbers" and you could be 100% confident that it would write the correct function. There was almost no point in looking at the code. Since then, the complexity floor of problems in the category "this is so simple that the model couldn't possibly get it wrong" is rising, and with it, my cognitive surrender to the model is increasing too. Why check it? It's obviously going to be correct.

If AI designs a terawatt fusion plant, then of course we're going to meticulously pore over every detail to ensure safety, reliability, efficiency, whatever. If we find no flaws in the design whatsoever, will we be less careful about the second one? The third one? What about the ten thousandth one? Will "a nuclear fusion plant" become something that models couldn't possibly get wrong?

Terence Tao is arguing that the human involvement in research is crucial, but doesn't convincingly justify why, in my opinion. He says that "human agency is a value of fundamental importance" and that we will need to build "thriving human communities that can understand [AI ideas] together" - not for the sake of correctness, which AI may surpass us on, but for, I guess, the possibility of reclaiming human meaning and purpose. I don't disagree with this at all, but it's not an argument, it's a statement of values. Unfortunately, the stark reality is that if AI does surpass humans, it will become the economically dominant strategy to not verify them and not double check them, but to just do whatever they say. This seems like a great way to raise p(doom). But as the models get better and better, and as I'm scrutinizing Claude's output less and less... I just hope that there are more Terence Taos out there than people like me.

There is a failure to understand that the process is the result. You don't study mathematics or computer science and information theory to produce commodities. You study them to transform your mind. The output of an LLM is useless without a human mind to comprehend it. We can have Super Intelligence, but if humans are incapable of comprehending it, it is just another useless dead artifact. Practice, applied over a lifetime, is what creates the capability for comprehension. Asking an LLM to give you an answer creates an artifact. Humans being humans, most of their requests boil down to "make me rich without having to work for it," so the request itself is paradoxical and impossible to satisfy. Philosophers have only been saying this for all of human history, so don't hold your breath for any breakthroughs.

I know many programmers for whom "developing domain understanding" is an abstract concept (if indeed, it is a concept for them at all). But in the age of big AI, I see more need for this, not less.

At least this is my observation: when my colleagues have been wholesale chucking stuff over to Claude, they've then been confronted with classic XY-Problem shit, poor user experiences, and over-complex solutions (which will mount future problems regardless of whether a person or an agent iterates on that code). Much of this can be solved by actually sitting down and thinking about it, and I mean at a code design level, not just a speccing level.

Many people don't realize that there are more useful outputs to solving a problem than just a mere solution. Obviously if one has a contractor mindset (you don't care about the after effects of a system) then this is of no relevance to you. Some companies promote that mindset, certainly ones that have no broader aspirations then getting acquired soon. That's fine - but many companies actually are about sustainability, and understanding in these places is paramount.

Like most of us I go back and forth between sheer optimism and fear for the future that AI may usher in. Recently I started vibe coding a fun video game with my ten year old. The experience is different than my work because it’s been such a joy to basically have a personal genie in a bottle help me make some personal art with a loved one regardless of either of our skill sets. The concept the author of this post is arguing now resonates with me more than it would have a few weeks ago. The sheer surface area that AI can create in our intellectual life is limitless and needs humans to explore. There can never be enough of us in that sense. Whether it’s as validators or creators.

I feel like “giving up understanding” is inevitable.

There’s some hubris in thinking we can understand everything. For truly difficult problems, it’s entirely possible that humans are simply incapable of comprehending why a solution is true. But ultimately the practical value of applying that solution to the real world is going to eclipse our need to understand it.

Math is just the beginning. I see it happening in other fields too, like physics and biology. Many of us software devs have already given up on understanding parts of our own systems for the exact same reason.

Seems like a losing battle.

"We’re gonna need a lot more mathematicians."

What the article really says is that we're going to need much smarter mathematicians. That is not possible for puny meat-brain humans. Humans are close to their ceiling. AIs are just getting started.

In practice, we're probably going to hit that limit first in IC design. I once went to a talk by the Intel engineering manager who headed the Pentium Pro effort. That was the first superscalar x86 CPU, and it took about 5,000 engineers at peak to design it. Getting that many people coordinated on one thing was a real achievement. Then Intel stayed with minor tweaks on that design for years.

We're soon going to be seeing designs of even greater complexity cranked out by AIs. No human will understand them at the gate level. Reading AI-written programming language code is bad enough. Reading AI-written Verilog may be beyond human comprehension, except in small sections.

All I hear is cope. I’m very certain that my job can be automated and that I don’t posses intelligence above the models these days. Sure, I’m needed in ways that AI can’t so certain things but I don’t want to pretend like it’s not going to improve on those areas. And unfortunately the Math community need to set aside their ego and come to terms with it. I don’t understand the “but humans are this and that” arguments. AI should bring humility among us. May be every other species got humbled by humans and now we have AI (May be not exactly the AI now but soon)

We're going to have people understand maths more now than ever before.

See proofs never before possible made by someones intuition.

But maybe they wont be mathematicians.

> Before approving construction, I would want communities of humans to understand why the design works and what justifies confidence in its safety. I would hope that we all would.

Why would mathematicians be the best at this or even able to contribute? Engineers and physicist seem like a much better choice. Mathematicians tend to not bother with messy things like whats physically possible, or human consequences etc.

I find this discussion odd. 50% of people mistake this guest post for a post by Tao himself. The post is by Amit Sahai, professor for computer science and math at UCLA.

Tao has been posting a flood of guest posts that stress inevitability and coping. Tao is fully invested in AI but needs to have the appearance of a broad discussion.

Sahai is of course exuberant about AI but wants to keep UCLA enrollment numbers high. The comments on Tao's blog are far less friendly than here, because by people see through the game that is being played.

Just look at the latest industry friendly post from Tao, where he pretends that it somehow summarizes the discussion so far:

https://terrytao.wordpress.com/2026/09/25/iciam-statement-on...

For my whole life I have never been subjected to an industry coordinated advertisement campaign that ruthlessly harnesses YouTubers, TikTokers, professors, open source people who all go in lockstep.

Tao, Gowers et.al. will go into history as the professors who ruined math.

The main reason to learn something is actually being able to communicate in the language of that subject. There are complex ideas in math that cannot be easily captured by the language of other fields. Pepole who don't study math cannot even understand what a worthwhile goal in math even is or how it could be useful to other fields.

You can't simply prompt a model to be "better" when "better" isnt even properly defined

> To take on this responsibility, we may need to broaden our view of what a mathematician can contribute. I have in mind something like a “deployable intellectual reserve”: communities of mathematically sophisticated people that humanity can call upon to help understand consequential AI-enabled breakthroughs.

> Our ability to understand difficult and unfamiliar ideas may become one of the most important contributions we can offer to society. We should be willing to bring that skill to problems far beyond our usual research interests. [3] Doing so asks us to expand our sense of our vocation.

Am I reading this wrong, or is he talking about what (mathematitian) Data Scientists have been doing for years? So he is basically saying that former Data Scientist that have turned into prompt/software engineers should go back to being data scientists.

In any case, people should stop trying to fit AI in the previous status quo. What we need is curious people, that is what we have always needed.

A few centuries ago there were no "mathematitians", there were mathematitians/philosophers/artists/physicists all in one person. So its not like "mathematitias" is something that has existed for millenia.

We need curious and ethical people.

Maybe AI brings back the age of a well rounded scientist/philosopher. I know this sounds counter intuitive because the article is saying that we cannot keep up with the AI.

A lot of the conversations around AI in the past years have been “we will delegate almost all work to AI, but the most decisions will be left to humans.”

I’m fairly confident that the opposite will be true. The most important decisions will be made by AI, and humans will only be left to guide decisions as a matter of taste. AI has the ability to be impartial, and immutable. You can endlessly probe and reason its decisions.

This isn’t true of our current human decisions. Where we see red tape, bureaucracy, rent seeking, status quo. AI will see through these human constructs.

I do like the idea that we will need more mathematicians, physicists, and scientists to understand the discoveries of AI. That might be the best possible outcome of AI, but I fear the opposite and we are painting ourselves into a corner of which we no longer have the knowledge to sustain ourselves and society itself collapses.

the dynamics of discovery on a population level are also not fully understood. there's no doubt in my mind that the machines will grab all the low hanging fruit in terms of novel ideas through mechanical recombination of existing public science, and they will do so faster than humans can. what is not understood is whether they will plateau.

today they benefit from human willingness to share publicly and improve ai systems, but once those systems start to threaten their livelihoods, will those dynamics change? can the machines push the frontiers or is the global network of human creativity and tenacity necessary?

> This is a guest post by Amit Sahai

Not Terrence Tao post. Beware.

If we accept that AI is going to do all of these things humans will be superfluous and will just be optimized away. It looks a lot like we see the birth of silicon based life by the efforts of carbon based life. Carbon based life will die and it will not even be because AI decided to kill it, it will be because feeding and watering it was less important than other concerns. Bacteria may continue to exist, though.

Is that already economic delamination? As in the singularity extracts the most valuable ressources (new thought generators) from the traditional economy keeping mankind alive and stable- towards itself?

> But we are now entering a time for humility: a time when all of us are going to know what it feels like to be unable to keep up.

Humility is the wrong word. We didn't feel it when the steam engine was introduced, so why now?

The problem is not that we fear the AI will do maths we can do. At least that's not the problem. Whether the AI can write better code, devise better proofs, etc, I still want to do it. The thing that worries us (the generation trying to enter the job market now) is just our livelihood. Food, rent & accomodation while I do the work. "Who's gonna pay me, and for what ?". That's the question I want answered. And please, if the response is some demeaning work like reviewing the gigantic ball with 1% meaningful insight, 90% slop and 9% other people pillaged work that LLMs will be dumping at my door all day long, then society may need more mathematicians, software engineers, etc but no one is gonna risk going along that path.

Russell once remarked that all of mathematics would be trivial to a sufficiently intelligent being. It builds conceptual tools for limited minds, that lets them understand far beyond their natural reach.

In a world with ASI, having that capacity is vital.

> "If AI fulfills its promise, we will encounter more beautiful and consequential ideas than we have ever seen."

It will be as it always has been: these ideas will further enrich the rich, at the expense of everyone else. We'll have our first quadrillionaire, while the masses are debating whether the minimum wage of $7.25/hr should be bumped up.

One question: LLM might answer correctly many logical and math problems, but I don't see any guarantee that when the LLM context receives new information the answer could get worst. Security in LLM answer is not a monotone increasing function of context size.

If we assume that trusting a model to execute an action is an ongoing exercise, since its trustworthiness is often discovered by the user organically as models develop, should trustworthiness be measured at the level of the model, or at the level of the human intent behind triggering it, whether explicitly or implicitly?

> The ideas and opinions presented here are entirely my own, but GPT 6 Astra was instrumental in helping me draft this note.

It really looks like Tao is on the path of accepting and embracing AI now.

Are you willing to pay for a lot more mathematicians?

Or it as with other professionals: shortage means shortage of cheap labor.

> Imagine that a future AI system proposes a radically new design for a one terawatt nuclear fusion power plant

This is an unfortunate example to choose, being as it is entirely confounded by the canonical illustration of the https://en.wikipedia.org/wiki/Law_of_triviality.

Life is all about tradeoffs. Mathematics have no tradeoffs. We need more engineers.

I keep clicking through on these because I am curious about what Terrence Tao has to say, and then they are always guest posts.

I think understanding the output of AI will become more and more likely impossible. It's not necessarily a bad thing, if we can benefit from AI produced knowledge, even if we didn't get there ourselves.

Always confused by text website that tries to prevent people from select or copy or right click.

"Imagine that a future AI system proposes a radically new design for a one terawatt nuclear fusion power plant."

I propose we enact a new law that makes it so ai engineers can only fly on ai designed planes powered by ai designed engines flown by ai. Maybe then we'll get some real progress out of this slop producing crapware or the problem will be solved a different way.

It would be nice to have more mathematicians, but we don't need more. Once AI math goes so far beyond human abilities, any human involvement is like an ant trying to understand quantum physics

you arent a real mathematician unless you lay around all day thinking about shapes

Mathletes, assemble!

> I have in mind something like a “deployable intellectual reserve”: communities of mathematically sophisticated people that humanity can call upon to help understand consequential AI-enabled breakthroughs.

I can’t afford to be one of your credentialed reserves, Amit. ADHD screwed me over in early life or I’d have breezed through that dual masters in cognets/compsci by 2003, and now that I’ve stabilized my brain and am burning through years of school at a rapid pace, my country’s socioeconomically ruined — I’ll be lucky to escape with my accounting degree as my school is visibly being sucked into the vortex every year I progress. The only hope left for me to be what you need of us is to self-study, but without a degree I’ll just be treated as a crank or an AI proxy/puppet if I slip up and talk about interesting math with anyone, so what’s even the point of taking that path? I originally pivoted my math skills into systems theory and process diagnostics instead, which of course now everyone has kicked to the curb and replaced with AI. I’d have made an excellent Susan Calvin — I was working to be a cognitive tech with diagnostics, zeroth law, and group psych specialties, AMA! — but the financial investment to provide the runway to take that lonely, dreary six year slow as molasses slog through maths that universities think is somehow an appropriate teaching velocity (six months for precalculus alone?!) in order to earn the mere chance to have my resume rejected by an AI firm that uses AI hiring and and can’t tolerate someone with a strong moral position regarding societal harms is a very bad choice, whether you use simple probability or game or systems theory to evaluate it. Taking that quarter-mil-plus burden as loans in the hopes of employment at the other end in a field actively having its social, reputational, and moral fabric being ripped apart by AI? That’s not just a bad bet, that’s chasing fool’s gold at the end of a fading rainbow in a desert mirage. So, with sincere apologies, while you’d benefit from having me on your ‘reserves’ list (I have written testimonials spanning some twenty-plus years to that effect) I’ll never come to your attention as a support tech or as an accountant, and I accepted that outcome years before anyone else realized this need for cogsci mathematicians with a teacher’s specialty of analogy-building and the ability to disregard interpersonal nonsense to focus on the needs of societies. Better luck with the next generation, though!

I think this is the most beautiful letter on the subject I've read all year, it brought a tear to my eye. It's like reading those famous STEM letters/essays from history.

Can we add guest post to the title? A lot of people here are attributing this to him incorrectly.

> Imagine that a future AI system proposes a radically new design for a one terawatt nuclear fusion power plant

I imagine:

* Massive budget overruns

* Year and year of delays

* It ending up wasting more energy than it produces, or just not working, period

* Politicians disclaiming responsibility and blaming it on "the AI"

* The contractor companies profiting immensely over the entire period, and ending up not having to worry about maintenance, warantees, etc.

TT is politely and gently trying to deal with the ego of many mathematicians, namely to teach them humility, which they will need to design their specialized maths AI models(agents?). Well, dunno if it will work out anything interesting, but he is giving a try.

I wonder if the current maths models, if any, are able to use formal solvers in their 'reasoning'. I wonder if a natural language interface is really that efficient, maybe a pure formal language hinted with intuitive "tokens". I remember the time I was learning real maths: "elegant", "brutal", "strong", etc were somewhat meaningfull.

> [This is a guest post by Amit Sahai. This blog post was initially written in a different file format and converted using AI. — T.]

No matter how good your case is - if you use AI slop to spam text, I will not read it. At the least they admitted to this sneaky slopness, which saves me time. I'd wish everyone would do so.

[flagged]

[dead]

[dead]

Mathematicians and physicists are just an organized cabal who are shamelessly profiting off of artificial complexity they create to maintain job security. The Yes people.

Watch me get downvoted some more. Nobody cares about the truth, we now know thanks to our beautiful president Trump telling us about that and the Fake News.

There's, like, one, two at most, mathematicians who are really thinking outside the box. The rest are like sheep. Or cult members with mass psychosis.

Waste, Fraud, and Abuse people are coming to academia, which is so hopelessly parasitised by the left.

"I have in mind something like a “deployable intellectual reserve”: communities of mathematically sophisticated people that humanity can call upon to help understand consequential AI-enabled breakthroughs. " -- I can't be the only one who thought about the scene where the scientists ask Deep Thought the ultimate question...