The wrong question

These are the days of AI’s black legend, and the AI doomers own the headlines. What matters is this: they may be right about the danger. But they may be right the way a stopped clock is right twice a day: by accident, and for none of the reasons written on its dial.

The doom scenario is real enough to take seriously, and sooner rather than later. But the reasons we are told it could happen are fatally wrong.

This story explains why, down to the math, while staying accessible to readers without a technical background, with animations and plots doing the heavy lifting where needed.

The AI Doom Debate by Tribes

Before we talk about boxes, contexts, or escape probabilities, you need to know the battlefield.

The AI doom debate has split into four recognizable tribes. Three think the machine may eventually kill us. One thinks the others need to calm down.

They disagree about timelines, probabilities, motives, and how seriously to take the whole thing. But almost all of them are still staring at the same variable: How powerful will the AI become?

That is where the argument has been stuck. And you will see that it is the wrong question altogether.

Now let’s meet the tribes.

The Rapture Bros

Eliezer Yudkowsky and Nate Soares — whose book is literally titled If Anyone Builds It, Everyone Dies — together with figures such as Dan Hendrycks and Andrew Critch, are among the most visible voices warning that advanced AI could pose an existential danger. Yudkowsky in particular has argued for extremely high probabilities of catastrophe from sufficiently advanced AI, while Hendrycks has pushed for AI extinction risk to be treated as a global priority.

Position, at the most extreme end of this camp: build sufficiently advanced AI without solving control first, and catastrophe may become overwhelmingly likely. p(doom): very high, with Yudkowsky near the extreme end.

They set the date for the end of the world, then asked you to preorder the book about it.

The Nervous Godfathers

Geoffrey Hinton and Yoshua Bengio are not exactly two random guys who discovered ChatGPT a few months ago and immediately started preparing the bunker. They are recipients of the 2018 Turing Award of the 2018 Turing Award, basically the Nobel Prize of computing, and Hinton later received the real thing, the Nobel Prize in Physics, for breakthroughs that made modern deep learning possible.

So after spending decades building the thing, making it better, and collecting some of science’s biggest prizes along the way, they suddenly looked at what they had created and thought: Hang on. How exactly are we supposed to stop this thing?

Position: they built the foundations of modern AI, and now they are warning everyone not to be too confident about where the building goes next. The danger is not certain, but it is serious enough that keep scaling and hope is not a strategy.

They built the fire, won the medals for combustion, and now show up at midnight asking whether anyone remembered to install the sprinklers.

The 25-Percenters

Read more

We are surrounded by myths all the time. Yes, even in science and technology.

By the time scientific results reach the public, they have often passed through expectations, money, marketing, and a lot of storytelling. The mathematical details that spoil a good headline rarely survive the trip intact.

AI is the extreme case

We have built an astonishingly powerful machine, but also an increasingly complicated one. When something breaks, the instinct is often to scale the machinery instead of questioning the mathematics underneath it.

But what if scale is not the problem at all? What if better mathematics can eliminate the problem before brute force even begins?

A small group of researchers is already exploring exactly that through geometry and symmetry, equivariance, topology, curved and hyperbolic representations, and related mathematics.

Most of this work still barely reaches the mainstream AI conversation.

That is where this story begins.

We are going to take some of the most familiar beliefs about AI and expose their weaknesses, and in some cases their outright falsehoods, using the mathematics that a more structurally sound AI could be built on.

You will see it for yourself through original animations that make these geometric and topological ideas almost impossible to miss: the real problem with AI may not be that our models are too small, but that we built them in the wrong mathematical space.

Don’t believe me?

Keep reading. By the end, your view of AI will be completely turned upside down.

Most of the AI World Still Isn’t Having This Conversation in Public

This is not another debate about consciousness, AGI, or whether AI will take your job.

I am talking about the mathematics underneath the models. As mentioned before, a small group of researchers is already working on geometry, equivariance, hyperbolic representations, symmetry, topology, and related ideas. But most of that work is still scattered across papers and small technical communities.

Meanwhile, if you are working on AI models or simply using them, you are probably getting tired of the same recipes for the shortcomings of current AI:

Bad representation? Add dimensions.
Weak understanding? Add examples.
Poor memory? Add context.
Weak performance? Train more.
Not enough capability? Give the model more freedom.

Sometimes that works, though with sloppy mathematics.

Often, it just gives you a bigger version of the same problem.

Which leaves a much more interesting question:

How much of what we force AI to learn by brute force is really a consequence of choosing the wrong mathematics in the first place?

That is the question behind everything that follows.

Five Dangerous Beliefs Killing AI, and the Mathematics That Fix Them

OK, you have already seen the problems and the false solutions the AI industry keeps repeating to deal with them.

Now comes the meat of the story.

In this section, you are going to see, as clearly as possible, how some of the myths and bad assumptions slowing AI down look very different once you take geometry and topology seriously and apply them directly to the problem.

Myth #1: More Examples Mean More Knowledge

Start with something embarrassingly simple. Something a current AI model can handle extremely well. For example:

The cat chased the mouse. The mouse was chased by the cat.

The words and the order changed, but they are still describing the same thing.

Now take something many of you have probably run into with AI: ask the model to deal with the same object rotated through dozens of different angles.

The pixels change every time, but the object is still the same object.

If the rotation does not matter to the task, treating every angle as a completely new case is just mathematically wasteful.

Geometry has a very simple way to handle this. Take one object and apply all the transformations that should leave its identity unchanged. The whole family of versions you get is what mathematicians call an orbit.

If those differences do not matter for the task, you can treat that entire orbit as one case instead of hundreds. The resulting space is called a quotient space.

That is the important part: orbits tell you which examples are really versions of the same thing. Quotients let you stop treating those versions as separate problems.

And yet many AI systems still pay for that redundancy through extra data, augmentation, and brute-force fitting.

The animation below makes the difference almost embarrassingly clear.

The system on the right does not magically know more. It simply knows one structural rule that saves it from relearning the same thing hundreds of times.

I call the cost of ignoring that invariance debt: every transformed copy learned separately is another payment.

The theorem underneath is simple. When the symmetry is real, an invariant function factors through the quotient.

In plain English:

Learn what actually changes. Stop relearning what should not matter.

Myth #2: The More of Your Conversation AI Can See, the Better It Remembers

This one will feel familiar to many.

Every few months, the industry announces a bigger context window, meaning the model can keep more of the conversation, documents, and previous instructions in front of it at once.

And from there comes the almost automatic conclusion:

more context = better memory.

But that quietly mixes up two very different things.

Having more of the past available to read is not the same as having a better memory of how that past should affect what happens next.

Imagine you go to dinner with someone who tells you right at the start:

“I hate mushrooms.”

An hour later, after talking about work, holidays, ex-partners, inflation, and why the waiter has vanished for twenty minutes, you order pizza for both of you.

With mushrooms.

The information was there. The problem was what happened to it along the way.

Geometry has a surprisingly useful language for exactly this distinction.

A connection is a rule for how something changes while it moves through a space.

And parallel transport describes how you carry a state along a path.

Now replace physical movement with context.

Instead of thinking of a conversation as a giant pile of old tokens sitting behind the current prompt, think of it as a journey during which the internal representation is continuously transformed.

In geometry, something quite strange can happen: you can travel around a closed path, return to exactly where you started, and still find that your state has changed because of the route you took.

Remember the pizza?

Your friend said, “I hate mushrooms.” An hour later, after the conversation wandered everywhere, you ordered mushrooms anyway.

A bigger context window can keep that sentence visible. Holonomy asks what the journey through the conversation actually left behind in the state.

That is what the animation below shows: on one side, the loop leaves nothing. On the other, the path itself leaves a measurable trace.

And that is the difference between remembering the sentence and remembering the journey.

In mathematics, that leftover change has a name: holonomy.

Myth #3: The Order of Your Prompt Does Not Change the Answer

Anyone who has spent enough time with an LLM knows how ridiculous this can get.

Give the model the same background text, the same question, and the same answer choices.

Now move the background text from before the question to after it.

Nothing else changed. Yet the answer can.

Modern models are sequence models, so of course order matters. The real problem is that we usually notice how much it mattered only after the output goes sideways.

Geometry gives us another way to look at that: one candidate is torsion.

No, prompt sensitivity is not torsion. Torsion is more specific: in the right geometry, it measures one particular failure of local transport to close as expected.

So the useful question is:

Can some prompt-order effects be described by a geometric quantity instead of remaining just another surprise in the output?

That is what the animation below shows.

If that idea works, we stop merely noticing that order changed the answer and start measuring the effect itself.

Much better than memorizing another prompt trick.

Myth #4: More Dimensions Mean a Better Representation

You have probably seen what happens when an AI conversation gets too large. After enough questions, exceptions, side branches, and corrections, the model starts mixing things that were supposed to stay separate.

The problem is not just memory. It is structure.

A lot of what we give an LLM is naturally hierarchical: a plan has subplans, code has dependencies, an argument has branches, a category has subcategories, and one condition may apply only inside one particular branch.

Most LLMs still have to represent all of that inside high-dimensional vector spaces.

So the usual instinct is obvious: when the representation starts struggling, make the model bigger. Add dimensions. Add parameters. Give it more capacity.

But there is another possibility. Maybe the space has the wrong geometry.

Think about a tree. Every time you move one level deeper, the number of branches can multiply. Keep going and the amount of structure grows exponentially.

Ordinary Euclidean space does not grow that way. In a fixed number of dimensions, the amount of room available as you move outward grows only polynomially.

Hyperbolic space behaves very differently. Its available space grows exponentially too.

That makes it unusually well suited to genuinely tree-like structures, because the space expands at the same broad exponential rate as the hierarchy you are trying to fit inside it.

Flat space makes the hierarchy fight for room. Hyperbolic space grows with it.

That is the intuition behind the animation below, showing that hyperbolic geometry can make life much easier when the structure you are trying to represent is genuinely tree-like.

Not because curved space looks cooler. Because the geometry itself grows the way the hierarchy grows.

And this is another good reason to stop treating curvature as some exotic mathematical decoration.

For hierarchical structure, the geometry itself can do part of the work that we otherwise keep trying to recover with longer vectors, more parameters, and more engineering.

So instead of automatically asking, “How many more dimensions do we need?”, sometimes the better question is much simpler:

Why are we still expanding a flat space when changing the curvature fits the structure better?

That is the shift, the right intuition.

Myth #5: If AI Hasn’t Seen It Before, It Can’t Know It

Show an AI the same 3D object from a new angle and one thing has not changed: it is still the same object.

You can teach that by showing the model angle after angle, or you can give it the rule for rotation.

In 3D, those rotations have a precise mathematical structure: SO(3). Quaternions are one elegant way to represent them.

So instead of asking:

Have I seen this angle before?

the model can work with:

I know what rotation does.

It still has to learn the object. It just does not have to relearn rotation every time.

For a rotation-invariant task, the idea is simply: f(gx)=f(x). So you can rotate the input as many times as you want, and the answer must remain the same.

The animation below shows the same idea with Coxeter geometry. One region plus the reflection rules can generate the others. That is the point: you do not need more examples. You need the rule that connects them.

On one side, the system learns the symmetry example by example. On the other, the transformation law is already part of the geometry, so one case can determine the related ones.

That gives us three very different strategies:

sample the symmetry → encode the symmetry → discover the symmetry

The first keeps feeding the model examples. The second builds in the rule itself. In the case of SO(3), that rule simply describes all possible rotations of a 3D object.

And the third goes one step further: AI discovers that kind of rule by itself.

That is the real jump we are waiting for in the next generation of AI.

Yes, you read that correctly… the title says lack of innovation.

Absurd, right?

NVIDIA is worth trillions. OpenAI owns the headlines. Jensen Huang is treated like a technological prophet, while Sam Altman is crowned someone like the Steve Jobs of intelligence. These are supposed to be the hottest, smartest, most innovative companies on Earth, the places everyone wants to work for and every investor wants to own.

Their genius appears beyond question.

But let’s step away from the marketing stage and go backstage, where the bolts, code, and machinery actually live.

There, a more dangerous question appears: What if the companies leading AI are not building the future, but blocking it?

Because the next breakthrough may not need a larger model, another generation of GPUs, or one more trillion parameters.

It may require abandoning the foundations NVIDIA and OpenAI have turned into dogma: flat statistics, fake neurons, and brute-force computation.

And that is where the masks come off — where we see the real face of these companies, and how the architecture that made them powerful may now be standing in the way of real AI.

A better path already exists. But AI must first escape the architecture that made today’s giants rich — and the technology dangerously fragile.

The current situation of NVIDIA and OpenAI

NVIDIA has two faces, like Janus.

The first is the face of marketing: the triumphant story Jensen Huang tells whenever he walks onto a stage carrying another GPU

The second face is backstage, where the reality is far less glamorous.

Strip away the theater and one uncomfortable question remains: why are the descendants of hardware built to render graphics and animate video games now sitting at the center of the so-called AI revolution?

For most of NVIDIA’s history, gaming was not some side business. It was the business. Some veteran gamers will remember that, as late as 2014, demand for high-end GeForce cards still drove much of NVIDIA’s GPU revenue.

Of course, we have all heard the GPU evangelists preach the gospel: billions of operations, massive parallelism, oceans of computational power.

But once the spectacle is gone, the miracle looks much simpler.

A GPU is an extraordinarily fast machine for multiplying and adding numbers across enormous matrices. A glorified mathematical coprocessor: brutally efficient at arithmetic, yet showing no trace of the intelligence celebrated so triumphantly on NVIDIA’s stages.

Fine. Then perhaps the real secret is not the chip, but CUDA, NVIDIA’s distinctive software platform. Surely this is where raw hardware becomes programmable intelligence. Is CUDA the missing bridge between matrix computation and cognition?

Unfortunately, CUDA is a venerable platform introduced in 2006. Over the past two decades, it has accumulated new libraries, tools, memory systems, language features, and AI-specific optimizations. But beneath all those additions, its basic computational worldview has changed surprisingly little.

And CUDA deserves credit. It took the parallel power of graphics processors beyond graphics and opened it to general-purpose computing. At the time, the idea was revolutionary: launch thousands of threads, divide the work into blocks and grids, and make the GPU process enormous quantities of numbers at once.

But two decades later, a geological era in technology, CUDA still rests on essentially the same idea: break a numerical problem into even more parallel operations and run them even faster.

That is the whole trick.

A remarkable engine for accelerated computation. But still no bridge from arithmetic to cognition. Nope.

The Case of OpenAI

Read more

Software Development Is Already on AI Life Support

Many of us saw the writing on the wall the moment the COVID pandemic lifted and the new generation of transformer chatbots arrived.

First, they gutted Stack Overflow.

Now they are coming for entire professions, and software development is the first white-collar field where the damage is impossible to ignore.

Junior and mid-level positions are shrinking at a pace the industry has never seen before, and the squeeze is now reaching senior roles too.

On the other side of the coin, for now, the view looks better.

AI engineers, and everyone whose output has been multiplied by an AI assistant, seem to be thriving.

But are they actually safe?

You Need an AI Workflow. And Now It Is Almost Too Late.

The first brutal lesson from the wreckage of tech jobs is simple:

This is no longer a matter of choice.

If you work in tech, you either build your own workflow of AI assistants now, or you start looking for a skilled trade outside the blast radius.

Yes, humans are still needed in the loop. Yes, today’s flat AI is still far from replacing a serious developer end to end. And yes, if you ship production code, you still supervise everything the assistant hands you.

But none of that is an argument against using AI.

It is an argument against using it badly.

And the worst way to use AI is to lean on a single assistant as if it were a magic black box.

Claude, Cursor, ChatGPT, Copilot, whatever name you prefer, none of them should be trusted alone.

Some of you already know this. Unfortunately, many still do not: one AI code assistant is not a workflow. It can miss the context, misunderstand the problem, and dress a bad fix in clean code.

So if your whole setup is one assistant and blind trust, you are not engineering with AI. You are working with a chimera: part autocomplete, part sycophantic parrot.

And because today’s AI still lacks real Bayesian judgment, you have to do something the model will not reliably do for you: read the confusion matrix.

Read more