Check out Taco Cohen's GATr (Geometric Algebra Transformer) which uses PGA, (Projective Geometric Algebra). That's basically a 3D Clifford Algebra with a null-square dimension added and some interpretation conventions. Flip a few signs in the tables and you get a hyperbolic or spherical geometry. It fits in tensor cores and other GPU hardware. Good for basic physics.
GPT found it pretty easy once past converting math in the pdf to Latex (seriously, there is no excuse for not embedding the Latex in the pdf, pdf might as well be encrypted). See bivector.net for tools, papers and SW.
Enon, thx for the pointer. Ive been meaning to dig into Cohen's GATr work. PGA sitting on tensor cores is exactly the kind of practical bridge that matters. And yeah the pdf-without-embedded-latex thing drives me nuts too... feels like archaeological excavation sometimes. will check bivector.net, appreciate the tip. Greetings
I agree with everything said here. Also, one of the problems with the human cognition (just my opinion) is the way it tends to see or at least intuit Ai as a kind of mirror. Not unlike the mirror in Harry Potter which makes you see what you want to see. I think Ai hallucinations are a shared experience and partly stem from a bias in the model to gratify the user.
That is indeed a sharp observation. the mirror thing is real, people project what they want onto the model and then get surprised when it reflects their own biases back. Hallucinations as a shared experience, I had NOT framed it that way but you are onto something. It s not just the model failing, its the whole feedback loop, yeah
Thank you. This has been driving me nuts. I’m like is humanity this dumb and crap is it a psyop to get us to believe it is?
It concerns me because AI isn’t being priced as a technology — it’s being priced as the future growth story that keeps the dollar, deficits, and valuations legible.
When that story shows strain, failure can’t be admitted without destabilizing the system that depends on it, so misfit is reframed as insufficient adoption or belief. That’s not gonna be a bubble popping — it’s internalized collapse, where correction is replaced by narrative load-bearing. But at least all the AI’s agree that it’s a reasonable conclusion so I hope they’re wrong 😜
Christine, "narrative load-bearing"... better than anything I've come up with. Im gonna stealing 😁 You nailed it: when the story becomes structurally necessary, failure just gets reclassified as insufficient faith. That's not a market, that's a religion with a ticker symbol. Thx for reading.
Spot on. When I mention Prof Amari's groundbreaking work on Information Geometry to people working on AI systems I get blank stares. Same with Pearl's causal inference. People working in AI must study more and deeper mathematics and statistics. Also, validations of models are always incomplete: it's simply not enough to report discrimination accuracy; calibration accuracy is vital. I recently validated an AI model offered by a company as a complete solution to a problem in healthcare: not only discrimination was low in the local population (area under the ROC around 60%), but calibration showed that the clinicians would simply be better off deciding based on a biased coin toss (according to the prevalence of the condition.)
The article has links to all the theory and fully working Takens' based transformer and all the links - ie. fully grounded (takens-transformer.com and more). All the best - Kevin R. Haylett
This is an aspect some people have been trying to work on, although maybe not with the right intuition shift described here, with things like Poincaré sphere embeddings for instance. The idea being that using a hyperbolic space was a natural way of expressing hierarchical relationship between points embedded in there. What I maybe had found missing in the literature in there was a more explicit connexion to Causal structure , as defined with Lorentzian manifold, in order to properly equip the latent space with the kind of tools that could better take advantage of the paths costs that you have outlined.
I am not a working mathematician at all, my grasp of the theoretical tools that are needed to say anything meaningful past my own "intuition" here is lacking to say the least, but I came across that kind of consideration from a very mundane problem at work years ago, that required to "follow the right path" along a chain of characters in order to find a precise concept there. And I got frustrated with things like Bert, Word2Vec etc... For their complete lack of ability to properly account for the curvature between points, where as you point out lays the real meaning.
This helped explain something I’ve been circling through design work with LLMs. I don’t come from a formal math or science background, but while building projects I noticed that changes in tone, pacing, and constraint seemed to shape the informational space the system moved through.
I didn’t have language for it at the time, but learning about latent space and thinking about it geometrically helped that experience click. Thanks for sharing, it made me feel like I was on the right path. I’m curious whether tone has come up explicitly in your work.
Check out Taco Cohen's GATr (Geometric Algebra Transformer) which uses PGA, (Projective Geometric Algebra). That's basically a 3D Clifford Algebra with a null-square dimension added and some interpretation conventions. Flip a few signs in the tables and you get a hyperbolic or spherical geometry. It fits in tensor cores and other GPU hardware. Good for basic physics.
GPT found it pretty easy once past converting math in the pdf to Latex (seriously, there is no excuse for not embedding the Latex in the pdf, pdf might as well be encrypted). See bivector.net for tools, papers and SW.
Enon, thx for the pointer. Ive been meaning to dig into Cohen's GATr work. PGA sitting on tensor cores is exactly the kind of practical bridge that matters. And yeah the pdf-without-embedded-latex thing drives me nuts too... feels like archaeological excavation sometimes. will check bivector.net, appreciate the tip. Greetings
I agree with everything said here. Also, one of the problems with the human cognition (just my opinion) is the way it tends to see or at least intuit Ai as a kind of mirror. Not unlike the mirror in Harry Potter which makes you see what you want to see. I think Ai hallucinations are a shared experience and partly stem from a bias in the model to gratify the user.
ExoArtDP
That is indeed a sharp observation. the mirror thing is real, people project what they want onto the model and then get surprised when it reflects their own biases back. Hallucinations as a shared experience, I had NOT framed it that way but you are onto something. It s not just the model failing, its the whole feedback loop, yeah
Thank you. This has been driving me nuts. I’m like is humanity this dumb and crap is it a psyop to get us to believe it is?
It concerns me because AI isn’t being priced as a technology — it’s being priced as the future growth story that keeps the dollar, deficits, and valuations legible.
When that story shows strain, failure can’t be admitted without destabilizing the system that depends on it, so misfit is reframed as insufficient adoption or belief. That’s not gonna be a bubble popping — it’s internalized collapse, where correction is replaced by narrative load-bearing. But at least all the AI’s agree that it’s a reasonable conclusion so I hope they’re wrong 😜
Christine, "narrative load-bearing"... better than anything I've come up with. Im gonna stealing 😁 You nailed it: when the story becomes structurally necessary, failure just gets reclassified as insufficient faith. That's not a market, that's a religion with a ticker symbol. Thx for reading.
Your insights here are very helpful and welcome! Thank you!
Glad it landed. thanks for reading; Prince
Spot on. When I mention Prof Amari's groundbreaking work on Information Geometry to people working on AI systems I get blank stares. Same with Pearl's causal inference. People working in AI must study more and deeper mathematics and statistics. Also, validations of models are always incomplete: it's simply not enough to report discrimination accuracy; calibration accuracy is vital. I recently validated an AI model offered by a company as a complete solution to a problem in healthcare: not only discrimination was low in the local population (area under the ROC around 60%), but calibration showed that the clinicians would simply be better off deciding based on a biased coin toss (according to the prevalence of the condition.)
What makes you think i have a math intuition about AI?
I agree with your conclusion. The rest seems to be pretentious mumble jumble that satisfies your ego. Happy masturbation!
Hi Jose, Perhaps, you would be interested in this.
https://kevinhaylett.substack.com/p/geofinitism-language-as-a-nonlinear
The article has links to all the theory and fully working Takens' based transformer and all the links - ie. fully grounded (takens-transformer.com and more). All the best - Kevin R. Haylett
This is an aspect some people have been trying to work on, although maybe not with the right intuition shift described here, with things like Poincaré sphere embeddings for instance. The idea being that using a hyperbolic space was a natural way of expressing hierarchical relationship between points embedded in there. What I maybe had found missing in the literature in there was a more explicit connexion to Causal structure , as defined with Lorentzian manifold, in order to properly equip the latent space with the kind of tools that could better take advantage of the paths costs that you have outlined.
I am not a working mathematician at all, my grasp of the theoretical tools that are needed to say anything meaningful past my own "intuition" here is lacking to say the least, but I came across that kind of consideration from a very mundane problem at work years ago, that required to "follow the right path" along a chain of characters in order to find a precise concept there. And I got frustrated with things like Bert, Word2Vec etc... For their complete lack of ability to properly account for the curvature between points, where as you point out lays the real meaning.
This helped explain something I’ve been circling through design work with LLMs. I don’t come from a formal math or science background, but while building projects I noticed that changes in tone, pacing, and constraint seemed to shape the informational space the system moved through.
I didn’t have language for it at the time, but learning about latent space and thinking about it geometrically helped that experience click. Thanks for sharing, it made me feel like I was on the right path. I’m curious whether tone has come up explicitly in your work.
i’m pretty sure alignment is responsible for all the major failure modes.