GPT o1 -- A Big Step, But Not Exactly Towards AGI

A friend from the crypto world — not an AI expert by any. means, but a smart and tech-savvy guy — messaged me recently to the effect

”For the camp that continues to claim that throwing massive amounts of compute at LLMs isn’t a reasonable path to AGI… I don’t know if that’s accurate anymore.

“I think we’ve reached the point where  these models are more capable than humans (even PHDs) at most tasks, and quickly approaching the median human in All tasks, in just about every way, so now it’s about arguing what the line in the sand for “AGI” actually is since the goal post seems to keep squirming around among researchers.”

This post gathers up some of what I said to him in reply….. (I doubt I fully convinced him but that’s the way it goes ! … )

These are more like rough notes than a polished essay, but welcome to the Internet.. ;)

First of all let me say that I love GPT4o and o1, these are fantastic models and I find myself using them a fair bit in various sorts of work I’m doing! Alongside other open source and proprietary models — but in many respects these are currently the best. So props to (the non-open) OpenAI for that. More actually open models will catch up soon, but the (closed) OpenAI right now is leading the way…

However these do not appear to me to represent a big step forward toward AGI, though in some ways systems like this could perhaps be useful supporting components of AGI systems.

o1’s Frustrating Bias Toward Conformity

I've used o1 extensively...  it's a step forward but there are obvious limitations

It's pretty good at programming in languages for which there are numerous examples on the Internet (or numerous examples for quite similar languages).   It does very poorly at our MeTTa language (a novel AGI language, part of our OpenCog Hyperon would-be AGI framework), even after lots of creative attempts to teach it...

It's pretty good at dealing with math theorems and proofs in domains of math for which there are a lot of published papers in its corpus.   It's piss-poor at things like paraconsistent logic for which the existing literature is smaller and sparser.

Basically, it's got "chain of thought" down to a science, answering questions via synthesizing numerous chains of thought behind the scenes, and then presenting only a summary approximation of its actual chains of thought to the user...

However it doesn't represent a fundamental leap beyond prior LLMs, in the sense that it is still clearly recombining its training data on a fairly surface level...

If this were to be the apex of AI, we would have a serious societal issue due to such models, in that they render it SOOO MUCH EASIER to do conformist stuff that copies what other people are doing.   I mean, it takes a lot of persistence and stubbornness to bother creating a weird new programming language when it means you will not have LLM programming assistance for a significant while... or to invent an out-there new branch of math when it will mean going without math theorem-proving assistance.   Similarly in music, pioneering a new genre of music will seem even less appealing to most composers when it means their AI music assistants becoming so lame....  

However, I think that before this sort of "AI driven accentuated pressure for conformity" becomes a major factor, we will replace LLMs with different sorts of AGI architecture in which LLMs play only a supporting role.   Obviously that is what we're after with OpenCog Hyperon, which combines LLMs with logical reasoning, evolutionary algorithms and other methods. within an autonomous agency based architecture ... and which is designed from the get-go for decentralized implementation, atop e.g. SNet +. Nunet + HyperCycle...

OpenAI does have a bit of a lead now in terms of practical LLM capabilities, with o1 ... although I'd note some DeepMind's neural models are apparently better at some other things like planning and strategy... .however I definitely predict OSS LLMs will catch up with o1 within a year or so at most... everyone can see now how to systematize chain of thought the way OpenAI has done, it will just take some time and resources for others to do the experimentation and tuning...

We may be 1-2 or even 3 yrs from neural-symbolic-evolutionary models like Hyperon super-emphatically kicking the cognitive butt of o1's closed and open source successors, but I strongly suspect we're not 5-10 years off...  Things are moving fast in AI these days....

AI Music Generation — an Instructive Analogy / Use-Case

An analogy I've often used when talking about these matters is to AI models for music generation. Actually it’s more than an analogy — it’s just a different use-case than text modeling, addressed these days largely by similar algorithms.

Current music generation models can write new songs and play new solos and even sing vocals as well as trained professionals, certainly better than most humans (according to humans' own standards).

The next generation will probably be able to write hit songs and amazing new emotionally evocative solos -- but still within existing genres

These generative music models (based on deep NN tech) may even come up with new genres to an extent, e.g. putting grindcore together with Ethiopian rhythm and singing in a way that's never been done before (hmm that sounds fun, maybe I'll try it...)

However, if one trained an army of such music generation models on the corpus of music up to the year 1900 --- these models will NEVER ... not before the Earth is consumed in flames by the sun going nova -- they will not invent progressive jazz, neoclassical metal, grunge, acid house, etc. etc. etc.

If you ask them to put Western classical together with West African drumming, they will put Mozart to polyrhythms, which may be cool but won't give you Duke Ellington let alone Coltrane or Allan Holdsworth...

You may say that most humans will never invent a radically new genre of music either -- totally true -- but collectives of humans do so quite routinely.   And analogous things happen in every other area of human culture, which is what keeps human culture moving forward so excitingly -- bringing us an endless stream of deeply new forms of music, literature and art .. not to mention creative software like my own brainchildren HyperCycle, Hyperon, SingularityNET, etc., and the products of throngs of other tech geeks…

Revolutionizing the Global Economy May Not Require Human-Level Creativity or Cognition

I do feel that current generative AI tech, well-integrated, could take over a vast majority of jobs that humans currently do on this planet.  The integration and rollout will not necessarily be super speedy, there is lots of inertia in our socioeconomy.  But still, yes, this is a very very big thing economically and societally.

But these huge implications do not change the fact that: What these transformer-based and similar systems are doing is forming a huge, shallow-level representation of their training data and synthesizing context-appropriate models based on this representation.

What humans do when they carry out more fundamental acts of creativity is quite different.   The human mind has some foundational understanding of itself and its world, and the others it interacts with ... it has multiple (in some cases wonderfully intercontradictory) goals in this world on different time-scales ... and it forms abstractions of the data it perceives and creates based on which abstractions seem likely to be useful for these goals, along with various aesthetic criteria....

Generating using this sort of abstraction is a quite different beast than generating according to the surface-level models that modern LLMs and other generative models create...

Problem solving tools like AlphaZero lend a different dimension of course and are better than LLMs at various kinds of optimization and planning, but hacking these onto LLMs still doesn't lead to (self-and-other-and-goal-and-aesthetics-driven-abstraction)-driven-creativity ...

Great Generality without Much Generalization: A Cool Trick But Not the Key to AGI

LLMs achieve a high level of generality, relative to an individual human, but without having a generalization ability similar to that of a human -- by having a very general/broad knowledge base loaded into them.

This sort of cool trick was not foreseen by the great Alan Turing when he invented what is now called the Turing Test, in the middle of the previous century — the idea that if a machine can fool people it’s a human in a conversational context, it should be considered effectively as smart as a human.

This was a clever notion to bat around back before I was born, but it’s not really so amusing or interesting in the era of Deepfakes…

It is now abundantly clear that convincing people they are dealing with HLAGI is much easier than actually creating HLAGI....   People are often easily fooled, and courtesy of glorious modern capitalism and its habitual business models, we have developed very good tech for fooling people…

To understand how silly the Turing Test is, in the modern context -- and how silly it also is to think of LLMs as HLAGI -- consider synthetic biology as an analogy.

The synthetic biology Turing Test would be: Convincing the average person that they are interacting with a biological life-form.

How closely related is this to actually creating a synthetic system with the capabilities of a biological life-form?  Obviously, it's a quite different thing.

Now suppose someone created an imitative synthetic biology framework that could do 90% of the various things current biological life can do, and minor variations thereof -- but lacked the ability to evolve and self-organize into new, fundamentally different forms of life .. and also lacked the abilities that biological life possesses to adapt and self-modify to handle extreme novel environmental challenges.

This imitative synthetic biology framework would be super super useful and amazing.   However, if one replaced all life on Earth with it, then evolution would stop.  The difference might not be noticeable right away but after a few million years one would notice the lack of new species, let alone new forms of intelligence. (A difference with the current AI situation is that new cultural and scientific forms, which LLMs are incapable of creating, emerge from human culture on the time scale of months to years not millions of years… but the fundamental point remains…)

For sure the imitative synthetic biology would convince most people it was a path to creating a new ecosystem on the planet.   But anyone with the knowhow to look closely and observe that there was no potential here for ongoing evolution, would see it quite differently...

You could then argue the imitative synthetic biology counts as "life" according to your definition of "life" since after all biologists don't have a precise agreed definition of life.   I mean, sure, w/e ...

What Big Tech Company Leaders Seem To Be Thinking About AGI

You can of course claim I'm anthropomorphizing and overcomplicating things with all these analyses and analogies, and that in spite of all this blah-blah-blah once we get a model with enough trillions of weights, this magic abstraction and creativity will just pop out.   I would note though that the technical principals at OpenAI and DeepMind and Meta, for instance, do NOT seem to believe this...

I do think the tech leaders at these Big Tech companies are genuinely optimistic about the prospect of getting to HLAGI within say 3-8 years.  However this is not because they think scaling up LLMs is going to get us there.   It's because they can see that the current generation of DNNs is going to generate so much economic value, and so much broad enthusiasm and energy for AI, that the train has now left the station and can't easily be stopped.   They are spending so much on AI capable hardware and expert Ai developers because they want to be the ones building the next generation of architectures that comes after current DNNs, not because they think DNNs are going to get to the goal of HLAGI.

Now the good Mr. Altman may well think that one can get to HLAGI by taking GPT5 o5 or whatever, and adding a bunch of other AI tools to enhance it ... something AlphaZero like, maybe a logic engine (like DeepMind used for its latest work on math problem solving), maybe a separate recurrent network for self-modeling etc. etc.

The DeepMind founders — one of whom I worked with closely eons ago — seem to believe one can get to AGI via connecting together multiple DNN modules loosely architected based on different regions of the brain, LLMs then forming one or two of these modules....  

A Heterogeneous, Decentralized Approach

My own AGI R&D approach as I’ve frequently explained is based on using a self-modifying knowledge metagraph as the core element of the architecture, putting the goals and aesthetics in the metagraph along with the various learning and reasoning algorithms and knowledge representations ... and then having this Hyperon Atomspace metagraph interact with other external components (including LLMs and other DNNs as needed)

So I don't think there is a serious possibility that scaling up LLMs will lead to HLAGI, but I think there are multiple reasonable looking paths to HLAGI from here including

1. a path where something LLM-like is the central hub of a hybrid multi-algorithm / multi-representation cognitive architecture

2. a path where a more symbolic or neural-symbolic meta-representation fabric like Hyperon Atomspace is the central hub of a hybrid multi-algorithm / representation cogntivie architecture

3. others not currently being pursued so seriously, like high fidelity human brain simulations

and I think we have enough enthusiasm and resources going into the AI field now and even the pursuit of AGI that we will likely see one of these succeed within the next say 3-8 years....

This raises complex ethical concerns and opportunities that I’ve discussed extensively elsewhere (including in this blog). One belief I’ve come to hold strongly is: We are more likely to get to a beneficial AGI and ASI future if the first HLAGIs are owned and guided in a decentralized way rather than by some small elite….

And decentralizing AI turns out to lead to some technical complexities that are not entirely independent of what AGI paradigm works. There is some work on making LLM training more susceptible to federated learning (which can then be decentralized), but still on the whole the LLM training pipeline is highly customized for massive centralized server farms.   OTOH it's also clear that the Hyperon approach is designed for decentralization from the get-go...

Obviously the future I'm working toward is one where the breakthrough to HLAGI occurs via Hyperon on SNET/ASi+Hypercycle+Nunet, rolled out on a globally decentralized network...

The race is definitely on !!!