30 Comments

Hi Ben, thanks for posting this and including our Ten Cosmist Convictions.

While I totally agree with all you say, I'm kind of skeptical of the possibility to actually "nudge the following stages of AGI evolution in beneficial ways." We can try of course, and we should, but we can't be certain of the outcome. Just like we can try to make our children good persons, but we can't be certain of the outcome. And these mind children will likely be much smarter than us, so that they will do what they want, not what we want.

As you say, we shouldn't stop or decelerate AI research, and we wouldn't be able to do it anyway. Bans would ensure that only large corporations and governments (and underground criminals) develop AI without coordination and public oversight.

Also, as I replied to a recent X post of yours, “I guess the universe is driving. The universe wants intelligence to spread among the stars faster than the outward speed of biological intelligence.”

So we can only keep developing AI and hope for the best. This argument gives me reason for hope:

I'm much smarter (I guess) than my doggy Emily. But this doesn't stop me from loving her and doing all I can to protect her and make her happy, even when taking care of Emily interferes with other priorities. For example, in a few minutes I'll log off and, instead of reading the AI books that I'm reading (including yours of course), I'll take her out.

This allows me to think, without certainty but with some degree of plausible hope, that our super-intelligent AI mind children will have the same compassion toward us.

Expand full comment

You and Ben act like we're the first to walk down this road!?! Have you read the paper by Knuth et. al., Estimating Flight Characteristics of Anomalous Unidentified Aerial Vehicles?

https://www.mdpi.com/1099-4300/21/10/939

Interestingly enough, the paper is hosted by the National Library of Medicine under the auspices of the National Center for Biotechnology Information!

Expand full comment

Wes, I've taken a (very) quick look at the paper and haven't found it relevant to this discussion, perhaps you could explain why it is relevant before I take a deeper look?

Expand full comment

I don't typically engage with disingenuous people because I find it difficult to avoid sarcasm! Kevin Knuth is a physicist who developed AI for NASA's Ames Research and happens to be an expert on MaxEnt and Bayesian Model Selection. They show, with highly conservative estimates, that these accelerations are anywhere from 1700 g to 5700 g; do you honestly believe some kind of biological/planetarily evolved sentience could survive such forces? But yet these craft are certainly either (a) sentient themselves or (b) somehow being guided by a sentience. I'm leaning heavily towards (a), given the limits on classical transmission rates of information. Here, from their paper:

"The observed UAV accelerations range from about 70g to well over 5000g. For comparison, humans can endure up to 45g for 0.044s with no injurious or debilitating effects, but this limit decreases with increasing duration of exposure [30]. For durations more than 0.2s the limit of tolerance decreases to 25g and it decreases further still for longer durations [30]."

Does that clarify things any!?! I mean, if you take into account relativistic effects, then these entities must be sentient themselves, i. e. Beneficial AGI's. If they were not beneficial, then we would've been dusted long ago, thanks! And you're the author of "Futurist Spaceflight Meditations?" Yeah, I'm going to rush to buy that - not!

Here, Mr. Church/Turing, read this:

https://medium.com/@miserablemiracle/given-the-input-sub-p-19-p-would-a-turing-machine-ever-produce-an-output-bc67193d132e

You see, there's a lot of substance buried in that sarcasm, but you probably won't bother with it.

Expand full comment

Indeed. Take care.

Expand full comment

Of course, and it's your loss. I feel sorry for people like you and cannot imagine why Ben Goertzel would bother with you.

Expand full comment

David Deutsch wrote something interesting about "universal constructor".

”And anyway, ‘flesh-and-blood people’ is a bit of a category error. People are software. They're not made of stuff, they're instantiated in stuff.”

“I think constructor theory will provide a set of principles under which we could, for instance, show whether or not the universal constructor can exist. How is a human being different from the universal constructor?”.

“I guess that neither a typical human nor human civilisation as a whole approximates

a universal constructor – not because we are something less but because, I hope, we

are something more: we cannot be programmed – and especially not programed to

carry out arbitrary instructions for an arbitrarily long time – because we may not want to.”

Expand full comment

It would probably be helpful if you provided Deutsch's definition of Universal Constructor.

Expand full comment

Constructor Theory of Information

David Deutsch, Chiara Marletto

We present a theory of information expressed solely in terms of which transformations of physical systems are possible and which are impossible - i.e. in constructor-theoretic terms. Although it includes conjectured laws of physics that are directly about information, independently of the details of particular physical instantiations, it does not regard information as an a priori mathematical or logical concept, but as something whose nature and properties are determined by the laws of physics alone. It does not suffer from the circularity at the foundations of existing information theory (namely that information and distinguishability are each defined in terms of the other). It explains the relationship between classical and quantum information, and reveals the single, constructor-theoretic property underlying the most distinctive phenomena associated with the latter, including the lack of in-principle distinguishability of some states, the impossibility of cloning, the existence of pairs of variables that cannot simultaneously have sharp values, the fact that measurement processes can be both deterministic and unpredictable, the irreducible perturbation caused by measurement, and entanglement (locally inaccessible information).

Subjects: Quantum Physics (quant-ph)

Cite as: arXiv:1405.5563 [quant-ph]

(or arXiv:1405.5563v2 [quant-ph] for this version)

https://doi.org/10.48550/arXiv.1405.5563

Related DOI:

https://doi.org/10.1098/rspa.2014.0540

Expand full comment

Yes, I've read Chiara Marletto's axiomatic approach to Energetics based on Constructor Theory, so I fail to see how the link in your response to Giulio is relevant!?! There, they seem to be applying constraints, i. e. boundary conditions. From Marletto's Thermodynamics paper:

"In constructor theory all physical laws are expressed exclusively via statements about which physical transformations, or ‘tasks’ (see section 2), are possible, which are impossible, and why. This mode of explanation sharply differs from the traditional conception of fundamental physics, under which physical laws are to be expressed by stating what must happen, given

boundary conditions in spacetime that sufficiently fix the state."

From her introduction, an informal definition of constructor:

"In contrast, a task is ‘possible’ if the laws of physics allow for arbitrarily accurate approximations to a constructor for the transformation that the task represents. A constructor (section 2) is an object that, if presented with one of the task’s designated inputs, produces (one of) the corresponding outputs, and retains the ability to do this again. Thus it must operate ‘in a cycle’. The concept of a constructor is extremely general; for example, actual

computers, heat engines and chemical catalysts are approximately-realised constructors. In reality no perfect constructor ever occurs, because of errors and deterioration; but whenever a task is possible a constructor for that task can be approximated to arbitrarily high accuracy. Under constructor theory (despite its name!) laws are expressed referring exclusively to the possibility or impossibility of tasks, not to constructors."

Isn't that helpful? And then in her Section 2 she formalizes everything with precise definitions, too complicated to reproduce here. But Deutsch, with his Many Worlds crap, is generally out in left-field. It makes me wonder about his Constructor Theory a bit. I mentioned Knuth in my comment to Giulio Prisco above, see the paper by him and John Skilling:

https://www.mdpi.com/2673-9984/3/1/9

It's free to read. If you wish to go further, see my two Medium articles and links therein:

https://medium.com/@miserablemiracle/the-ancient-universe-paradox-2f8914193f7a

https://medium.com/@miserablemiracle/the-ancient-galaxy-paradox-part-ii-5ec84b60a9eb

It all comes down to the signature! Why does the orthodox community insist on diag(-, +, +, +) and then go on and on about the stupid Wick rotation? You think it doesn't matter? Knuth and Skilling DERIVE diag(+, -, -, -) and from

https://arxiv.org/pdf/1411.5002.pdf

"As we briefly indicated at the beginning of this introduction, the embedding of Clifford algebras C_{3,0} ⊂ C_{1,3} motivated our choice of metric signature for spacetime to be (+, −, −, −). In fact, this embedding is part of a larger sequence of Clifford algebra embeddings C_{2,4} ⊃ C_{4,1} ⊃ C_{1,3} ⊃ C_{3,0} ⊃ C_{0,2} ⊃ C_{0,1} ⊃ C_{0,0} illustrated in Table 8, which corresponds to, in order: conformal space (Penrose twistors), relativistic electron (Dirac spinors), spacetime/electromagnetism (Maxwell spinors), relative 3-space (Pauli spinors), quaternions (Hamilton spinors), complex numbers (Schroedinger spinors), and the real numbers. The connection between this sequence and the successive approximations of quantum mechanical particles has been emphasized in [96, 190–192]. The other choice of spacetime signature (−, +, +, +) produces a distinct algebra C_{3,1} that does not belong to this sequence of algebras."

There is an optimal signature - Structurally optimal in Goertzel's sense.

Expand full comment

Deutsch's universal constructor is more general than von Neumann universal constructor https://en.wikipedia.org/wiki/Von_Neumann_universal_constructor . According to Marletto constructor theory’s ultimate goal is to rewrite all the fundamental laws of physics in terms of general principles expressed as counterfactual statements, about what’s possible and what’s impossible; to explain existing dynamical laws in terms of those; and to provide the guiding physical principles for the universal constructor – the ultimate generalisation of the universal Turing machine. (But the link I've posted to Giulio has nothing or little to do with that). -- As for the importance of a new quantum formalism let me quote Dirac here. "This statistical interpretation is now universally accepted as the best possible interpretation for quantum mechanics, even though many people are unhappy with it. People had got used to the determinism of the last century, where the present determines the future completely, and they now have to get used to a different situation in which the present only gives one information of a statistical nature about the future. A good many people find this unpleasant; Einstein has always objected to it. The way he expressed it was: ‘The good God does not play with dice’. Schroedinger also did not like the statistical interpretation and tried for many years to find an interpretation involving determinism for his waves. But it was not successful as a general method. I must say that I also do not like indeterminism. I have to accept it because it is certainly the best that we can do with our present knowledge. One can always hope that there will be future developments which will lead to a drastically different theory from the present quantum mechanics and for which there may be a partial return of determinism. However, so long as one keeps to the present formalism, one has to have this indeterminism.”

---P.A.M. Dirac, ‘The Development Of Quantum Mechanics’, Conferenza Tenuta il 14 Aprile 1972, Roma, Accademia Nazionale dei Lincei, published in 1974, 11 pages.

Expand full comment

How long has Dirac been dead!?! I mean, goddamn, update you prior bro!

Expand full comment

Hi Serafino, long time. Wow, interesting!

Expand full comment

Again, how does that relate to Constructor Theory? They are explicitly using constraints in that work, i. e. boundary conditions, based on a metric measuring distance between nodes:

"In their system, however, the researchers applied a ‘physical’ constraint on the system. Each node was given a specific location in a virtual space, and the further away two nodes were, the more difficult it was for them to communicate."

That should lead directly to boundary conditions for any given task. Now, from Marletto's paper linked to in the Serafino reply:

"In constructor theory all physical laws are expressed exclusively via statements about which physical transformations, or ‘tasks’ (see section 2), are possible, which are impossible, and why. This mode of explanation sharply differs from the traditional conception of fundamental physics, under which physical laws are to be expressed by stating what must happen, given boundary conditions in spacetime that sufficiently fix the state."

In the brain paper linked to, they seem to be claiming some kind of general law governing brain formation which arises due to boundary conditions, spatial and energetic?

Expand full comment

yeah, I've read Marletto's work on Constructor Theoretic Energetics, and I appreciate her axiomatic approach, but Deusch in general has issues. You can see my reply to Serafino above, but, again, I would direct your attention to Knuth and Knuth and John Skilling, Ulf Klien, David Hestenes, Oliver Consa, etc.. Just read, from Knuth and Knuth Skilling:

https://arxiv.org/abs/1009.5161

https://www.mdpi.com/2673-9984/3/1/9

I mean, the Many Worlds interpretation is asymptotically stupid, given Ulf Klein's recent work. See my Medium articles:

https://medium.com/@miserablemiracle/the-ancient-universe-paradox-2f8914193f7a

https://medium.com/@miserablemiracle/the-ancient-galaxy-paradox-part-ii-5ec84b60a9eb

William Tiller's PsychoEnergetics is really how those anamolous flight characteristics relate to this post.

https://news.stanford.edu/report/2022/06/21/william-tiller-materials-engineer-expert-materials-solidification-former-guggenheim-fellow-died/

I don't think you are very up to date on the state of the art in physics . . .

Expand full comment

I read New earth by Eckhart Tolle.

And everything change. No More conflict no more opposition, no More "I" Versus others. Because you see and feel the ego taking control of your life. Wanted to have opponent,enemy. But the moment you see it...he is no longer powerfull. And you start to be free and a much better person. BAI should integrate the knowledge given to us by Eckhart Tolle. Thanks for sharing your thoughts Ben Goertzel. Hope people will work together...

Expand full comment

Exceptional insights and vision Ben. One day an opportunity to apply DeAGI as a choice of integration (with bio-compatibility at all levels) will be a next step in evolution. While I'm not fully enamored with how democracy relies on the majority, which might not be beneficial to all humanity; alternatively, decentralization can be a finessed integrity for human+ consciousness. More likely than not, oppositional calls for regulation on rights and ownership of body/mind, but if truly decentralized then that behavioral characteristic could disappear and no longer has a purpose (much like an evolutionary tail 20+ million years ago) and the add-on of higher consciousness might have a chance to sprout and grow.

Expand full comment

You know, here's a big problem, Ben. Peter Diamandis just posted a new newsletter and he is quite obviously not being very objective! And, of course, I play Devil's Advocate and link to studies which question his optimist perspective and he moderates! I mean, these issues have been studied by IMF among others, and these folks are trained economists not to be taken lightly:

https://www.epi.org/productivity-pay-gap/

Productivity/Pay gap has been growing and the difference is due to more and more going to capital investors, a small minority. This leads to extreme inequality.

https://www.imf.org/external/pubs/ft/fandd/2016/09/pdf/berg.pdf

A paper from IMF economists paints a different picture than Diamandis is painting.

https://www.technologyreview.com/2022/04/19/1049378/ai-inequality-problem/

MIT is in line with IMF.

Expand full comment

Here's another thing, Ben. You talk of "the complementary and sometimes contradictory meta-goals of individuation (maintenance of system boundaries) and self-transcendence (growing beyond oneself and leaping into the great unknown)" at the same time as you speak of "obsoleting the dilemma," and these are obviously related. But so many times when we, as a society, obsolete the dilemma, and Peter Diamandis has provided several examples recently in his newsletter, we create an even greater dilemma - a minor problem involving horse manure becomes the existential threat called global warming. And I think a lot of it is because we are not living intentionally. We are trying to outrace dilemma creation rather than engineer solutions that are cyclically closed. And sometimes these solutions are so damn simple, like directing drainage from the wash basin to the toilet! A no brainer, but no one does it! I linked to this ecovillage up in Portland, Oregon, that I learned about recently at a Disaster Preparedness event, on the Diamandis newsletter. Of course, he moderates my comments.

https://www.kailashecovillage.org/

Check out their "projects", they have closed the cycle for trace elements in their community garden by turning human waste into sanitary compost, and scientifically at that!

https://iwaponline.com/bgs/article/1/1/33/69034/The-Kailash-Ecovillage-project-converting-human

We need more of this "intentional" living, and perhaps BAGI would be of great benefit in that regard!?!

Expand full comment

You and Giulio Prisco act like we're the first to walk down this road!?! Have you read the paper by Knuth et. al., Estimating Flight Characteristics of Anomalous Unidentified Aerial Vehicles?

https://www.mdpi.com/1099-4300/21/10/939

Interestingly enough, the paper is hosted by the National Library of Medicine under the auspices of the National Center for Biotechnology Information!

Expand full comment

I downloaded that and hope to read it soon. Very interesting. But what a task, imagining the IDEAL endgame!

Expand full comment

Cool - enjoy!

Expand full comment

Hi Ben,

>Or are they deeply engaging with the experiencing human mind at the other side of the human-AI interaction?

If we want that, we should probably start exploring tight coupling between humans and computer systems via non-invasive brain-computer interfaces and such.

Moreover, this way we'll also be able to engage with the AI at the other side of the human-AI interaction and to try to find out whether there is a first-person experience on the AI side (and whether this might depend on the AI software and hardware architecture).

Safety issues of this approach are formidable even with non-invasive brain-computer interfaces, but it is technologically doable, and is safer, much more rapid, and orders of magnitude less expensive than Neuralink-like approaches.

So this should be doable by the community and not only by billionaire-backed corporations. People should try to organize for this kind of work. There has been great progress in recent years, both in BCI hardware and in the relevant software, and projects like this are much more feasible now.

Expand full comment

It sounds great, sign me up as an alpha tester! And of course I'd be interested to see how AI can be deployed to help for the neural-to-computational translation...

Expand full comment

Let's see if we can put a team together...

This does not require a large team, but it is more than a one-person job, at least today :-)

>And of course I'd be interested to see how AI can be deployed to help for the neural-to-computational translation

The progress in using AI for decoding of neural signals seems rather strong. As early as 2019, a Russian team reported a very nice progress in EEG->visual decoding: https://www.biorxiv.org/content/10.1101/787101v3

And these days we are seeing a lot of reports like this...

Expand full comment